2024-05-11T07:54:59.358 INFO:root:teuthology version: 0.0.1.dev271+g1ae7ad8 2024-05-11T07:54:59.359 DEBUG:teuthology.run:Teuthology command: teuthology --description fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/backtrace} --archive /home/teuthworker/archive/rishabh-2024-05-11_05:57:22-fs-wip-rishabh-testing-20240510.053205-testing-default-smithi/7702178 --verbose --owner scheduled_rishabh@teuthology --name rishabh-2024-05-11_05:57:22-fs-wip-rishabh-testing-20240510.053205-testing-default-smithi -- /home/teuthworker/archive/rishabh-2024-05-11_05:57:22-fs-wip-rishabh-testing-20240510.053205-testing-default-smithi/7702178/orig.config.yaml 2024-05-11T07:54:59.404 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-11T07:54:59.508 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2024-05-11_05:57:22-fs-wip-rishabh-testing-20240510.053205-testing-default-smithi/7702178 branch: wip-rishabh-testing-20240510.053205 description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/backtrace} email: null first_in_suite: false job_id: '7702178' kernel: client: sha1: distro flavor: default kdb: true sha1: fb9a6a44107e0b9f9ca2d9a78f2308f5dd49911e ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: rishabh-2024-05-11_05:57:22-fs-wip-rishabh-testing-20240510.053205-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-rishabh-testing-20240510.053205 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy sha1: 308506802b0bdea4df7050cfc1fcbdc04fea6a6e ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 308506802b0bdea4df7050cfc1fcbdc04fea6a6e kclient: syntax: v1 selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 subvolume_version: 2 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-rishabh-testing-20240510.053205 sha1: 308506802b0bdea4df7050cfc1fcbdc04fea6a6e owner: scheduled_rishabh@teuthology priority: 75 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 4085 sha1: 308506802b0bdea4df7050cfc1fcbdc04fea6a6e sleep_before_teardown: 0 subset: 100/512 suite: fs suite_branch: wip-rishabh-testing-20240510.053205 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_308506802b0bdea4df7050cfc1fcbdc04fea6a6e/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 308506802b0bdea4df7050cfc1fcbdc04fea6a6e syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi167.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBK0JOctBAw4YxRwVXlCl/LZiTxFjn9XYNRMgJBHFDKxCGhC/FlEfWF9jjJ1rw+Eky8U41KhvsOQw5nWR/Aw7zyc= smithi204.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPfxZSuUgNSMOyo/zhhjozhvdbiffHNZPmOu9r4pL2pJdOF/GmDg947fBKRdCppJba0CuOJjGYyUrua3hf+tOgE= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - sequential: - mgrmodules - kclient: null - cephfs_test_runner: modules: - tasks.cephfs.test_backtrace teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 1ae7ad82388e92a475afff437d49054826c019a1 timestamp: 2024-05-11_05:57:22 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-05-11T07:54:59.508 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_308506802b0bdea4df7050cfc1fcbdc04fea6a6e/qa; will attempt to use it 2024-05-11T07:54:59.509 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_308506802b0bdea4df7050cfc1fcbdc04fea6a6e/qa/tasks 2024-05-11T07:54:59.509 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-05-11T07:54:59.511 INFO:teuthology.task.internal:Checking packages... 2024-05-11T07:54:59.534 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '308506802b0bdea4df7050cfc1fcbdc04fea6a6e' 2024-05-11T07:54:59.534 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-11T07:54:59.534 INFO:teuthology.packaging:ref: None 2024-05-11T07:54:59.534 INFO:teuthology.packaging:tag: None 2024-05-11T07:54:59.534 INFO:teuthology.packaging:branch: wip-rishabh-testing-20240510.053205 2024-05-11T07:54:59.534 INFO:teuthology.packaging:sha1: 308506802b0bdea4df7050cfc1fcbdc04fea6a6e 2024-05-11T07:54:59.534 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-rishabh-testing-20240510.053205 2024-05-11T07:54:59.751 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3601.g30850680 2024-05-11T07:54:59.752 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-05-11T07:54:59.759 INFO:teuthology.task.internal:no buildpackages task found 2024-05-11T07:54:59.759 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-05-11T07:54:59.794 INFO:teuthology.task.internal:Saving configuration 2024-05-11T07:54:59.812 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-05-11T07:54:59.824 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-11T07:54:59.843 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi167.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-05-11_05:57:22-fs-wip-rishabh-testing-20240510.053205-testing-default-smithi/7702178', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-11 07:49:42.522507', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBK0JOctBAw4YxRwVXlCl/LZiTxFjn9XYNRMgJBHFDKxCGhC/FlEfWF9jjJ1rw+Eky8U41KhvsOQw5nWR/Aw7zyc='} 2024-05-11T07:54:59.859 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi204.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-05-11_05:57:22-fs-wip-rishabh-testing-20240510.053205-testing-default-smithi/7702178', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-11 07:49:42.524349', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPfxZSuUgNSMOyo/zhhjozhvdbiffHNZPmOu9r4pL2pJdOF/GmDg947fBKRdCppJba0CuOJjGYyUrua3hf+tOgE='} 2024-05-11T07:54:59.859 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-05-11T07:54:59.865 INFO:teuthology.task.internal:roles: ubuntu@smithi167.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-05-11T07:54:59.865 INFO:teuthology.task.internal:roles: ubuntu@smithi204.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-05-11T07:54:59.865 INFO:teuthology.run_tasks:Running task console_log... 2024-05-11T07:54:59.990 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fa67e633160>, signals=[15]) 2024-05-11T07:54:59.991 INFO:teuthology.run_tasks:Running task internal.connect... 2024-05-11T07:54:59.997 INFO:teuthology.task.internal:Opening connections... 2024-05-11T07:54:59.997 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi167.front.sepia.ceph.com 2024-05-11T07:54:59.999 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-11T07:55:00.074 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi204.front.sepia.ceph.com 2024-05-11T07:55:00.075 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi204.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-11T07:55:00.147 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-05-11T07:55:00.153 DEBUG:teuthology.orchestra.run.smithi167:> uname -m 2024-05-11T07:55:00.171 INFO:teuthology.orchestra.run.smithi167.stdout:x86_64 2024-05-11T07:55:00.171 DEBUG:teuthology.orchestra.run.smithi167:> cat /etc/os-release 2024-05-11T07:55:00.228 INFO:teuthology.orchestra.run.smithi167.stdout:NAME="CentOS Stream" 2024-05-11T07:55:00.228 INFO:teuthology.orchestra.run.smithi167.stdout:VERSION="9" 2024-05-11T07:55:00.228 INFO:teuthology.orchestra.run.smithi167.stdout:ID="centos" 2024-05-11T07:55:00.229 INFO:teuthology.orchestra.run.smithi167.stdout:ID_LIKE="rhel fedora" 2024-05-11T07:55:00.229 INFO:teuthology.orchestra.run.smithi167.stdout:VERSION_ID="9" 2024-05-11T07:55:00.229 INFO:teuthology.orchestra.run.smithi167.stdout:PLATFORM_ID="platform:el9" 2024-05-11T07:55:00.229 INFO:teuthology.orchestra.run.smithi167.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-11T07:55:00.229 INFO:teuthology.orchestra.run.smithi167.stdout:ANSI_COLOR="0;31" 2024-05-11T07:55:00.229 INFO:teuthology.orchestra.run.smithi167.stdout:LOGO="fedora-logo-icon" 2024-05-11T07:55:00.229 INFO:teuthology.orchestra.run.smithi167.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-11T07:55:00.229 INFO:teuthology.orchestra.run.smithi167.stdout:HOME_URL="https://centos.org/" 2024-05-11T07:55:00.229 INFO:teuthology.orchestra.run.smithi167.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-11T07:55:00.229 INFO:teuthology.orchestra.run.smithi167.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-11T07:55:00.229 INFO:teuthology.orchestra.run.smithi167.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-11T07:55:00.230 INFO:teuthology.lock.ops:Updating smithi167.front.sepia.ceph.com on lock server 2024-05-11T07:55:00.253 DEBUG:teuthology.orchestra.run.smithi204:> uname -m 2024-05-11T07:55:00.270 INFO:teuthology.orchestra.run.smithi204.stdout:x86_64 2024-05-11T07:55:00.270 DEBUG:teuthology.orchestra.run.smithi204:> cat /etc/os-release 2024-05-11T07:55:00.328 INFO:teuthology.orchestra.run.smithi204.stdout:NAME="CentOS Stream" 2024-05-11T07:55:00.328 INFO:teuthology.orchestra.run.smithi204.stdout:VERSION="9" 2024-05-11T07:55:00.328 INFO:teuthology.orchestra.run.smithi204.stdout:ID="centos" 2024-05-11T07:55:00.328 INFO:teuthology.orchestra.run.smithi204.stdout:ID_LIKE="rhel fedora" 2024-05-11T07:55:00.328 INFO:teuthology.orchestra.run.smithi204.stdout:VERSION_ID="9" 2024-05-11T07:55:00.328 INFO:teuthology.orchestra.run.smithi204.stdout:PLATFORM_ID="platform:el9" 2024-05-11T07:55:00.328 INFO:teuthology.orchestra.run.smithi204.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-11T07:55:00.328 INFO:teuthology.orchestra.run.smithi204.stdout:ANSI_COLOR="0;31" 2024-05-11T07:55:00.328 INFO:teuthology.orchestra.run.smithi204.stdout:LOGO="fedora-logo-icon" 2024-05-11T07:55:00.329 INFO:teuthology.orchestra.run.smithi204.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-11T07:55:00.329 INFO:teuthology.orchestra.run.smithi204.stdout:HOME_URL="https://centos.org/" 2024-05-11T07:55:00.329 INFO:teuthology.orchestra.run.smithi204.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-11T07:55:00.329 INFO:teuthology.orchestra.run.smithi204.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-11T07:55:00.329 INFO:teuthology.orchestra.run.smithi204.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-11T07:55:00.329 INFO:teuthology.lock.ops:Updating smithi204.front.sepia.ceph.com on lock server 2024-05-11T07:55:00.344 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-05-11T07:55:00.353 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-05-11T07:55:00.385 INFO:teuthology.task.internal:Checking for old test directory... 2024-05-11T07:55:00.385 DEBUG:teuthology.orchestra.run.smithi167:> test '!' -e /home/ubuntu/cephtest 2024-05-11T07:55:00.388 DEBUG:teuthology.orchestra.run.smithi204:> test '!' -e /home/ubuntu/cephtest 2024-05-11T07:55:00.406 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-05-11T07:55:00.412 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-05-11T07:55:00.412 DEBUG:teuthology.orchestra.run.smithi167:> test -z $(ls -A /var/lib/ceph) 2024-05-11T07:55:00.446 DEBUG:teuthology.orchestra.run.smithi204:> test -z $(ls -A /var/lib/ceph) 2024-05-11T07:55:00.553 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-05-11T07:55:00.609 INFO:teuthology.run_tasks:Running task kernel... 2024-05-11T07:55:00.624 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'flavor': 'default', 'kdb': True, 'sha1': 'fb9a6a44107e0b9f9ca2d9a78f2308f5dd49911e'} 2024-05-11T07:55:00.624 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}, 'client.2': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}, 'client.3': {'sha1': 'distro'}} 2024-05-11T07:55:00.624 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}}, timeout 300 2024-05-11T07:55:00.624 DEBUG:teuthology.orchestra.run.smithi167:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-11T07:55:00.625 DEBUG:teuthology.orchestra.run.smithi204:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-11T07:55:00.641 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-11T07:55:00.641 DEBUG:teuthology.orchestra.run.smithi167:> uname -r 2024-05-11T07:55:00.642 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-11T07:55:00.642 DEBUG:teuthology.orchestra.run.smithi204:> uname -r 2024-05-11T07:55:00.698 INFO:teuthology.orchestra.run.smithi167.stdout:5.14.0-437.el9.x86_64 2024-05-11T07:55:00.698 INFO:teuthology.task.kernel:Running kernel on smithi167: 5.14.0-437.el9.x86_64 2024-05-11T07:55:00.698 DEBUG:teuthology.orchestra.run.smithi167:> sudo yum install -y kernel 2024-05-11T07:55:00.700 INFO:teuthology.orchestra.run.smithi204.stdout:5.14.0-437.el9.x86_64 2024-05-11T07:55:00.701 INFO:teuthology.task.kernel:Running kernel on smithi204: 5.14.0-437.el9.x86_64 2024-05-11T07:55:00.701 DEBUG:teuthology.orchestra.run.smithi204:> sudo yum install -y kernel 2024-05-11T07:55:03.055 INFO:teuthology.orchestra.run.smithi167.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.1 MB 00:00 2024-05-11T07:55:03.838 INFO:teuthology.orchestra.run.smithi204.stdout:CentOS Stream 9 - BaseOS 5.2 MB/s | 8.1 MB 00:01 2024-05-11T07:55:05.573 INFO:teuthology.orchestra.run.smithi204.stdout:CentOS Stream 9 - AppStream 30 MB/s | 19 MB 00:00 2024-05-11T07:55:06.230 INFO:teuthology.orchestra.run.smithi167.stdout:CentOS Stream 9 - AppStream 9.1 MB/s | 19 MB 00:02 2024-05-11T07:55:11.232 INFO:teuthology.orchestra.run.smithi204.stdout:CentOS Stream 9 - CRB 3.0 MB/s | 6.2 MB 00:02 2024-05-11T07:55:11.680 INFO:teuthology.orchestra.run.smithi167.stdout:CentOS Stream 9 - CRB 3.5 MB/s | 6.2 MB 00:01 2024-05-11T07:55:12.866 INFO:teuthology.orchestra.run.smithi204.stdout:CentOS Stream 9 - Extras packages 37 kB/s | 16 kB 00:00 2024-05-11T07:55:13.323 INFO:teuthology.orchestra.run.smithi167.stdout:CentOS Stream 9 - Extras packages 54 kB/s | 16 kB 00:00 2024-05-11T07:55:14.402 INFO:teuthology.orchestra.run.smithi167.stdout:Extra Packages for Enterprise Linux 22 MB/s | 21 MB 00:00 2024-05-11T07:55:15.292 INFO:teuthology.orchestra.run.smithi204.stdout:Extra Packages for Enterprise Linux 9.2 MB/s | 21 MB 00:02 2024-05-11T07:55:19.475 INFO:teuthology.orchestra.run.smithi167.stdout:lab-extras 48 kB/s | 1.7 kB 00:00 2024-05-11T07:55:20.509 INFO:teuthology.orchestra.run.smithi204.stdout:lab-extras 17 kB/s | 1.7 kB 00:00 2024-05-11T07:55:21.497 INFO:teuthology.orchestra.run.smithi167.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-05-11T07:55:21.498 INFO:teuthology.orchestra.run.smithi167.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-05-11T07:55:21.587 INFO:teuthology.orchestra.run.smithi167.stdout:Dependencies resolved. 2024-05-11T07:55:21.596 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2024-05-11T07:55:21.596 INFO:teuthology.orchestra.run.smithi167.stdout: Package Architecture Version Repository Size 2024-05-11T07:55:21.596 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2024-05-11T07:55:21.596 INFO:teuthology.orchestra.run.smithi167.stdout:Installing: 2024-05-11T07:55:21.596 INFO:teuthology.orchestra.run.smithi167.stdout: kernel x86_64 5.14.0-444.el9 baseos 484 k 2024-05-11T07:55:21.596 INFO:teuthology.orchestra.run.smithi167.stdout:Installing dependencies: 2024-05-11T07:55:21.596 INFO:teuthology.orchestra.run.smithi167.stdout: kernel-core x86_64 5.14.0-444.el9 baseos 15 M 2024-05-11T07:55:21.596 INFO:teuthology.orchestra.run.smithi167.stdout: kernel-modules x86_64 5.14.0-444.el9 baseos 34 M 2024-05-11T07:55:21.596 INFO:teuthology.orchestra.run.smithi167.stdout: kernel-modules-core x86_64 5.14.0-444.el9 baseos 28 M 2024-05-11T07:55:21.596 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T07:55:21.596 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction Summary 2024-05-11T07:55:21.596 INFO:teuthology.orchestra.run.smithi167.stdout:================================================================================ 2024-05-11T07:55:21.597 INFO:teuthology.orchestra.run.smithi167.stdout:Install 4 Packages 2024-05-11T07:55:21.597 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T07:55:21.597 INFO:teuthology.orchestra.run.smithi167.stdout:Total download size: 78 M 2024-05-11T07:55:21.597 INFO:teuthology.orchestra.run.smithi167.stdout:Installed size: 123 M 2024-05-11T07:55:21.597 INFO:teuthology.orchestra.run.smithi167.stdout:Downloading Packages: 2024-05-11T07:55:21.801 INFO:teuthology.orchestra.run.smithi167.stdout:(1/4): kernel-5.14.0-444.el9.x86_64.rpm 3.0 MB/s | 484 kB 00:00 2024-05-11T07:55:22.102 INFO:teuthology.orchestra.run.smithi167.stdout:(2/4): kernel-core-5.14.0-444.el9.x86_64.rpm 34 MB/s | 15 MB 00:00 2024-05-11T07:55:22.510 INFO:teuthology.orchestra.run.smithi167.stdout:(3/4): kernel-modules-5.14.0-444.el9.x86_64.rpm 39 MB/s | 34 MB 00:00 2024-05-11T07:55:22.523 INFO:teuthology.orchestra.run.smithi204.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-05-11T07:55:22.523 INFO:teuthology.orchestra.run.smithi204.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-05-11T07:55:22.622 INFO:teuthology.orchestra.run.smithi204.stdout:Dependencies resolved. 2024-05-11T07:55:22.628 INFO:teuthology.orchestra.run.smithi204.stdout:================================================================================ 2024-05-11T07:55:22.628 INFO:teuthology.orchestra.run.smithi204.stdout: Package Architecture Version Repository Size 2024-05-11T07:55:22.628 INFO:teuthology.orchestra.run.smithi204.stdout:================================================================================ 2024-05-11T07:55:22.628 INFO:teuthology.orchestra.run.smithi204.stdout:Installing: 2024-05-11T07:55:22.628 INFO:teuthology.orchestra.run.smithi204.stdout: kernel x86_64 5.14.0-444.el9 baseos 484 k 2024-05-11T07:55:22.628 INFO:teuthology.orchestra.run.smithi204.stdout:Installing dependencies: 2024-05-11T07:55:22.628 INFO:teuthology.orchestra.run.smithi204.stdout: kernel-core x86_64 5.14.0-444.el9 baseos 15 M 2024-05-11T07:55:22.628 INFO:teuthology.orchestra.run.smithi204.stdout: kernel-modules x86_64 5.14.0-444.el9 baseos 34 M 2024-05-11T07:55:22.628 INFO:teuthology.orchestra.run.smithi204.stdout: kernel-modules-core x86_64 5.14.0-444.el9 baseos 28 M 2024-05-11T07:55:22.629 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T07:55:22.629 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction Summary 2024-05-11T07:55:22.629 INFO:teuthology.orchestra.run.smithi204.stdout:================================================================================ 2024-05-11T07:55:22.629 INFO:teuthology.orchestra.run.smithi204.stdout:Install 4 Packages 2024-05-11T07:55:22.629 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T07:55:22.629 INFO:teuthology.orchestra.run.smithi204.stdout:Total download size: 78 M 2024-05-11T07:55:22.629 INFO:teuthology.orchestra.run.smithi204.stdout:Installed size: 123 M 2024-05-11T07:55:22.630 INFO:teuthology.orchestra.run.smithi204.stdout:Downloading Packages: 2024-05-11T07:55:22.769 INFO:teuthology.orchestra.run.smithi167.stdout:(4/4): kernel-modules-core-5.14.0-444.el9.x86_6 29 MB/s | 28 MB 00:00 2024-05-11T07:55:22.769 INFO:teuthology.orchestra.run.smithi167.stdout:-------------------------------------------------------------------------------- 2024-05-11T07:55:22.769 INFO:teuthology.orchestra.run.smithi167.stdout:Total 66 MB/s | 78 MB 00:01 2024-05-11T07:55:22.969 INFO:teuthology.orchestra.run.smithi204.stdout:(1/4): kernel-5.14.0-444.el9.x86_64.rpm 2.4 MB/s | 484 kB 00:00 2024-05-11T07:55:23.454 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction check 2024-05-11T07:55:23.486 INFO:teuthology.orchestra.run.smithi204.stdout:(2/4): kernel-core-5.14.0-444.el9.x86_64.rpm 22 MB/s | 15 MB 00:00 2024-05-11T07:55:23.806 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction check succeeded. 2024-05-11T07:55:23.807 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction test 2024-05-11T07:55:24.044 INFO:teuthology.orchestra.run.smithi204.stdout:(3/4): kernel-modules-5.14.0-444.el9.x86_64.rpm 27 MB/s | 34 MB 00:01 2024-05-11T07:55:24.478 INFO:teuthology.orchestra.run.smithi204.stdout:(4/4): kernel-modules-core-5.14.0-444.el9.x86_6 19 MB/s | 28 MB 00:01 2024-05-11T07:55:24.478 INFO:teuthology.orchestra.run.smithi204.stdout:-------------------------------------------------------------------------------- 2024-05-11T07:55:24.478 INFO:teuthology.orchestra.run.smithi204.stdout:Total 42 MB/s | 78 MB 00:01 2024-05-11T07:55:24.739 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction test succeeded. 2024-05-11T07:55:24.739 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction 2024-05-11T07:55:25.158 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction check 2024-05-11T07:55:25.514 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction check succeeded. 2024-05-11T07:55:25.515 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction test 2024-05-11T07:55:26.457 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction test succeeded. 2024-05-11T07:55:26.457 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction 2024-05-11T07:55:27.174 INFO:teuthology.orchestra.run.smithi167.stdout: Preparing : 1/1 2024-05-11T07:55:27.878 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : kernel-modules-core-5.14.0-444.el9.x86_64 1/4 2024-05-11T07:55:27.970 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-11T07:55:28.877 INFO:teuthology.orchestra.run.smithi204.stdout: Preparing : 1/1 2024-05-11T07:55:29.533 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : kernel-modules-core-5.14.0-444.el9.x86_64 1/4 2024-05-11T07:55:29.612 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-11T07:55:30.662 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-11T07:55:30.801 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-11T07:55:32.293 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-11T07:55:32.436 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-11T07:55:35.512 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-11T07:55:35.637 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : kernel-5.14.0-444.el9.x86_64 4/4 2024-05-11T07:55:37.158 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-11T07:55:37.213 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : kernel-5.14.0-444.el9.x86_64 4/4 2024-05-11T07:55:40.394 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: kernel-modules-core-5.14.0-444.el9.x86_64 4/4 2024-05-11T07:55:41.949 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: kernel-modules-core-5.14.0-444.el9.x86_64 4/4 2024-05-11T07:56:09.947 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: kernel-core-5.14.0-444.el9.x86_64 4/4 2024-05-11T07:56:09.985 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: kernel-modules-5.14.0-444.el9.x86_64 4/4 2024-05-11T07:56:10.260 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: kernel-5.14.0-444.el9.x86_64 4/4 2024-05-11T07:56:10.260 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : kernel-5.14.0-444.el9.x86_64 1/4 2024-05-11T07:56:10.260 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-11T07:56:10.260 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-11T07:56:11.073 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : kernel-modules-core-5.14.0-444.el9.x86_64 4/4 2024-05-11T07:56:11.073 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T07:56:11.073 INFO:teuthology.orchestra.run.smithi167.stdout:Installed: 2024-05-11T07:56:11.073 INFO:teuthology.orchestra.run.smithi167.stdout: kernel-5.14.0-444.el9.x86_64 2024-05-11T07:56:11.074 INFO:teuthology.orchestra.run.smithi167.stdout: kernel-core-5.14.0-444.el9.x86_64 2024-05-11T07:56:11.074 INFO:teuthology.orchestra.run.smithi167.stdout: kernel-modules-5.14.0-444.el9.x86_64 2024-05-11T07:56:11.074 INFO:teuthology.orchestra.run.smithi167.stdout: kernel-modules-core-5.14.0-444.el9.x86_64 2024-05-11T07:56:11.074 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T07:56:11.074 INFO:teuthology.orchestra.run.smithi167.stdout:Complete! 2024-05-11T07:56:11.406 DEBUG:teuthology.orchestra.run.smithi167:> rpm -q kernel | sort -rV | head -n 1 2024-05-11T07:56:11.482 INFO:teuthology.orchestra.run.smithi167.stdout:kernel-5.14.0-444.el9.x86_64 2024-05-11T07:56:11.483 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-444.el9.x86_64 2024-05-11T07:56:11.484 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-437.el9.x86_64 Expected: 5.14.0-444.el9.x86_64 2024-05-11T07:56:11.484 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-05-11T07:56:11.484 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-05-11T07:56:11.484 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-05-11T07:56:11.484 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi167.front.sepia.ceph.com, path=None, version=distro) 2024-05-11T07:56:11.484 DEBUG:teuthology.orchestra.run.smithi167:> rpm -q kernel | sort -rV | head -n 1 2024-05-11T07:56:11.534 INFO:teuthology.orchestra.run.smithi167.stdout:kernel-5.14.0-444.el9.x86_64 2024-05-11T07:56:11.535 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-444.el9.x86_64 2024-05-11T07:56:11.535 DEBUG:teuthology.orchestra.run.smithi167:> sudo rpm -qi grub2-tools 2024-05-11T07:56:11.565 INFO:teuthology.orchestra.run.smithi167.stdout:Name : grub2-tools 2024-05-11T07:56:11.565 INFO:teuthology.orchestra.run.smithi167.stdout:Epoch : 1 2024-05-11T07:56:11.565 INFO:teuthology.orchestra.run.smithi167.stdout:Version : 2.06 2024-05-11T07:56:11.565 INFO:teuthology.orchestra.run.smithi167.stdout:Release : 68.el9 2024-05-11T07:56:11.566 INFO:teuthology.orchestra.run.smithi167.stdout:Architecture: x86_64 2024-05-11T07:56:11.566 INFO:teuthology.orchestra.run.smithi167.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-11T07:56:11.566 INFO:teuthology.orchestra.run.smithi167.stdout:Group : Unspecified 2024-05-11T07:56:11.566 INFO:teuthology.orchestra.run.smithi167.stdout:Size : 8264254 2024-05-11T07:56:11.566 INFO:teuthology.orchestra.run.smithi167.stdout:License : GPLv3+ 2024-05-11T07:56:11.566 INFO:teuthology.orchestra.run.smithi167.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-11T07:56:11.566 INFO:teuthology.orchestra.run.smithi167.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-11T07:56:11.566 INFO:teuthology.orchestra.run.smithi167.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-11T07:56:11.566 INFO:teuthology.orchestra.run.smithi167.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-11T07:56:11.566 INFO:teuthology.orchestra.run.smithi167.stdout:Packager : builder@centos.org 2024-05-11T07:56:11.566 INFO:teuthology.orchestra.run.smithi167.stdout:Vendor : CentOS 2024-05-11T07:56:11.566 INFO:teuthology.orchestra.run.smithi167.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-11T07:56:11.566 INFO:teuthology.orchestra.run.smithi167.stdout:Summary : Support tools for GRUB. 2024-05-11T07:56:11.566 INFO:teuthology.orchestra.run.smithi167.stdout:Description : 2024-05-11T07:56:11.567 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T07:56:11.567 INFO:teuthology.orchestra.run.smithi167.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-11T07:56:11.567 INFO:teuthology.orchestra.run.smithi167.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-11T07:56:11.567 INFO:teuthology.orchestra.run.smithi167.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-11T07:56:11.567 INFO:teuthology.orchestra.run.smithi167.stdout:hardware devices. 2024-05-11T07:56:11.567 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T07:56:11.567 INFO:teuthology.orchestra.run.smithi167.stdout:This subpackage provides tools for support of all platforms. 2024-05-11T07:56:11.568 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-11T07:56:11.568 INFO:teuthology.task.kernel:Updating grub on smithi167 to boot 5.14.0-444.el9.x86_64 2024-05-11T07:56:11.568 DEBUG:teuthology.orchestra.run.smithi167:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-11T07:56:11.573 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: kernel-core-5.14.0-444.el9.x86_64 4/4 2024-05-11T07:56:11.611 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: kernel-modules-5.14.0-444.el9.x86_64 4/4 2024-05-11T07:56:11.959 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: kernel-5.14.0-444.el9.x86_64 4/4 2024-05-11T07:56:11.959 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : kernel-5.14.0-444.el9.x86_64 1/4 2024-05-11T07:56:11.959 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-11T07:56:11.959 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-11T07:56:12.250 INFO:teuthology.orchestra.run.smithi167.stderr:Generating grub configuration file ... 2024-05-11T07:56:12.751 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : kernel-modules-core-5.14.0-444.el9.x86_64 4/4 2024-05-11T07:56:12.751 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T07:56:12.751 INFO:teuthology.orchestra.run.smithi204.stdout:Installed: 2024-05-11T07:56:12.751 INFO:teuthology.orchestra.run.smithi204.stdout: kernel-5.14.0-444.el9.x86_64 2024-05-11T07:56:12.751 INFO:teuthology.orchestra.run.smithi204.stdout: kernel-core-5.14.0-444.el9.x86_64 2024-05-11T07:56:12.751 INFO:teuthology.orchestra.run.smithi204.stdout: kernel-modules-5.14.0-444.el9.x86_64 2024-05-11T07:56:12.751 INFO:teuthology.orchestra.run.smithi204.stdout: kernel-modules-core-5.14.0-444.el9.x86_64 2024-05-11T07:56:12.751 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T07:56:12.751 INFO:teuthology.orchestra.run.smithi204.stdout:Complete! 2024-05-11T07:56:13.093 DEBUG:teuthology.orchestra.run.smithi204:> rpm -q kernel | sort -rV | head -n 1 2024-05-11T07:56:13.177 INFO:teuthology.orchestra.run.smithi204.stdout:kernel-5.14.0-444.el9.x86_64 2024-05-11T07:56:13.178 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-444.el9.x86_64 2024-05-11T07:56:13.178 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-437.el9.x86_64 Expected: 5.14.0-444.el9.x86_64 2024-05-11T07:56:13.178 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-05-11T07:56:13.178 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-05-11T07:56:13.178 INFO:teuthology.task.kernel:Installing distro kernel on client.1... 2024-05-11T07:56:13.178 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi204.front.sepia.ceph.com, path=None, version=distro) 2024-05-11T07:56:13.178 DEBUG:teuthology.orchestra.run.smithi204:> rpm -q kernel | sort -rV | head -n 1 2024-05-11T07:56:13.228 INFO:teuthology.orchestra.run.smithi204.stdout:kernel-5.14.0-444.el9.x86_64 2024-05-11T07:56:13.391 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-444.el9.x86_64 2024-05-11T07:56:13.391 DEBUG:teuthology.orchestra.run.smithi204:> sudo rpm -qi grub2-tools 2024-05-11T07:56:13.430 INFO:teuthology.orchestra.run.smithi204.stdout:Name : grub2-tools 2024-05-11T07:56:13.430 INFO:teuthology.orchestra.run.smithi204.stdout:Epoch : 1 2024-05-11T07:56:13.430 INFO:teuthology.orchestra.run.smithi204.stdout:Version : 2.06 2024-05-11T07:56:13.430 INFO:teuthology.orchestra.run.smithi204.stdout:Release : 68.el9 2024-05-11T07:56:13.430 INFO:teuthology.orchestra.run.smithi204.stdout:Architecture: x86_64 2024-05-11T07:56:13.430 INFO:teuthology.orchestra.run.smithi204.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-11T07:56:13.430 INFO:teuthology.orchestra.run.smithi204.stdout:Group : Unspecified 2024-05-11T07:56:13.430 INFO:teuthology.orchestra.run.smithi204.stdout:Size : 8264254 2024-05-11T07:56:13.430 INFO:teuthology.orchestra.run.smithi204.stdout:License : GPLv3+ 2024-05-11T07:56:13.430 INFO:teuthology.orchestra.run.smithi204.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-11T07:56:13.430 INFO:teuthology.orchestra.run.smithi204.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-11T07:56:13.430 INFO:teuthology.orchestra.run.smithi204.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-11T07:56:13.431 INFO:teuthology.orchestra.run.smithi204.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-11T07:56:13.431 INFO:teuthology.orchestra.run.smithi204.stdout:Packager : builder@centos.org 2024-05-11T07:56:13.431 INFO:teuthology.orchestra.run.smithi204.stdout:Vendor : CentOS 2024-05-11T07:56:13.431 INFO:teuthology.orchestra.run.smithi204.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-11T07:56:13.431 INFO:teuthology.orchestra.run.smithi204.stdout:Summary : Support tools for GRUB. 2024-05-11T07:56:13.431 INFO:teuthology.orchestra.run.smithi204.stdout:Description : 2024-05-11T07:56:13.431 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T07:56:13.431 INFO:teuthology.orchestra.run.smithi204.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-11T07:56:13.431 INFO:teuthology.orchestra.run.smithi204.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-11T07:56:13.431 INFO:teuthology.orchestra.run.smithi204.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-11T07:56:13.431 INFO:teuthology.orchestra.run.smithi204.stdout:hardware devices. 2024-05-11T07:56:13.431 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T07:56:13.431 INFO:teuthology.orchestra.run.smithi204.stdout:This subpackage provides tools for support of all platforms. 2024-05-11T07:56:13.432 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-11T07:56:13.432 INFO:teuthology.task.kernel:Updating grub on smithi204 to boot 5.14.0-444.el9.x86_64 2024-05-11T07:56:13.432 DEBUG:teuthology.orchestra.run.smithi204:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-11T07:56:13.726 INFO:teuthology.orchestra.run.smithi167.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-11T07:56:13.762 INFO:teuthology.orchestra.run.smithi167.stderr:done 2024-05-11T07:56:13.765 DEBUG:teuthology.orchestra.run.smithi167:> mktemp 2024-05-11T07:56:13.822 INFO:teuthology.orchestra.run.smithi167.stdout:/tmp/tmp.2QL81qtprO 2024-05-11T07:56:13.823 DEBUG:teuthology.orchestra.run.smithi167:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.2QL81qtprO 2024-05-11T07:56:13.894 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod 0666 /tmp/tmp.2QL81qtprO 2024-05-11T07:56:14.017 INFO:teuthology.orchestra.run.smithi204.stderr:Generating grub configuration file ... 2024-05-11T07:56:14.036 DEBUG:teuthology.orchestra.remote:smithi167:/tmp/tmp.2QL81qtprO is 6KB 2024-05-11T07:56:14.056 DEBUG:teuthology.orchestra.run.smithi167:> rm -fr /tmp/tmp.2QL81qtprO 2024-05-11T07:56:14.070 DEBUG:teuthology.orchestra.run.smithi167:> sudo /bin/ls /boot/loader/entries || true 2024-05-11T07:56:14.131 INFO:teuthology.orchestra.run.smithi167.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-11T07:56:14.131 INFO:teuthology.orchestra.run.smithi167.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-11T07:56:14.131 INFO:teuthology.orchestra.run.smithi167.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-11T07:56:14.131 INFO:teuthology.orchestra.run.smithi167.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-444.el9.x86_64.conf 2024-05-11T07:56:14.132 DEBUG:teuthology.orchestra.run.smithi167:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-5.14.0-444.el9.x86_64 2024-05-11T07:56:14.277 DEBUG:teuthology.orchestra.run.smithi167:> sudo shutdown -r now 2024-05-11T07:56:15.470 INFO:teuthology.orchestra.run.smithi204.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-11T07:56:15.509 INFO:teuthology.orchestra.run.smithi204.stderr:done 2024-05-11T07:56:15.513 DEBUG:teuthology.orchestra.run.smithi204:> mktemp 2024-05-11T07:56:15.528 INFO:teuthology.orchestra.run.smithi204.stdout:/tmp/tmp.TiIQkl5uX6 2024-05-11T07:56:15.528 DEBUG:teuthology.orchestra.run.smithi204:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.TiIQkl5uX6 2024-05-11T07:56:15.593 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod 0666 /tmp/tmp.TiIQkl5uX6 2024-05-11T07:56:15.739 DEBUG:teuthology.orchestra.remote:smithi204:/tmp/tmp.TiIQkl5uX6 is 6KB 2024-05-11T07:56:15.789 DEBUG:teuthology.orchestra.run.smithi204:> rm -fr /tmp/tmp.TiIQkl5uX6 2024-05-11T07:56:15.803 DEBUG:teuthology.orchestra.run.smithi204:> sudo /bin/ls /boot/loader/entries || true 2024-05-11T07:56:15.865 INFO:teuthology.orchestra.run.smithi204.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-11T07:56:15.865 INFO:teuthology.orchestra.run.smithi204.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-11T07:56:15.865 INFO:teuthology.orchestra.run.smithi204.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-11T07:56:15.865 INFO:teuthology.orchestra.run.smithi204.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-444.el9.x86_64.conf 2024-05-11T07:56:15.867 DEBUG:teuthology.orchestra.run.smithi204:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-5.14.0-444.el9.x86_64 2024-05-11T07:56:16.019 DEBUG:teuthology.orchestra.run.smithi204:> sudo shutdown -r now 2024-05-11T07:56:44.307 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-05-11T07:56:44.308 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi167.front.sepia.ceph.com' 2024-05-11T07:56:44.309 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-11T07:56:46.022 INFO:teuthology.task.kernel:Checking client client.1 for new kernel version... 2024-05-11T07:56:46.022 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi204.front.sepia.ceph.com' 2024-05-11T07:56:46.023 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi204.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-11T07:57:19.007 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.167 2024-05-11T07:57:20.546 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.204 2024-05-11T07:57:28.012 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi167.front.sepia.ceph.com' 2024-05-11T07:57:28.013 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-11T07:57:29.549 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi204.front.sepia.ceph.com' 2024-05-11T07:57:29.550 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi204.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-11T07:57:46.398 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.167 2024-05-11T07:57:58.407 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi167.front.sepia.ceph.com' 2024-05-11T07:57:58.408 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-11T07:58:01.474 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.167 2024-05-11T07:58:16.487 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi167.front.sepia.ceph.com' 2024-05-11T07:58:16.488 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-11T07:58:16.855 DEBUG:teuthology.orchestra.run.smithi167:> true 2024-05-11T07:58:17.242 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi167.front.sepia.ceph.com' 2024-05-11T07:58:17.242 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-444.el9.x86_64"... 2024-05-11T07:58:17.242 DEBUG:teuthology.orchestra.run.smithi167:> uname -r 2024-05-11T07:58:17.259 INFO:teuthology.orchestra.run.smithi167.stdout:5.14.0-444.el9.x86_64 2024-05-11T07:58:17.259 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-444.el9.x86_64 vs 5.14.0-444.el9.x86_64 2024-05-11T07:58:17.259 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-11T07:58:17.259 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-11T07:58:18.260 DEBUG:teuthology.parallel:result is None 2024-05-11T07:58:29.552 DEBUG:teuthology.orchestra.remote:timed out 2024-05-11T07:58:41.553 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi204.front.sepia.ceph.com' 2024-05-11T07:58:41.554 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi204.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-11T07:58:41.938 DEBUG:teuthology.orchestra.run.smithi204:> true 2024-05-11T07:58:42.316 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi204.front.sepia.ceph.com' 2024-05-11T07:58:42.316 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "5.14.0-444.el9.x86_64"... 2024-05-11T07:58:42.316 DEBUG:teuthology.orchestra.run.smithi204:> uname -r 2024-05-11T07:58:42.333 INFO:teuthology.orchestra.run.smithi204.stdout:5.14.0-444.el9.x86_64 2024-05-11T07:58:42.334 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-444.el9.x86_64 vs 5.14.0-444.el9.x86_64 2024-05-11T07:58:42.334 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-11T07:58:42.334 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-11T07:58:43.335 DEBUG:teuthology.parallel:result is None 2024-05-11T07:58:43.335 INFO:teuthology.run_tasks:Running task internal.base... 2024-05-11T07:58:43.342 INFO:teuthology.task.internal:Creating test directory... 2024-05-11T07:58:43.343 DEBUG:teuthology.orchestra.run.smithi167:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-11T07:58:43.346 DEBUG:teuthology.orchestra.run.smithi204:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-11T07:58:43.364 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-05-11T07:58:43.384 INFO:teuthology.run_tasks:Running task internal.archive... 2024-05-11T07:58:43.467 INFO:teuthology.task.internal:Creating archive directory... 2024-05-11T07:58:43.468 DEBUG:teuthology.orchestra.run.smithi167:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-11T07:58:43.470 DEBUG:teuthology.orchestra.run.smithi204:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-11T07:58:43.639 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-05-11T07:58:43.646 INFO:teuthology.task.internal:Enabling coredump saving... 2024-05-11T07:58:43.646 DEBUG:teuthology.orchestra.run.smithi167:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-11T07:58:43.649 DEBUG:teuthology.orchestra.run.smithi204:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-11T07:58:43.719 INFO:teuthology.orchestra.run.smithi167.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-11T07:58:43.720 INFO:teuthology.orchestra.run.smithi204.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-11T07:58:43.763 INFO:teuthology.orchestra.run.smithi204.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-11T07:58:43.772 INFO:teuthology.orchestra.run.smithi167.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-11T07:58:43.774 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-05-11T07:58:43.781 INFO:teuthology.task.internal:Configuring sudo... 2024-05-11T07:58:43.782 DEBUG:teuthology.orchestra.run.smithi167:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-11T07:58:43.784 DEBUG:teuthology.orchestra.run.smithi204:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-11T07:58:43.813 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-05-11T07:58:43.837 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-05-11T07:58:43.837 DEBUG:teuthology.orchestra.run.smithi167:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-11T07:58:43.854 DEBUG:teuthology.orchestra.run.smithi204:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-11T07:58:43.872 DEBUG:teuthology.orchestra.run.smithi167:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-11T07:58:43.957 DEBUG:teuthology.orchestra.run.smithi167:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-11T07:58:44.006 DEBUG:teuthology.orchestra.run.smithi167:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-11T07:58:44.090 DEBUG:teuthology.orchestra.run.smithi167:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-11T07:58:44.132 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-05-11T07:58:44.132 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-11T07:58:44.198 DEBUG:teuthology.orchestra.run.smithi204:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-11T07:58:44.243 DEBUG:teuthology.orchestra.run.smithi204:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-11T07:58:44.297 DEBUG:teuthology.orchestra.run.smithi204:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-11T07:58:44.381 DEBUG:teuthology.orchestra.run.smithi204:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-11T07:58:44.425 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-11T07:58:44.425 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-11T07:58:44.492 DEBUG:teuthology.orchestra.run.smithi167:> sudo service rsyslog restart 2024-05-11T07:58:44.494 DEBUG:teuthology.orchestra.run.smithi204:> sudo service rsyslog restart 2024-05-11T07:58:44.537 INFO:teuthology.orchestra.run.smithi167.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-11T07:58:44.580 INFO:teuthology.orchestra.run.smithi204.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-11T07:58:44.923 INFO:teuthology.run_tasks:Running task internal.timer... 2024-05-11T07:58:44.931 INFO:teuthology.task.internal:Starting timer... 2024-05-11T07:58:44.931 INFO:teuthology.run_tasks:Running task pcp... 2024-05-11T07:58:44.963 INFO:teuthology.run_tasks:Running task selinux... 2024-05-11T07:58:45.054 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-05-11T07:58:45.054 DEBUG:teuthology.orchestra.run.smithi167:> sudo service auditd rotate 2024-05-11T07:58:45.097 INFO:teuthology.orchestra.run.smithi167.stdout:Rotating logs: 2024-05-11T07:58:45.099 DEBUG:teuthology.orchestra.run.smithi204:> sudo service auditd rotate 2024-05-11T07:58:45.168 INFO:teuthology.orchestra.run.smithi204.stdout:Rotating logs: 2024-05-11T07:58:45.170 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-05-11T07:58:45.171 DEBUG:teuthology.orchestra.run.smithi167:> /usr/sbin/getenforce 2024-05-11T07:58:45.198 INFO:teuthology.orchestra.run.smithi167.stdout:Permissive 2024-05-11T07:58:45.199 DEBUG:teuthology.orchestra.run.smithi204:> /usr/sbin/getenforce 2024-05-11T07:58:45.232 INFO:teuthology.orchestra.run.smithi204.stdout:Permissive 2024-05-11T07:58:45.233 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi167.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi204.front.sepia.ceph.com': 'permissive'} 2024-05-11T07:58:45.233 DEBUG:teuthology.orchestra.run.smithi167:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-11T07:58:45.267 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-11T07:58:45.268 DEBUG:teuthology.orchestra.run.smithi204:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-11T07:58:45.299 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-11T07:58:45.300 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-05-11T07:58:45.300 DEBUG:teuthology.orchestra.run.smithi167:> sudo /usr/sbin/setenforce permissive 2024-05-11T07:58:45.343 DEBUG:teuthology.orchestra.run.smithi204:> sudo /usr/sbin/setenforce permissive 2024-05-11T07:58:45.369 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-05-11T07:58:45.379 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-05-11T07:58:45.384 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-05-11T07:58:45.515 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-05-11T07:58:45.531 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-05-11T07:58:45.532 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi167.front.sepia.ceph.com,smithi204.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-05-11T08:02:47.698 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi167.front.sepia.ceph.com'), Remote(name='ubuntu@smithi204.front.sepia.ceph.com')] 2024-05-11T08:02:47.699 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi167.front.sepia.ceph.com' 2024-05-11T08:02:47.701 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-11T08:02:47.778 DEBUG:teuthology.orchestra.run.smithi167:> true 2024-05-11T08:02:47.858 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi167.front.sepia.ceph.com' 2024-05-11T08:02:47.859 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi204.front.sepia.ceph.com' 2024-05-11T08:02:47.860 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi204.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-11T08:02:47.933 DEBUG:teuthology.orchestra.run.smithi204:> true 2024-05-11T08:02:48.009 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi204.front.sepia.ceph.com' 2024-05-11T08:02:48.010 INFO:teuthology.run_tasks:Running task clock... 2024-05-11T08:02:48.020 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-05-11T08:02:48.020 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-11T08:02:48.021 DEBUG:teuthology.orchestra.run.smithi167:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-11T08:02:48.023 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-11T08:02:48.023 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-11T08:02:48.051 INFO:teuthology.orchestra.run.smithi167.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-11T08:02:48.066 INFO:teuthology.orchestra.run.smithi167.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-11T08:02:48.079 INFO:teuthology.orchestra.run.smithi204.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-11T08:02:48.091 INFO:teuthology.orchestra.run.smithi167.stderr:sudo: ntpd: command not found 2024-05-11T08:02:48.094 INFO:teuthology.orchestra.run.smithi204.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-11T08:02:48.103 INFO:teuthology.orchestra.run.smithi167.stdout:506 Cannot talk to daemon 2024-05-11T08:02:48.117 INFO:teuthology.orchestra.run.smithi167.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-11T08:02:48.118 INFO:teuthology.orchestra.run.smithi204.stderr:sudo: ntpd: command not found 2024-05-11T08:02:48.130 INFO:teuthology.orchestra.run.smithi204.stdout:506 Cannot talk to daemon 2024-05-11T08:02:48.131 INFO:teuthology.orchestra.run.smithi167.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-11T08:02:48.144 INFO:teuthology.orchestra.run.smithi204.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-11T08:02:48.157 INFO:teuthology.orchestra.run.smithi204.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-11T08:02:48.184 INFO:teuthology.orchestra.run.smithi167.stderr:bash: line 1: ntpq: command not found 2024-05-11T08:02:48.189 INFO:teuthology.orchestra.run.smithi167.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-11T08:02:48.189 INFO:teuthology.orchestra.run.smithi167.stdout:=============================================================================== 2024-05-11T08:02:48.189 INFO:teuthology.orchestra.run.smithi167.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-11T08:02:48.189 INFO:teuthology.orchestra.run.smithi167.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-11T08:02:48.189 INFO:teuthology.orchestra.run.smithi167.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-11T08:02:48.189 INFO:teuthology.orchestra.run.smithi167.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-11T08:02:48.212 INFO:teuthology.orchestra.run.smithi204.stderr:bash: line 1: ntpq: command not found 2024-05-11T08:02:48.215 INFO:teuthology.orchestra.run.smithi204.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-11T08:02:48.215 INFO:teuthology.orchestra.run.smithi204.stdout:=============================================================================== 2024-05-11T08:02:48.216 INFO:teuthology.orchestra.run.smithi204.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-11T08:02:48.216 INFO:teuthology.orchestra.run.smithi204.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-11T08:02:48.216 INFO:teuthology.orchestra.run.smithi204.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-11T08:02:48.216 INFO:teuthology.orchestra.run.smithi204.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-11T08:02:48.217 INFO:teuthology.run_tasks:Running task install... 2024-05-11T08:02:48.225 DEBUG:teuthology.task.install:project ceph 2024-05-11T08:02:48.225 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '308506802b0bdea4df7050cfc1fcbdc04fea6a6e'}} 2024-05-11T08:02:48.225 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '308506802b0bdea4df7050cfc1fcbdc04fea6a6e'} 2024-05-11T08:02:48.225 INFO:teuthology.task.install:Using flavor: default 2024-05-11T08:02:48.231 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-05-11T08:02:48.231 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-05-11T08:02:48.231 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '308506802b0bdea4df7050cfc1fcbdc04fea6a6e', 'tag': None, 'wait_for_package': False} 2024-05-11T08:02:48.232 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=308506802b0bdea4df7050cfc1fcbdc04fea6a6e 2024-05-11T08:02:48.234 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '308506802b0bdea4df7050cfc1fcbdc04fea6a6e', 'tag': None, 'wait_for_package': False} 2024-05-11T08:02:48.234 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=308506802b0bdea4df7050cfc1fcbdc04fea6a6e 2024-05-11T08:02:48.374 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240510.053205/308506802b0bdea4df7050cfc1fcbdc04fea6a6e/centos/9/flavors/default/ 2024-05-11T08:02:48.374 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3601.g30850680 2024-05-11T08:02:48.383 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240510.053205/308506802b0bdea4df7050cfc1fcbdc04fea6a6e/centos/9/flavors/default/ 2024-05-11T08:02:48.383 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3601.g30850680 2024-05-11T08:02:48.547 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240510.053205/308506802b0bdea4df7050cfc1fcbdc04fea6a6e/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240510.053205/308506802b0bdea4df7050cfc1fcbdc04fea6a6e/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240510.053205/308506802b0bdea4df7050cfc1fcbdc04fea6a6e/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-11T08:02:48.548 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-05-11T08:02:48.548 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-11T08:02:48.557 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240510.053205/308506802b0bdea4df7050cfc1fcbdc04fea6a6e/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240510.053205/308506802b0bdea4df7050cfc1fcbdc04fea6a6e/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240510.053205/308506802b0bdea4df7050cfc1fcbdc04fea6a6e/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-11T08:02:48.557 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-11T08:02:48.557 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-11T08:02:48.575 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-11T08:02:48.575 DEBUG:teuthology.orchestra.run.smithi167:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/308506802b0bdea4df7050cfc1fcbdc04fea6a6e/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-11T08:02:48.584 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-11T08:02:48.584 DEBUG:teuthology.orchestra.run.smithi204:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/308506802b0bdea4df7050cfc1fcbdc04fea6a6e/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-11T08:02:48.641 DEBUG:teuthology.orchestra.run.smithi167:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-11T08:02:48.650 DEBUG:teuthology.orchestra.run.smithi204:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-11T08:02:48.717 DEBUG:teuthology.orchestra.run.smithi167:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-11T08:02:48.724 DEBUG:teuthology.orchestra.run.smithi204:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-11T08:02:48.781 INFO:teuthology.orchestra.run.smithi167.stdout:check_obsoletes = 1 2024-05-11T08:02:48.784 DEBUG:teuthology.orchestra.run.smithi167:> sudo yum clean all 2024-05-11T08:02:48.789 INFO:teuthology.orchestra.run.smithi204.stdout:check_obsoletes = 1 2024-05-11T08:02:48.791 DEBUG:teuthology.orchestra.run.smithi204:> sudo yum clean all 2024-05-11T08:02:49.018 INFO:teuthology.orchestra.run.smithi167.stdout:54 files removed 2024-05-11T08:02:49.025 INFO:teuthology.orchestra.run.smithi204.stdout:54 files removed 2024-05-11T08:02:49.045 DEBUG:teuthology.orchestra.run.smithi167:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-11T08:02:49.055 DEBUG:teuthology.orchestra.run.smithi204:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-11T08:02:49.920 INFO:teuthology.orchestra.run.smithi204.stdout:ceph packages for x86_64 142 kB/s | 84 kB 00:00 2024-05-11T08:02:49.953 INFO:teuthology.orchestra.run.smithi167.stdout:ceph packages for x86_64 132 kB/s | 84 kB 00:00 2024-05-11T08:02:50.470 INFO:teuthology.orchestra.run.smithi167.stdout:ceph noarch packages 25 kB/s | 12 kB 00:00 2024-05-11T08:02:50.800 INFO:teuthology.orchestra.run.smithi204.stdout:ceph noarch packages 14 kB/s | 12 kB 00:00 2024-05-11T08:02:51.179 INFO:teuthology.orchestra.run.smithi167.stdout:ceph source packages 2.9 kB/s | 1.9 kB 00:00 2024-05-11T08:02:51.484 INFO:teuthology.orchestra.run.smithi204.stdout:ceph source packages 3.0 kB/s | 1.9 kB 00:00 2024-05-11T08:02:51.937 INFO:teuthology.orchestra.run.smithi167.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.1 MB 00:00 2024-05-11T08:02:52.068 INFO:teuthology.orchestra.run.smithi204.stdout:CentOS Stream 9 - BaseOS 15 MB/s | 8.1 MB 00:00 2024-05-11T08:02:54.928 INFO:teuthology.orchestra.run.smithi204.stdout:CentOS Stream 9 - AppStream 11 MB/s | 19 MB 00:01 2024-05-11T08:02:59.754 INFO:teuthology.orchestra.run.smithi204.stdout:CentOS Stream 9 - CRB 5.0 MB/s | 6.2 MB 00:01 2024-05-11T08:03:01.180 INFO:teuthology.orchestra.run.smithi204.stdout:CentOS Stream 9 - Extras packages 66 kB/s | 16 kB 00:00 2024-05-11T08:03:02.674 INFO:teuthology.orchestra.run.smithi204.stdout:Extra Packages for Enterprise Linux 16 MB/s | 21 MB 00:01 2024-05-11T08:03:07.849 INFO:teuthology.orchestra.run.smithi204.stdout:lab-extras 19 kB/s | 1.7 kB 00:00 2024-05-11T08:03:08.463 INFO:teuthology.orchestra.run.smithi167.stdout:CentOS Stream 9 - AppStream 1.2 MB/s | 19 MB 00:15 2024-05-11T08:03:10.176 INFO:teuthology.orchestra.run.smithi204.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-11T08:03:10.177 INFO:teuthology.orchestra.run.smithi204.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-11T08:03:10.187 INFO:teuthology.orchestra.run.smithi204.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-11T08:03:10.189 INFO:teuthology.orchestra.run.smithi204.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-11T08:03:10.189 INFO:teuthology.orchestra.run.smithi204.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-11T08:03:10.190 INFO:teuthology.orchestra.run.smithi204.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-11T08:03:10.191 INFO:teuthology.orchestra.run.smithi204.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-11T08:03:10.194 INFO:teuthology.orchestra.run.smithi204.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-11T08:03:10.194 INFO:teuthology.orchestra.run.smithi204.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-11T08:03:10.255 INFO:teuthology.orchestra.run.smithi204.stdout:Dependencies resolved. 2024-05-11T08:03:10.269 INFO:teuthology.orchestra.run.smithi204.stdout:======================================================================================= 2024-05-11T08:03:10.269 INFO:teuthology.orchestra.run.smithi204.stdout: Package Arch Version Repository Size 2024-05-11T08:03:10.269 INFO:teuthology.orchestra.run.smithi204.stdout:======================================================================================= 2024-05-11T08:03:10.269 INFO:teuthology.orchestra.run.smithi204.stdout:Installing: 2024-05-11T08:03:10.270 INFO:teuthology.orchestra.run.smithi204.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-11T08:03:10.270 INFO:teuthology.orchestra.run.smithi204.stdout: ceph x86_64 2:19.0.0-3601.g30850680.el9 ceph 6.6 k 2024-05-11T08:03:10.270 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-base x86_64 2:19.0.0-3601.g30850680.el9 ceph 5.4 M 2024-05-11T08:03:10.270 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-fuse x86_64 2:19.0.0-3601.g30850680.el9 ceph 890 k 2024-05-11T08:03:10.270 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3601.g30850680.el9 ceph 145 k 2024-05-11T08:03:10.270 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr x86_64 2:19.0.0-3601.g30850680.el9 ceph 1.5 M 2024-05-11T08:03:10.270 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3601.g30850680.el9 ceph-noarch 142 k 2024-05-11T08:03:10.270 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3601.g30850680.el9 ceph-noarch 3.5 M 2024-05-11T08:03:10.270 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3601.g30850680.el9 ceph-noarch 7.4 M 2024-05-11T08:03:10.270 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-rook noarch 2:19.0.0-3601.g30850680.el9 ceph-noarch 49 k 2024-05-11T08:03:10.270 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-radosgw x86_64 2:19.0.0-3601.g30850680.el9 ceph 11 M 2024-05-11T08:03:10.270 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-test x86_64 2:19.0.0-3601.g30850680.el9 ceph 49 M 2024-05-11T08:03:10.271 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-volume noarch 2:19.0.0-3601.g30850680.el9 ceph-noarch 273 k 2024-05-11T08:03:10.271 INFO:teuthology.orchestra.run.smithi204.stdout: cephadm noarch 2:19.0.0-3601.g30850680.el9 ceph-noarch 764 k 2024-05-11T08:03:10.271 INFO:teuthology.orchestra.run.smithi204.stdout: cephfs-mirror x86_64 2:19.0.0-3601.g30850680.el9 ceph 225 k 2024-05-11T08:03:10.271 INFO:teuthology.orchestra.run.smithi204.stdout: cephfs-shell x86_64 2:19.0.0-3601.g30850680.el9 ceph 27 k 2024-05-11T08:03:10.271 INFO:teuthology.orchestra.run.smithi204.stdout: cephfs-top noarch 2:19.0.0-3601.g30850680.el9 ceph-noarch 18 k 2024-05-11T08:03:10.271 INFO:teuthology.orchestra.run.smithi204.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-05-11T08:03:10.271 INFO:teuthology.orchestra.run.smithi204.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-11T08:03:10.271 INFO:teuthology.orchestra.run.smithi204.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-11T08:03:10.271 INFO:teuthology.orchestra.run.smithi204.stdout: libcephfs-devel x86_64 2:19.0.0-3601.g30850680.el9 ceph 32 k 2024-05-11T08:03:10.271 INFO:teuthology.orchestra.run.smithi204.stdout: libcephfs2 x86_64 2:19.0.0-3601.g30850680.el9 ceph 746 k 2024-05-11T08:03:10.271 INFO:teuthology.orchestra.run.smithi204.stdout: librados-devel x86_64 2:19.0.0-3601.g30850680.el9 ceph 127 k 2024-05-11T08:03:10.271 INFO:teuthology.orchestra.run.smithi204.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-11T08:03:10.272 INFO:teuthology.orchestra.run.smithi204.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-11T08:03:10.272 INFO:teuthology.orchestra.run.smithi204.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-11T08:03:10.272 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-11T08:03:10.272 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-11T08:03:10.272 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-11T08:03:10.272 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cephfs x86_64 2:19.0.0-3601.g30850680.el9 ceph 161 k 2024-05-11T08:03:10.272 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rados x86_64 2:19.0.0-3601.g30850680.el9 ceph 321 k 2024-05-11T08:03:10.272 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rbd x86_64 2:19.0.0-3601.g30850680.el9 ceph 300 k 2024-05-11T08:03:10.272 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rgw x86_64 2:19.0.0-3601.g30850680.el9 ceph 100 k 2024-05-11T08:03:10.272 INFO:teuthology.orchestra.run.smithi204.stdout: rbd-fuse x86_64 2:19.0.0-3601.g30850680.el9 ceph 86 k 2024-05-11T08:03:10.272 INFO:teuthology.orchestra.run.smithi204.stdout: rbd-mirror x86_64 2:19.0.0-3601.g30850680.el9 ceph 3.1 M 2024-05-11T08:03:10.272 INFO:teuthology.orchestra.run.smithi204.stdout: rbd-nbd x86_64 2:19.0.0-3601.g30850680.el9 ceph 171 k 2024-05-11T08:03:10.272 INFO:teuthology.orchestra.run.smithi204.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-11T08:03:10.273 INFO:teuthology.orchestra.run.smithi204.stdout:Upgrading: 2024-05-11T08:03:10.273 INFO:teuthology.orchestra.run.smithi204.stdout: NetworkManager x86_64 1:1.47.5-1.el9 baseos 2.3 M 2024-05-11T08:03:10.273 INFO:teuthology.orchestra.run.smithi204.stdout: NetworkManager-libnm x86_64 1:1.47.5-1.el9 baseos 1.8 M 2024-05-11T08:03:10.273 INFO:teuthology.orchestra.run.smithi204.stdout: NetworkManager-team x86_64 1:1.47.5-1.el9 baseos 42 k 2024-05-11T08:03:10.273 INFO:teuthology.orchestra.run.smithi204.stdout: NetworkManager-tui x86_64 1:1.47.5-1.el9 baseos 248 k 2024-05-11T08:03:10.273 INFO:teuthology.orchestra.run.smithi204.stdout: librados2 x86_64 2:19.0.0-3601.g30850680.el9 ceph 3.4 M 2024-05-11T08:03:10.273 INFO:teuthology.orchestra.run.smithi204.stdout: librbd1 x86_64 2:19.0.0-3601.g30850680.el9 ceph 3.1 M 2024-05-11T08:03:10.273 INFO:teuthology.orchestra.run.smithi204.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-11T08:03:10.273 INFO:teuthology.orchestra.run.smithi204.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-11T08:03:10.273 INFO:teuthology.orchestra.run.smithi204.stdout:Installing dependencies: 2024-05-11T08:03:10.274 INFO:teuthology.orchestra.run.smithi204.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-11T08:03:10.274 INFO:teuthology.orchestra.run.smithi204.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-11T08:03:10.274 INFO:teuthology.orchestra.run.smithi204.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-11T08:03:10.274 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-common x86_64 2:19.0.0-3601.g30850680.el9 ceph 21 M 2024-05-11T08:03:10.274 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3601.g30850680.el9 ceph-noarch 34 k 2024-05-11T08:03:10.274 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mds x86_64 2:19.0.0-3601.g30850680.el9 ceph 2.4 M 2024-05-11T08:03:10.274 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3601.g30850680.el9 ceph-noarch 274 k 2024-05-11T08:03:10.274 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mon x86_64 2:19.0.0-3601.g30850680.el9 ceph 4.7 M 2024-05-11T08:03:10.274 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-osd x86_64 2:19.0.0-3601.g30850680.el9 ceph 17 M 2024-05-11T08:03:10.274 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3601.g30850680.el9 ceph-noarch 16 k 2024-05-11T08:03:10.274 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-selinux x86_64 2:19.0.0-3601.g30850680.el9 ceph 25 k 2024-05-11T08:03:10.274 INFO:teuthology.orchestra.run.smithi204.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-11T08:03:10.274 INFO:teuthology.orchestra.run.smithi204.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-11T08:03:10.274 INFO:teuthology.orchestra.run.smithi204.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-11T08:03:10.275 INFO:teuthology.orchestra.run.smithi204.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-11T08:03:10.275 INFO:teuthology.orchestra.run.smithi204.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-11T08:03:10.275 INFO:teuthology.orchestra.run.smithi204.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-11T08:03:10.275 INFO:teuthology.orchestra.run.smithi204.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-11T08:03:10.275 INFO:teuthology.orchestra.run.smithi204.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-11T08:03:10.275 INFO:teuthology.orchestra.run.smithi204.stdout: libcephsqlite x86_64 2:19.0.0-3601.g30850680.el9 ceph 163 k 2024-05-11T08:03:10.275 INFO:teuthology.orchestra.run.smithi204.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-11T08:03:10.275 INFO:teuthology.orchestra.run.smithi204.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-11T08:03:10.275 INFO:teuthology.orchestra.run.smithi204.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-11T08:03:10.275 INFO:teuthology.orchestra.run.smithi204.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-11T08:03:10.275 INFO:teuthology.orchestra.run.smithi204.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-11T08:03:10.275 INFO:teuthology.orchestra.run.smithi204.stdout: libradosstriper1 x86_64 2:19.0.0-3601.g30850680.el9 ceph 501 k 2024-05-11T08:03:10.275 INFO:teuthology.orchestra.run.smithi204.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-11T08:03:10.276 INFO:teuthology.orchestra.run.smithi204.stdout: librgw2 x86_64 2:19.0.0-3601.g30850680.el9 ceph 5.4 M 2024-05-11T08:03:10.276 INFO:teuthology.orchestra.run.smithi204.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-11T08:03:10.276 INFO:teuthology.orchestra.run.smithi204.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-11T08:03:10.276 INFO:teuthology.orchestra.run.smithi204.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-11T08:03:10.276 INFO:teuthology.orchestra.run.smithi204.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-11T08:03:10.276 INFO:teuthology.orchestra.run.smithi204.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-11T08:03:10.276 INFO:teuthology.orchestra.run.smithi204.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-05-11T08:03:10.276 INFO:teuthology.orchestra.run.smithi204.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-11T08:03:10.276 INFO:teuthology.orchestra.run.smithi204.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-11T08:03:10.276 INFO:teuthology.orchestra.run.smithi204.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-11T08:03:10.276 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-11T08:03:10.276 INFO:teuthology.orchestra.run.smithi204.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-11T08:03:10.277 INFO:teuthology.orchestra.run.smithi204.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-11T08:03:10.277 INFO:teuthology.orchestra.run.smithi204.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-11T08:03:10.277 INFO:teuthology.orchestra.run.smithi204.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-11T08:03:10.277 INFO:teuthology.orchestra.run.smithi204.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-11T08:03:10.277 INFO:teuthology.orchestra.run.smithi204.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-11T08:03:10.277 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-11T08:03:10.277 INFO:teuthology.orchestra.run.smithi204.stdout: python3-ceph-argparse x86_64 2:19.0.0-3601.g30850680.el9 ceph 45 k 2024-05-11T08:03:10.277 INFO:teuthology.orchestra.run.smithi204.stdout: python3-ceph-common x86_64 2:19.0.0-3601.g30850680.el9 ceph 135 k 2024-05-11T08:03:10.277 INFO:teuthology.orchestra.run.smithi204.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-11T08:03:10.277 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-11T08:03:10.277 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-11T08:03:10.277 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-11T08:03:10.277 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-11T08:03:10.278 INFO:teuthology.orchestra.run.smithi204.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-05-11T08:03:10.278 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-11T08:03:10.278 INFO:teuthology.orchestra.run.smithi204.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-11T08:03:10.278 INFO:teuthology.orchestra.run.smithi204.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-11T08:03:10.278 INFO:teuthology.orchestra.run.smithi204.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-11T08:03:10.278 INFO:teuthology.orchestra.run.smithi204.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-11T08:03:10.278 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-11T08:03:10.278 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-11T08:03:10.278 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-11T08:03:10.278 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-11T08:03:10.278 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-11T08:03:10.279 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-11T08:03:10.279 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-11T08:03:10.279 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-11T08:03:10.279 INFO:teuthology.orchestra.run.smithi204.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-11T08:03:10.279 INFO:teuthology.orchestra.run.smithi204.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-11T08:03:10.279 INFO:teuthology.orchestra.run.smithi204.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-11T08:03:10.279 INFO:teuthology.orchestra.run.smithi204.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-11T08:03:10.279 INFO:teuthology.orchestra.run.smithi204.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-11T08:03:10.279 INFO:teuthology.orchestra.run.smithi204.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-11T08:03:10.279 INFO:teuthology.orchestra.run.smithi204.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-11T08:03:10.279 INFO:teuthology.orchestra.run.smithi204.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-11T08:03:10.279 INFO:teuthology.orchestra.run.smithi204.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-11T08:03:10.279 INFO:teuthology.orchestra.run.smithi204.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-11T08:03:10.280 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-11T08:03:10.280 INFO:teuthology.orchestra.run.smithi204.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-11T08:03:10.280 INFO:teuthology.orchestra.run.smithi204.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-11T08:03:10.280 INFO:teuthology.orchestra.run.smithi204.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-11T08:03:10.280 INFO:teuthology.orchestra.run.smithi204.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-11T08:03:10.280 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-11T08:03:10.280 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-11T08:03:10.280 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-11T08:03:10.280 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-11T08:03:10.280 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-11T08:03:10.280 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-11T08:03:10.280 INFO:teuthology.orchestra.run.smithi204.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-11T08:03:10.280 INFO:teuthology.orchestra.run.smithi204.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-11T08:03:10.280 INFO:teuthology.orchestra.run.smithi204.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-11T08:03:10.281 INFO:teuthology.orchestra.run.smithi204.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-11T08:03:10.281 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-11T08:03:10.281 INFO:teuthology.orchestra.run.smithi204.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-11T08:03:10.281 INFO:teuthology.orchestra.run.smithi204.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-11T08:03:10.281 INFO:teuthology.orchestra.run.smithi204.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-11T08:03:10.281 INFO:teuthology.orchestra.run.smithi204.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-11T08:03:10.281 INFO:teuthology.orchestra.run.smithi204.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-11T08:03:10.281 INFO:teuthology.orchestra.run.smithi204.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-11T08:03:10.281 INFO:teuthology.orchestra.run.smithi204.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-11T08:03:10.281 INFO:teuthology.orchestra.run.smithi204.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-11T08:03:10.281 INFO:teuthology.orchestra.run.smithi204.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-11T08:03:10.281 INFO:teuthology.orchestra.run.smithi204.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-11T08:03:10.282 INFO:teuthology.orchestra.run.smithi204.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-11T08:03:10.282 INFO:teuthology.orchestra.run.smithi204.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-11T08:03:10.282 INFO:teuthology.orchestra.run.smithi204.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-11T08:03:10.282 INFO:teuthology.orchestra.run.smithi204.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-11T08:03:10.282 INFO:teuthology.orchestra.run.smithi204.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-11T08:03:10.282 INFO:teuthology.orchestra.run.smithi204.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-11T08:03:10.282 INFO:teuthology.orchestra.run.smithi204.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-11T08:03:10.282 INFO:teuthology.orchestra.run.smithi204.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-11T08:03:10.282 INFO:teuthology.orchestra.run.smithi204.stdout:Installing weak dependencies: 2024-05-11T08:03:10.282 INFO:teuthology.orchestra.run.smithi204.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-11T08:03:10.282 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T08:03:10.282 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction Summary 2024-05-11T08:03:10.283 INFO:teuthology.orchestra.run.smithi204.stdout:======================================================================================= 2024-05-11T08:03:10.283 INFO:teuthology.orchestra.run.smithi204.stdout:Install 148 Packages 2024-05-11T08:03:10.283 INFO:teuthology.orchestra.run.smithi204.stdout:Upgrade 8 Packages 2024-05-11T08:03:10.283 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T08:03:10.284 INFO:teuthology.orchestra.run.smithi204.stdout:Total download size: 226 M 2024-05-11T08:03:10.284 INFO:teuthology.orchestra.run.smithi204.stdout:Downloading Packages: 2024-05-11T08:03:11.234 INFO:teuthology.orchestra.run.smithi204.stdout:(1/156): ceph-19.0.0-3601.g30850680.el9.x86_64. 26 kB/s | 6.6 kB 00:00 2024-05-11T08:03:11.843 INFO:teuthology.orchestra.run.smithi204.stdout:(2/156): ceph-fuse-19.0.0-3601.g30850680.el9.x8 1.4 MB/s | 890 kB 00:00 2024-05-11T08:03:11.935 INFO:teuthology.orchestra.run.smithi204.stdout:(3/156): ceph-immutable-object-cache-19.0.0-360 1.5 MB/s | 145 kB 00:00 2024-05-11T08:03:12.547 INFO:teuthology.orchestra.run.smithi167.stdout:CentOS Stream 9 - CRB 11 MB/s | 6.2 MB 00:00 2024-05-11T08:03:12.952 INFO:teuthology.orchestra.run.smithi204.stdout:(4/156): ceph-mds-19.0.0-3601.g30850680.el9.x86 2.3 MB/s | 2.4 MB 00:01 2024-05-11T08:03:13.360 INFO:teuthology.orchestra.run.smithi204.stdout:(5/156): ceph-base-19.0.0-3601.g30850680.el9.x8 2.3 MB/s | 5.4 MB 00:02 2024-05-11T08:03:13.760 INFO:teuthology.orchestra.run.smithi204.stdout:(6/156): ceph-mgr-19.0.0-3601.g30850680.el9.x86 1.9 MB/s | 1.5 MB 00:00 2024-05-11T08:03:14.065 INFO:teuthology.orchestra.run.smithi167.stdout:CentOS Stream 9 - Extras packages 47 kB/s | 16 kB 00:00 2024-05-11T08:03:14.827 INFO:teuthology.orchestra.run.smithi204.stdout:(7/156): ceph-mon-19.0.0-3601.g30850680.el9.x86 3.2 MB/s | 4.7 MB 00:01 2024-05-11T08:03:16.627 INFO:teuthology.orchestra.run.smithi204.stdout:(8/156): ceph-radosgw-19.0.0-3601.g30850680.el9 5.9 MB/s | 11 MB 00:01 2024-05-11T08:03:16.678 INFO:teuthology.orchestra.run.smithi204.stdout:(9/156): ceph-selinux-19.0.0-3601.g30850680.el9 503 kB/s | 25 kB 00:00 2024-05-11T08:03:18.528 INFO:teuthology.orchestra.run.smithi204.stdout:(10/156): ceph-common-19.0.0-3601.g30850680.el9 2.8 MB/s | 21 MB 00:07 2024-05-11T08:03:19.053 INFO:teuthology.orchestra.run.smithi204.stdout:(11/156): cephfs-mirror-19.0.0-3601.g30850680.e 428 kB/s | 225 kB 00:00 2024-05-11T08:03:19.170 INFO:teuthology.orchestra.run.smithi204.stdout:(12/156): cephfs-shell-19.0.0-3601.g30850680.el 232 kB/s | 27 kB 00:00 2024-05-11T08:03:19.320 INFO:teuthology.orchestra.run.smithi204.stdout:(13/156): libcephfs-devel-19.0.0-3601.g30850680 211 kB/s | 32 kB 00:00 2024-05-11T08:03:20.104 INFO:teuthology.orchestra.run.smithi204.stdout:(14/156): ceph-test-19.0.0-3601.g30850680.el9.x 14 MB/s | 49 MB 00:03 2024-05-11T08:03:20.138 INFO:teuthology.orchestra.run.smithi204.stdout:(15/156): libcephfs2-19.0.0-3601.g30850680.el9. 913 kB/s | 746 kB 00:00 2024-05-11T08:03:20.163 INFO:teuthology.orchestra.run.smithi204.stdout:(16/156): libcephsqlite-19.0.0-3601.g30850680.e 2.7 MB/s | 163 kB 00:00 2024-05-11T08:03:20.230 INFO:teuthology.orchestra.run.smithi204.stdout:(17/156): libradosstriper1-19.0.0-3601.g3085068 7.3 MB/s | 501 kB 00:00 2024-05-11T08:03:20.255 INFO:teuthology.orchestra.run.smithi204.stdout:(18/156): librados-devel-19.0.0-3601.g30850680. 1.1 MB/s | 127 kB 00:00 2024-05-11T08:03:20.356 INFO:teuthology.orchestra.run.smithi204.stdout:(19/156): python3-ceph-argparse-19.0.0-3601.g30 452 kB/s | 45 kB 00:00 2024-05-11T08:03:20.427 INFO:teuthology.orchestra.run.smithi167.stdout:Extra Packages for Enterprise Linux 3.4 MB/s | 21 MB 00:06 2024-05-11T08:03:20.456 INFO:teuthology.orchestra.run.smithi204.stdout:(20/156): python3-ceph-common-19.0.0-3601.g3085 1.3 MB/s | 135 kB 00:00 2024-05-11T08:03:20.590 INFO:teuthology.orchestra.run.smithi204.stdout:(21/156): librgw2-19.0.0-3601.g30850680.el9.x86 15 MB/s | 5.4 MB 00:00 2024-05-11T08:03:20.653 INFO:teuthology.orchestra.run.smithi204.stdout:(22/156): python3-cephfs-19.0.0-3601.g30850680. 921 kB/s | 161 kB 00:00 2024-05-11T08:03:20.657 INFO:teuthology.orchestra.run.smithi204.stdout:(23/156): python3-rados-19.0.0-3601.g30850680.e 4.7 MB/s | 321 kB 00:00 2024-05-11T08:03:20.715 INFO:teuthology.orchestra.run.smithi204.stdout:(24/156): python3-rgw-19.0.0-3601.g30850680.el9 1.7 MB/s | 100 kB 00:00 2024-05-11T08:03:20.741 INFO:teuthology.orchestra.run.smithi204.stdout:(25/156): python3-rbd-19.0.0-3601.g30850680.el9 2.7 MB/s | 300 kB 00:00 2024-05-11T08:03:20.766 INFO:teuthology.orchestra.run.smithi204.stdout:(26/156): rbd-fuse-19.0.0-3601.g30850680.el9.x8 1.7 MB/s | 86 kB 00:00 2024-05-11T08:03:20.816 INFO:teuthology.orchestra.run.smithi204.stdout:(27/156): rbd-nbd-19.0.0-3601.g30850680.el9.x86 3.3 MB/s | 171 kB 00:00 2024-05-11T08:03:20.866 INFO:teuthology.orchestra.run.smithi204.stdout:(28/156): ceph-grafana-dashboards-19.0.0-3601.g 685 kB/s | 34 kB 00:00 2024-05-11T08:03:20.925 INFO:teuthology.orchestra.run.smithi204.stdout:(29/156): ceph-mgr-cephadm-19.0.0-3601.g3085068 2.4 MB/s | 142 kB 00:00 2024-05-11T08:03:21.200 INFO:teuthology.orchestra.run.smithi204.stdout:(30/156): ceph-mgr-dashboard-19.0.0-3601.g30850 13 MB/s | 3.5 MB 00:00 2024-05-11T08:03:21.801 INFO:teuthology.orchestra.run.smithi204.stdout:(31/156): rbd-mirror-19.0.0-3601.g30850680.el9. 2.9 MB/s | 3.1 MB 00:01 2024-05-11T08:03:21.968 INFO:teuthology.orchestra.run.smithi204.stdout:(32/156): ceph-mgr-diskprediction-local-19.0.0- 9.6 MB/s | 7.4 MB 00:00 2024-05-11T08:03:22.002 INFO:teuthology.orchestra.run.smithi204.stdout:(33/156): ceph-mgr-modules-core-19.0.0-3601.g30 1.3 MB/s | 274 kB 00:00 2024-05-11T08:03:22.027 INFO:teuthology.orchestra.run.smithi204.stdout:(34/156): ceph-mgr-rook-19.0.0-3601.g30850680.e 846 kB/s | 49 kB 00:00 2024-05-11T08:03:22.061 INFO:teuthology.orchestra.run.smithi204.stdout:(35/156): ceph-prometheus-alerts-19.0.0-3601.g3 280 kB/s | 16 kB 00:00 2024-05-11T08:03:22.086 INFO:teuthology.orchestra.run.smithi204.stdout:(36/156): ceph-volume-19.0.0-3601.g30850680.el9 4.5 MB/s | 273 kB 00:00 2024-05-11T08:03:22.128 INFO:teuthology.orchestra.run.smithi204.stdout:(37/156): cephfs-top-19.0.0-3601.g30850680.el9. 437 kB/s | 18 kB 00:00 2024-05-11T08:03:22.253 INFO:teuthology.orchestra.run.smithi204.stdout:(38/156): fuse-2.9.9-15.el9.x86_64.rpm 643 kB/s | 80 kB 00:00 2024-05-11T08:03:22.278 INFO:teuthology.orchestra.run.smithi204.stdout:(39/156): cephadm-19.0.0-3601.g30850680.el9.noa 3.4 MB/s | 764 kB 00:00 2024-05-11T08:03:22.362 INFO:teuthology.orchestra.run.smithi204.stdout:(40/156): libgfortran-11.4.1-3.el9.x86_64.rpm 7.2 MB/s | 803 kB 00:00 2024-05-11T08:03:22.404 INFO:teuthology.orchestra.run.smithi204.stdout:(41/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 6.0 MB/s | 253 kB 00:00 2024-05-11T08:03:22.438 INFO:teuthology.orchestra.run.smithi204.stdout:(42/156): libquadmath-11.4.1-3.el9.x86_64.rpm 1.2 MB/s | 195 kB 00:00 2024-05-11T08:03:22.480 INFO:teuthology.orchestra.run.smithi204.stdout:(43/156): python3-cryptography-36.0.1-4.el9.x86 16 MB/s | 1.2 MB 00:00 2024-05-11T08:03:22.496 INFO:teuthology.orchestra.run.smithi204.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-05-11T08:03:22.522 INFO:teuthology.orchestra.run.smithi204.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 3.1 MB/s | 135 kB 00:00 2024-05-11T08:03:22.547 INFO:teuthology.orchestra.run.smithi204.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-05-11T08:03:22.572 INFO:teuthology.orchestra.run.smithi204.stdout:(47/156): python3-urllib3-1.26.5-5.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-05-11T08:03:22.723 INFO:teuthology.orchestra.run.smithi204.stdout:(48/156): automake-1.16.2-8.el9.noarch.rpm 3.8 MB/s | 689 kB 00:00 2024-05-11T08:03:22.890 INFO:teuthology.orchestra.run.smithi204.stdout:(49/156): ceph-osd-19.0.0-3601.g30850680.el9.x8 1.8 MB/s | 17 MB 00:09 2024-05-11T08:03:22.915 INFO:teuthology.orchestra.run.smithi204.stdout:(50/156): boost-program-options-1.75.0-8.el9.x8 557 kB/s | 107 kB 00:00 2024-05-11T08:03:22.957 INFO:teuthology.orchestra.run.smithi204.stdout:(51/156): bison-3.7.4-5.el9.x86_64.rpm 2.4 MB/s | 943 kB 00:00 2024-05-11T08:03:22.974 INFO:teuthology.orchestra.run.smithi204.stdout:(52/156): elfutils-libelf-devel-0.190-2.el9.x86 257 kB/s | 21 kB 00:00 2024-05-11T08:03:22.999 INFO:teuthology.orchestra.run.smithi204.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 780 kB/s | 33 kB 00:00 2024-05-11T08:03:23.025 INFO:teuthology.orchestra.run.smithi204.stdout:(54/156): flex-2.6.4-9.el9.x86_64.rpm 2.8 MB/s | 313 kB 00:00 2024-05-11T08:03:23.050 INFO:teuthology.orchestra.run.smithi204.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 362 kB/s | 18 kB 00:00 2024-05-11T08:03:23.075 INFO:teuthology.orchestra.run.smithi204.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 221 kB/s | 11 kB 00:00 2024-05-11T08:03:23.101 INFO:teuthology.orchestra.run.smithi204.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-05-11T08:03:23.126 INFO:teuthology.orchestra.run.smithi204.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 903 kB/s | 45 kB 00:00 2024-05-11T08:03:23.159 INFO:teuthology.orchestra.run.smithi204.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 662 kB 00:00 2024-05-11T08:03:23.193 INFO:teuthology.orchestra.run.smithi204.stdout:(60/156): libtool-2.4.6-45.el9.x86_64.rpm 8.5 MB/s | 582 kB 00:00 2024-05-11T08:03:23.219 INFO:teuthology.orchestra.run.smithi204.stdout:(61/156): libuuid-devel-2.37.4-18.el9.x86_64.rp 433 kB/s | 25 kB 00:00 2024-05-11T08:03:23.244 INFO:teuthology.orchestra.run.smithi204.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 945 kB/s | 47 kB 00:00 2024-05-11T08:03:23.278 INFO:teuthology.orchestra.run.smithi204.stdout:(63/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.9 MB/s | 292 kB 00:00 2024-05-11T08:03:23.303 INFO:teuthology.orchestra.run.smithi204.stdout:(64/156): lua-5.4.4-4.el9.x86_64.rpm 3.1 MB/s | 188 kB 00:00 2024-05-11T08:03:23.328 INFO:teuthology.orchestra.run.smithi204.stdout:(65/156): openblas-0.3.21-2.el9.x86_64.rpm 702 kB/s | 35 kB 00:00 2024-05-11T08:03:23.379 INFO:teuthology.orchestra.run.smithi204.stdout:(66/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 7.4 MB/s | 3.0 MB 00:00 2024-05-11T08:03:23.471 INFO:teuthology.orchestra.run.smithi204.stdout:(67/156): openblas-openmp-0.3.21-2.el9.x86_64.r 28 MB/s | 4.7 MB 00:00 2024-05-11T08:03:23.538 INFO:teuthology.orchestra.run.smithi204.stdout:(68/156): openssl-devel-3.2.1-1.el9.x86_64.rpm 21 MB/s | 4.4 MB 00:00 2024-05-11T08:03:23.579 INFO:teuthology.orchestra.run.smithi204.stdout:(69/156): postgresql-13.14-1.el9.x86_64.rpm 7.9 MB/s | 1.6 MB 00:00 2024-05-11T08:03:23.605 INFO:teuthology.orchestra.run.smithi204.stdout:(70/156): postgresql-private-libs-13.14-1.el9.x 2.0 MB/s | 133 kB 00:00 2024-05-11T08:03:23.647 INFO:teuthology.orchestra.run.smithi204.stdout:(71/156): postgresql-contrib-13.14-1.el9.x86_64 5.1 MB/s | 911 kB 00:00 2024-05-11T08:03:23.689 INFO:teuthology.orchestra.run.smithi204.stdout:(72/156): protobuf-3.14.0-13.el9.x86_64.rpm 12 MB/s | 1.0 MB 00:00 2024-05-11T08:03:23.714 INFO:teuthology.orchestra.run.smithi204.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 87 kB 00:00 2024-05-11T08:03:23.758 INFO:teuthology.orchestra.run.smithi204.stdout:(74/156): python3-devel-3.9.18-3.el9.x86_64.rpm 5.7 MB/s | 244 kB 00:00 2024-05-11T08:03:23.831 INFO:teuthology.orchestra.run.smithi204.stdout:(75/156): postgresql-server-13.14-1.el9.x86_64. 23 MB/s | 5.8 MB 00:00 2024-05-11T08:03:23.857 INFO:teuthology.orchestra.run.smithi204.stdout:(76/156): python3-jinja2-2.11.3-5.el9.noarch.rp 2.4 MB/s | 248 kB 00:00 2024-05-11T08:03:23.882 INFO:teuthology.orchestra.run.smithi204.stdout:(77/156): python3-jsonpatch-1.21-16.el9.noarch. 511 kB/s | 26 kB 00:00 2024-05-11T08:03:23.907 INFO:teuthology.orchestra.run.smithi204.stdout:(78/156): python3-jsonpointer-2.0-4.el9.noarch. 382 kB/s | 19 kB 00:00 2024-05-11T08:03:23.932 INFO:teuthology.orchestra.run.smithi204.stdout:(79/156): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 172 kB 00:00 2024-05-11T08:03:23.958 INFO:teuthology.orchestra.run.smithi204.stdout:(80/156): python3-markupsafe-1.1.1-12.el9.x86_6 691 kB/s | 35 kB 00:00 2024-05-11T08:03:24.033 INFO:teuthology.orchestra.run.smithi204.stdout:(81/156): python3-babel-2.9.1-2.el9.noarch.rpm 17 MB/s | 6.0 MB 00:00 2024-05-11T08:03:24.058 INFO:teuthology.orchestra.run.smithi204.stdout:(82/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.4 MB/s | 250 kB 00:00 2024-05-11T08:03:24.083 INFO:teuthology.orchestra.run.smithi204.stdout:(83/156): python3-oauthlib-3.1.1-5.el9.noarch.r 4.3 MB/s | 222 kB 00:00 2024-05-11T08:03:24.101 INFO:teuthology.orchestra.run.smithi204.stdout:(84/156): python3-packaging-20.9-5.el9.noarch.r 1.8 MB/s | 77 kB 00:00 2024-05-11T08:03:24.117 INFO:teuthology.orchestra.run.smithi204.stdout:(85/156): python3-prettytable-0.7.2-27.el9.noar 1.2 MB/s | 42 kB 00:00 2024-05-11T08:03:24.151 INFO:teuthology.orchestra.run.smithi204.stdout:(86/156): python3-protobuf-3.14.0-13.el9.noarch 5.2 MB/s | 269 kB 00:00 2024-05-11T08:03:24.168 INFO:teuthology.orchestra.run.smithi204.stdout:(87/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-05-11T08:03:24.193 INFO:teuthology.orchestra.run.smithi204.stdout:(88/156): python3-pyasn1-modules-0.4.8-6.el9.no 6.5 MB/s | 279 kB 00:00 2024-05-11T08:03:24.219 INFO:teuthology.orchestra.run.smithi204.stdout:(89/156): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-05-11T08:03:24.244 INFO:teuthology.orchestra.run.smithi204.stdout:(90/156): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-05-11T08:03:24.320 INFO:teuthology.orchestra.run.smithi204.stdout:(91/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 13 MB/s | 5.1 MB 00:00 2024-05-11T08:03:24.345 INFO:teuthology.orchestra.run.smithi204.stdout:(92/156): python3-toml-0.10.2-6.el9.noarch.rpm 416 kB/s | 42 kB 00:00 2024-05-11T08:03:24.370 INFO:teuthology.orchestra.run.smithi204.stdout:(93/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 881 kB/s | 44 kB 00:00 2024-05-11T08:03:24.395 INFO:teuthology.orchestra.run.smithi204.stdout:(94/156): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-05-11T08:03:24.421 INFO:teuthology.orchestra.run.smithi204.stdout:(95/156): qatlib-service-24.02.0-1.el9.x86_64.r 715 kB/s | 36 kB 00:00 2024-05-11T08:03:24.446 INFO:teuthology.orchestra.run.smithi204.stdout:(96/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 946 kB/s | 47 kB 00:00 2024-05-11T08:03:24.471 INFO:teuthology.orchestra.run.smithi204.stdout:(97/156): socat-1.7.4.1-5.el9.x86_64.rpm 5.9 MB/s | 305 kB 00:00 2024-05-11T08:03:24.497 INFO:teuthology.orchestra.run.smithi204.stdout:(98/156): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-05-11T08:03:24.522 INFO:teuthology.orchestra.run.smithi204.stdout:(99/156): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.8 MB/s | 94 kB 00:00 2024-05-11T08:03:24.547 INFO:teuthology.orchestra.run.smithi204.stdout:(100/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-05-11T08:03:24.572 INFO:teuthology.orchestra.run.smithi204.stdout:(101/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 889 kB/s | 45 kB 00:00 2024-05-11T08:03:24.865 INFO:teuthology.orchestra.run.smithi204.stdout:(102/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 25 MB/s | 16 MB 00:00 2024-05-11T08:03:25.215 INFO:teuthology.orchestra.run.smithi204.stdout:(103/156): abseil-cpp-20211102.0-3.el9.x86_64.r 1.5 MB/s | 548 kB 00:00 2024-05-11T08:03:25.290 INFO:teuthology.orchestra.run.smithi204.stdout:(104/156): gperftools-libs-2.9.1-2.el9.x86_64.r 4.0 MB/s | 309 kB 00:00 2024-05-11T08:03:25.349 INFO:teuthology.orchestra.run.smithi204.stdout:(105/156): grpc-data-1.46.7-10.el9.noarch.rpm 332 kB/s | 19 kB 00:00 2024-05-11T08:03:25.391 INFO:teuthology.orchestra.run.smithi204.stdout:(106/156): lua-devel-5.4.4-4.el9.x86_64.rpm 26 kB/s | 22 kB 00:00 2024-05-11T08:03:25.524 INFO:teuthology.orchestra.run.smithi167.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-05-11T08:03:25.591 INFO:teuthology.orchestra.run.smithi204.stdout:(107/156): libarrow-9.0.0-11.el9.x86_64.rpm 18 MB/s | 4.4 MB 00:00 2024-05-11T08:03:25.617 INFO:teuthology.orchestra.run.smithi204.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 115 kB/s | 26 kB 00:00 2024-05-11T08:03:25.650 INFO:teuthology.orchestra.run.smithi204.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 831 kB/s | 49 kB 00:00 2024-05-11T08:03:25.709 INFO:teuthology.orchestra.run.smithi204.stdout:(110/156): luarocks-3.9.2-1.el9.noarch.rpm 2.5 MB/s | 151 kB 00:00 2024-05-11T08:03:25.734 INFO:teuthology.orchestra.run.smithi204.stdout:(111/156): libunwind-1.6.2-1.el9.x86_64.rpm 574 kB/s | 67 kB 00:00 2024-05-11T08:03:25.785 INFO:teuthology.orchestra.run.smithi204.stdout:(112/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 11 MB/s | 839 kB 00:00 2024-05-11T08:03:25.827 INFO:teuthology.orchestra.run.smithi204.stdout:(113/156): protobuf-compiler-3.14.0-13.el9.x86_ 689 kB/s | 863 kB 00:01 2024-05-11T08:03:25.852 INFO:teuthology.orchestra.run.smithi204.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 648 kB/s | 43 kB 00:00 2024-05-11T08:03:25.911 INFO:teuthology.orchestra.run.smithi204.stdout:(115/156): python3-asyncssh-2.13.2-5.el9.noarch 3.0 MB/s | 548 kB 00:00 2024-05-11T08:03:25.936 INFO:teuthology.orchestra.run.smithi204.stdout:(116/156): python3-certifi-2021.10.8-2.el9.noar 179 kB/s | 15 kB 00:00 2024-05-11T08:03:25.978 INFO:teuthology.orchestra.run.smithi204.stdout:(117/156): python3-cheroot-8.6.0-4.el9.noarch.r 2.5 MB/s | 172 kB 00:00 2024-05-11T08:03:26.007 INFO:teuthology.orchestra.run.smithi204.stdout:(118/156): python3-cherrypy-18.6.1-2.el9.noarch 5.3 MB/s | 358 kB 00:00 2024-05-11T08:03:26.020 INFO:teuthology.orchestra.run.smithi204.stdout:(119/156): python3-cachetools-4.2.4-1.el9.noarc 167 kB/s | 32 kB 00:00 2024-05-11T08:03:26.045 INFO:teuthology.orchestra.run.smithi204.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 4.4 MB/s | 303 kB 00:00 2024-05-11T08:03:26.063 INFO:teuthology.orchestra.run.smithi204.stdout:(121/156): python3-colorama-0.4.4-12.el9.noarch 599 kB/s | 35 kB 00:00 2024-05-11T08:03:26.121 INFO:teuthology.orchestra.run.smithi204.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 2.4 MB/s | 144 kB 00:00 2024-05-11T08:03:26.172 INFO:teuthology.orchestra.run.smithi204.stdout:(123/156): python3-grpcio-1.46.7-10.el9.x86_64. 16 MB/s | 2.0 MB 00:00 2024-05-11T08:03:26.197 INFO:teuthology.orchestra.run.smithi204.stdout:(124/156): python3-jaraco-8.2.1-3.el9.noarch.rp 142 kB/s | 11 kB 00:00 2024-05-11T08:03:26.222 INFO:teuthology.orchestra.run.smithi204.stdout:(125/156): python3-google-auth-2.29.0-1.el9.noa 1.0 MB/s | 206 kB 00:00 2024-05-11T08:03:26.248 INFO:teuthology.orchestra.run.smithi204.stdout:(126/156): python3-jaraco-classes-3.2.1-5.el9.n 234 kB/s | 18 kB 00:00 2024-05-11T08:03:26.273 INFO:teuthology.orchestra.run.smithi204.stdout:(127/156): python3-jaraco-collections-3.0.0-8.e 307 kB/s | 23 kB 00:00 2024-05-11T08:03:26.298 INFO:teuthology.orchestra.run.smithi204.stdout:(128/156): python3-jaraco-functools-3.5.0-2.el9 257 kB/s | 19 kB 00:00 2024-05-11T08:03:26.323 INFO:teuthology.orchestra.run.smithi204.stdout:(129/156): python3-jaraco-text-3.2.0-6.el9.noar 260 kB/s | 20 kB 00:00 2024-05-11T08:03:26.357 INFO:teuthology.orchestra.run.smithi204.stdout:(130/156): python3-logutils-0.3.5-21.el9.noarch 791 kB/s | 46 kB 00:00 2024-05-11T08:03:26.382 INFO:teuthology.orchestra.run.smithi204.stdout:(131/156): python3-more-itertools-8.12.0-2.el9. 1.3 MB/s | 79 kB 00:00 2024-05-11T08:03:26.416 INFO:teuthology.orchestra.run.smithi204.stdout:(132/156): python3-kubernetes-26.1.0-2.el9.noar 7.2 MB/s | 1.0 MB 00:00 2024-05-11T08:03:26.441 INFO:teuthology.orchestra.run.smithi204.stdout:(133/156): python3-natsort-7.1.1-5.el9.noarch.r 687 kB/s | 58 kB 00:00 2024-05-11T08:03:26.466 INFO:teuthology.orchestra.run.smithi204.stdout:(134/156): python3-pecan-1.4.2-3.el9.noarch.rpm 3.2 MB/s | 272 kB 00:00 2024-05-11T08:03:26.492 INFO:teuthology.orchestra.run.smithi204.stdout:(135/156): python3-portend-3.1.0-2.el9.noarch.r 217 kB/s | 16 kB 00:00 2024-05-11T08:03:26.525 INFO:teuthology.orchestra.run.smithi204.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 481 kB/s | 28 kB 00:00 2024-05-11T08:03:26.551 INFO:teuthology.orchestra.run.smithi204.stdout:(137/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 823 kB/s | 90 kB 00:00 2024-05-11T08:03:26.567 INFO:teuthology.orchestra.run.smithi204.stdout:(138/156): python3-repoze-lru-0.7-16.el9.noarch 407 kB/s | 31 kB 00:00 2024-05-11T08:03:26.593 INFO:teuthology.orchestra.run.smithi204.stdout:(139/156): python3-routes-2.5.1-5.el9.noarch.rp 2.7 MB/s | 188 kB 00:00 2024-05-11T08:03:26.618 INFO:teuthology.orchestra.run.smithi204.stdout:(140/156): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2024-05-11T08:03:26.635 INFO:teuthology.orchestra.run.smithi204.stdout:(141/156): python3-tempora-5.0.0-2.el9.noarch.r 534 kB/s | 36 kB 00:00 2024-05-11T08:03:26.652 INFO:teuthology.orchestra.run.smithi204.stdout:(142/156): python3-typing-extensions-4.4.0-2.el 868 kB/s | 51 kB 00:00 2024-05-11T08:03:26.677 INFO:teuthology.orchestra.run.smithi204.stdout:(143/156): python3-webob-1.8.7-6.el9.noarch.rpm 3.8 MB/s | 230 kB 00:00 2024-05-11T08:03:26.711 INFO:teuthology.orchestra.run.smithi204.stdout:(144/156): python3-werkzeug-2.0.3-3.el9.1.noarc 7.1 MB/s | 427 kB 00:00 2024-05-11T08:03:26.745 INFO:teuthology.orchestra.run.smithi204.stdout:(145/156): python3-websocket-client-1.2.3-2.el9 820 kB/s | 90 kB 00:00 2024-05-11T08:03:26.770 INFO:teuthology.orchestra.run.smithi204.stdout:(146/156): python3-zc-lockfile-2.0-10.el9.noarc 216 kB/s | 20 kB 00:00 2024-05-11T08:03:26.795 INFO:teuthology.orchestra.run.smithi204.stdout:(147/156): re2-20211101-3.el9.x86_64.rpm 2.2 MB/s | 192 kB 00:00 2024-05-11T08:03:26.912 INFO:teuthology.orchestra.run.smithi204.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-05-11T08:03:27.104 INFO:teuthology.orchestra.run.smithi204.stdout:(149/156): NetworkManager-1.47.5-1.el9.x86_64.r 12 MB/s | 2.3 MB 00:00 2024-05-11T08:03:27.171 INFO:teuthology.orchestra.run.smithi204.stdout:(150/156): librbd1-19.0.0-3601.g30850680.el9.x8 8.4 MB/s | 3.1 MB 00:00 2024-05-11T08:03:27.221 INFO:teuthology.orchestra.run.smithi204.stdout:(151/156): NetworkManager-libnm-1.47.5-1.el9.x8 16 MB/s | 1.8 MB 00:00 2024-05-11T08:03:27.263 INFO:teuthology.orchestra.run.smithi204.stdout:(152/156): NetworkManager-tui-1.47.5-1.el9.x86_ 5.8 MB/s | 248 kB 00:00 2024-05-11T08:03:27.289 INFO:teuthology.orchestra.run.smithi204.stdout:(153/156): NetworkManager-team-1.47.5-1.el9.x86 356 kB/s | 42 kB 00:00 2024-05-11T08:03:27.347 INFO:teuthology.orchestra.run.smithi204.stdout:(154/156): openssl-3.2.1-1.el9.x86_64.rpm 16 MB/s | 1.3 MB 00:00 2024-05-11T08:03:27.473 INFO:teuthology.orchestra.run.smithi204.stdout:(155/156): openssl-libs-3.2.1-1.el9.x86_64.rpm 13 MB/s | 2.4 MB 00:00 2024-05-11T08:03:27.664 INFO:teuthology.orchestra.run.smithi204.stdout:(156/156): librados2-19.0.0-3601.g30850680.el9. 3.8 MB/s | 3.4 MB 00:00 2024-05-11T08:03:27.671 INFO:teuthology.orchestra.run.smithi204.stdout:-------------------------------------------------------------------------------- 2024-05-11T08:03:27.671 INFO:teuthology.orchestra.run.smithi204.stdout:Total 13 MB/s | 226 MB 00:17 2024-05-11T08:03:27.851 INFO:teuthology.orchestra.run.smithi167.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-11T08:03:27.851 INFO:teuthology.orchestra.run.smithi167.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-11T08:03:27.862 INFO:teuthology.orchestra.run.smithi167.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-11T08:03:27.863 INFO:teuthology.orchestra.run.smithi167.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-11T08:03:27.864 INFO:teuthology.orchestra.run.smithi167.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-11T08:03:27.864 INFO:teuthology.orchestra.run.smithi167.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-11T08:03:27.866 INFO:teuthology.orchestra.run.smithi167.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-11T08:03:27.868 INFO:teuthology.orchestra.run.smithi167.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-11T08:03:27.869 INFO:teuthology.orchestra.run.smithi167.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-11T08:03:27.929 INFO:teuthology.orchestra.run.smithi167.stdout:Dependencies resolved. 2024-05-11T08:03:27.946 INFO:teuthology.orchestra.run.smithi167.stdout:======================================================================================= 2024-05-11T08:03:27.947 INFO:teuthology.orchestra.run.smithi167.stdout: Package Arch Version Repository Size 2024-05-11T08:03:27.947 INFO:teuthology.orchestra.run.smithi167.stdout:======================================================================================= 2024-05-11T08:03:27.947 INFO:teuthology.orchestra.run.smithi167.stdout:Installing: 2024-05-11T08:03:27.947 INFO:teuthology.orchestra.run.smithi167.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-11T08:03:27.947 INFO:teuthology.orchestra.run.smithi167.stdout: ceph x86_64 2:19.0.0-3601.g30850680.el9 ceph 6.6 k 2024-05-11T08:03:27.947 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-base x86_64 2:19.0.0-3601.g30850680.el9 ceph 5.4 M 2024-05-11T08:03:27.947 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-fuse x86_64 2:19.0.0-3601.g30850680.el9 ceph 890 k 2024-05-11T08:03:27.947 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3601.g30850680.el9 ceph 145 k 2024-05-11T08:03:27.947 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr x86_64 2:19.0.0-3601.g30850680.el9 ceph 1.5 M 2024-05-11T08:03:27.948 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3601.g30850680.el9 ceph-noarch 142 k 2024-05-11T08:03:27.948 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3601.g30850680.el9 ceph-noarch 3.5 M 2024-05-11T08:03:27.948 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3601.g30850680.el9 ceph-noarch 7.4 M 2024-05-11T08:03:27.948 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-rook noarch 2:19.0.0-3601.g30850680.el9 ceph-noarch 49 k 2024-05-11T08:03:27.948 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-radosgw x86_64 2:19.0.0-3601.g30850680.el9 ceph 11 M 2024-05-11T08:03:27.948 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-test x86_64 2:19.0.0-3601.g30850680.el9 ceph 49 M 2024-05-11T08:03:27.948 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-volume noarch 2:19.0.0-3601.g30850680.el9 ceph-noarch 273 k 2024-05-11T08:03:27.948 INFO:teuthology.orchestra.run.smithi167.stdout: cephadm noarch 2:19.0.0-3601.g30850680.el9 ceph-noarch 764 k 2024-05-11T08:03:27.948 INFO:teuthology.orchestra.run.smithi167.stdout: cephfs-mirror x86_64 2:19.0.0-3601.g30850680.el9 ceph 225 k 2024-05-11T08:03:27.949 INFO:teuthology.orchestra.run.smithi167.stdout: cephfs-shell x86_64 2:19.0.0-3601.g30850680.el9 ceph 27 k 2024-05-11T08:03:27.949 INFO:teuthology.orchestra.run.smithi167.stdout: cephfs-top noarch 2:19.0.0-3601.g30850680.el9 ceph-noarch 18 k 2024-05-11T08:03:27.949 INFO:teuthology.orchestra.run.smithi167.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-05-11T08:03:27.949 INFO:teuthology.orchestra.run.smithi167.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-11T08:03:27.949 INFO:teuthology.orchestra.run.smithi167.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-11T08:03:27.949 INFO:teuthology.orchestra.run.smithi167.stdout: libcephfs-devel x86_64 2:19.0.0-3601.g30850680.el9 ceph 32 k 2024-05-11T08:03:27.949 INFO:teuthology.orchestra.run.smithi167.stdout: libcephfs2 x86_64 2:19.0.0-3601.g30850680.el9 ceph 746 k 2024-05-11T08:03:27.949 INFO:teuthology.orchestra.run.smithi167.stdout: librados-devel x86_64 2:19.0.0-3601.g30850680.el9 ceph 127 k 2024-05-11T08:03:27.949 INFO:teuthology.orchestra.run.smithi167.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-11T08:03:27.950 INFO:teuthology.orchestra.run.smithi167.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-11T08:03:27.950 INFO:teuthology.orchestra.run.smithi167.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-11T08:03:27.950 INFO:teuthology.orchestra.run.smithi167.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-11T08:03:27.950 INFO:teuthology.orchestra.run.smithi167.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-11T08:03:27.950 INFO:teuthology.orchestra.run.smithi167.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-11T08:03:27.950 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cephfs x86_64 2:19.0.0-3601.g30850680.el9 ceph 161 k 2024-05-11T08:03:27.950 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rados x86_64 2:19.0.0-3601.g30850680.el9 ceph 321 k 2024-05-11T08:03:27.950 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rbd x86_64 2:19.0.0-3601.g30850680.el9 ceph 300 k 2024-05-11T08:03:27.951 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rgw x86_64 2:19.0.0-3601.g30850680.el9 ceph 100 k 2024-05-11T08:03:27.951 INFO:teuthology.orchestra.run.smithi167.stdout: rbd-fuse x86_64 2:19.0.0-3601.g30850680.el9 ceph 86 k 2024-05-11T08:03:27.951 INFO:teuthology.orchestra.run.smithi167.stdout: rbd-mirror x86_64 2:19.0.0-3601.g30850680.el9 ceph 3.1 M 2024-05-11T08:03:27.951 INFO:teuthology.orchestra.run.smithi167.stdout: rbd-nbd x86_64 2:19.0.0-3601.g30850680.el9 ceph 171 k 2024-05-11T08:03:27.951 INFO:teuthology.orchestra.run.smithi167.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-11T08:03:27.951 INFO:teuthology.orchestra.run.smithi167.stdout:Upgrading: 2024-05-11T08:03:27.951 INFO:teuthology.orchestra.run.smithi167.stdout: NetworkManager x86_64 1:1.47.5-1.el9 baseos 2.3 M 2024-05-11T08:03:27.951 INFO:teuthology.orchestra.run.smithi167.stdout: NetworkManager-libnm x86_64 1:1.47.5-1.el9 baseos 1.8 M 2024-05-11T08:03:27.952 INFO:teuthology.orchestra.run.smithi167.stdout: NetworkManager-team x86_64 1:1.47.5-1.el9 baseos 42 k 2024-05-11T08:03:27.952 INFO:teuthology.orchestra.run.smithi167.stdout: NetworkManager-tui x86_64 1:1.47.5-1.el9 baseos 248 k 2024-05-11T08:03:27.952 INFO:teuthology.orchestra.run.smithi167.stdout: librados2 x86_64 2:19.0.0-3601.g30850680.el9 ceph 3.4 M 2024-05-11T08:03:27.952 INFO:teuthology.orchestra.run.smithi167.stdout: librbd1 x86_64 2:19.0.0-3601.g30850680.el9 ceph 3.1 M 2024-05-11T08:03:27.952 INFO:teuthology.orchestra.run.smithi167.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-11T08:03:27.952 INFO:teuthology.orchestra.run.smithi167.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-11T08:03:27.952 INFO:teuthology.orchestra.run.smithi167.stdout:Installing dependencies: 2024-05-11T08:03:27.952 INFO:teuthology.orchestra.run.smithi167.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-11T08:03:27.952 INFO:teuthology.orchestra.run.smithi167.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-11T08:03:27.953 INFO:teuthology.orchestra.run.smithi167.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-11T08:03:27.953 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-common x86_64 2:19.0.0-3601.g30850680.el9 ceph 21 M 2024-05-11T08:03:27.953 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3601.g30850680.el9 ceph-noarch 34 k 2024-05-11T08:03:27.953 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mds x86_64 2:19.0.0-3601.g30850680.el9 ceph 2.4 M 2024-05-11T08:03:27.953 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3601.g30850680.el9 ceph-noarch 274 k 2024-05-11T08:03:27.953 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mon x86_64 2:19.0.0-3601.g30850680.el9 ceph 4.7 M 2024-05-11T08:03:27.953 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-osd x86_64 2:19.0.0-3601.g30850680.el9 ceph 17 M 2024-05-11T08:03:27.953 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3601.g30850680.el9 ceph-noarch 16 k 2024-05-11T08:03:27.953 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-selinux x86_64 2:19.0.0-3601.g30850680.el9 ceph 25 k 2024-05-11T08:03:27.953 INFO:teuthology.orchestra.run.smithi167.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-11T08:03:27.954 INFO:teuthology.orchestra.run.smithi167.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-11T08:03:27.954 INFO:teuthology.orchestra.run.smithi167.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-11T08:03:27.954 INFO:teuthology.orchestra.run.smithi167.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-11T08:03:27.954 INFO:teuthology.orchestra.run.smithi167.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-11T08:03:27.954 INFO:teuthology.orchestra.run.smithi167.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-11T08:03:27.954 INFO:teuthology.orchestra.run.smithi167.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-11T08:03:27.954 INFO:teuthology.orchestra.run.smithi167.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-11T08:03:27.954 INFO:teuthology.orchestra.run.smithi167.stdout: libcephsqlite x86_64 2:19.0.0-3601.g30850680.el9 ceph 163 k 2024-05-11T08:03:27.954 INFO:teuthology.orchestra.run.smithi167.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-11T08:03:27.954 INFO:teuthology.orchestra.run.smithi167.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-11T08:03:27.955 INFO:teuthology.orchestra.run.smithi167.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-11T08:03:27.955 INFO:teuthology.orchestra.run.smithi167.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-11T08:03:27.955 INFO:teuthology.orchestra.run.smithi167.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-11T08:03:27.955 INFO:teuthology.orchestra.run.smithi167.stdout: libradosstriper1 x86_64 2:19.0.0-3601.g30850680.el9 ceph 501 k 2024-05-11T08:03:27.955 INFO:teuthology.orchestra.run.smithi167.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-11T08:03:27.955 INFO:teuthology.orchestra.run.smithi167.stdout: librgw2 x86_64 2:19.0.0-3601.g30850680.el9 ceph 5.4 M 2024-05-11T08:03:27.955 INFO:teuthology.orchestra.run.smithi167.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-11T08:03:27.955 INFO:teuthology.orchestra.run.smithi167.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-11T08:03:27.955 INFO:teuthology.orchestra.run.smithi167.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-11T08:03:27.956 INFO:teuthology.orchestra.run.smithi167.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-11T08:03:27.956 INFO:teuthology.orchestra.run.smithi167.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-11T08:03:27.956 INFO:teuthology.orchestra.run.smithi167.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-05-11T08:03:27.956 INFO:teuthology.orchestra.run.smithi167.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-11T08:03:27.956 INFO:teuthology.orchestra.run.smithi167.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-11T08:03:27.956 INFO:teuthology.orchestra.run.smithi167.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-11T08:03:27.956 INFO:teuthology.orchestra.run.smithi167.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-11T08:03:27.956 INFO:teuthology.orchestra.run.smithi167.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-11T08:03:27.956 INFO:teuthology.orchestra.run.smithi167.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-11T08:03:27.956 INFO:teuthology.orchestra.run.smithi167.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-11T08:03:27.957 INFO:teuthology.orchestra.run.smithi167.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-11T08:03:27.957 INFO:teuthology.orchestra.run.smithi167.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-11T08:03:27.957 INFO:teuthology.orchestra.run.smithi167.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-11T08:03:27.957 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-11T08:03:27.957 INFO:teuthology.orchestra.run.smithi167.stdout: python3-ceph-argparse x86_64 2:19.0.0-3601.g30850680.el9 ceph 45 k 2024-05-11T08:03:27.957 INFO:teuthology.orchestra.run.smithi167.stdout: python3-ceph-common x86_64 2:19.0.0-3601.g30850680.el9 ceph 135 k 2024-05-11T08:03:27.957 INFO:teuthology.orchestra.run.smithi167.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-11T08:03:27.957 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-11T08:03:27.957 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-11T08:03:27.958 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-11T08:03:27.958 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-11T08:03:27.958 INFO:teuthology.orchestra.run.smithi167.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-05-11T08:03:27.958 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-11T08:03:27.958 INFO:teuthology.orchestra.run.smithi167.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-11T08:03:27.958 INFO:teuthology.orchestra.run.smithi167.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-11T08:03:27.958 INFO:teuthology.orchestra.run.smithi167.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-11T08:03:27.958 INFO:teuthology.orchestra.run.smithi167.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-11T08:03:27.958 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-11T08:03:27.958 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-11T08:03:27.958 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-11T08:03:27.959 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-11T08:03:27.959 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-11T08:03:27.959 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-11T08:03:27.959 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-11T08:03:27.959 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-11T08:03:27.959 INFO:teuthology.orchestra.run.smithi167.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-11T08:03:27.959 INFO:teuthology.orchestra.run.smithi167.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-11T08:03:27.959 INFO:teuthology.orchestra.run.smithi167.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-11T08:03:27.959 INFO:teuthology.orchestra.run.smithi167.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-11T08:03:27.959 INFO:teuthology.orchestra.run.smithi167.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-11T08:03:27.959 INFO:teuthology.orchestra.run.smithi167.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-11T08:03:27.959 INFO:teuthology.orchestra.run.smithi167.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-11T08:03:27.960 INFO:teuthology.orchestra.run.smithi167.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-11T08:03:27.960 INFO:teuthology.orchestra.run.smithi167.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-11T08:03:27.960 INFO:teuthology.orchestra.run.smithi167.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-11T08:03:27.960 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-11T08:03:27.960 INFO:teuthology.orchestra.run.smithi167.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-11T08:03:27.960 INFO:teuthology.orchestra.run.smithi167.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-11T08:03:27.960 INFO:teuthology.orchestra.run.smithi167.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-11T08:03:27.960 INFO:teuthology.orchestra.run.smithi167.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-11T08:03:27.960 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-11T08:03:27.960 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-11T08:03:27.960 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-11T08:03:27.960 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-11T08:03:27.961 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-11T08:03:27.961 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-11T08:03:27.961 INFO:teuthology.orchestra.run.smithi167.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-11T08:03:27.961 INFO:teuthology.orchestra.run.smithi167.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-11T08:03:27.961 INFO:teuthology.orchestra.run.smithi167.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-11T08:03:27.961 INFO:teuthology.orchestra.run.smithi167.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-11T08:03:27.961 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-11T08:03:27.961 INFO:teuthology.orchestra.run.smithi167.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-11T08:03:27.961 INFO:teuthology.orchestra.run.smithi167.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-11T08:03:27.961 INFO:teuthology.orchestra.run.smithi167.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-11T08:03:27.961 INFO:teuthology.orchestra.run.smithi167.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-11T08:03:27.961 INFO:teuthology.orchestra.run.smithi167.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-11T08:03:27.962 INFO:teuthology.orchestra.run.smithi167.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-11T08:03:27.962 INFO:teuthology.orchestra.run.smithi167.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-11T08:03:27.962 INFO:teuthology.orchestra.run.smithi167.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-11T08:03:27.962 INFO:teuthology.orchestra.run.smithi167.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-11T08:03:27.962 INFO:teuthology.orchestra.run.smithi167.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-11T08:03:27.962 INFO:teuthology.orchestra.run.smithi167.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-11T08:03:27.962 INFO:teuthology.orchestra.run.smithi167.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-11T08:03:27.962 INFO:teuthology.orchestra.run.smithi167.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-11T08:03:27.962 INFO:teuthology.orchestra.run.smithi167.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-11T08:03:27.962 INFO:teuthology.orchestra.run.smithi167.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-11T08:03:27.962 INFO:teuthology.orchestra.run.smithi167.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-11T08:03:27.962 INFO:teuthology.orchestra.run.smithi167.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-11T08:03:27.963 INFO:teuthology.orchestra.run.smithi167.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-11T08:03:27.963 INFO:teuthology.orchestra.run.smithi167.stdout:Installing weak dependencies: 2024-05-11T08:03:27.963 INFO:teuthology.orchestra.run.smithi167.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-11T08:03:27.963 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T08:03:27.963 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction Summary 2024-05-11T08:03:27.963 INFO:teuthology.orchestra.run.smithi167.stdout:======================================================================================= 2024-05-11T08:03:27.963 INFO:teuthology.orchestra.run.smithi167.stdout:Install 148 Packages 2024-05-11T08:03:27.963 INFO:teuthology.orchestra.run.smithi167.stdout:Upgrade 8 Packages 2024-05-11T08:03:27.963 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T08:03:27.964 INFO:teuthology.orchestra.run.smithi167.stdout:Total download size: 226 M 2024-05-11T08:03:27.964 INFO:teuthology.orchestra.run.smithi167.stdout:Downloading Packages: 2024-05-11T08:03:28.493 INFO:teuthology.orchestra.run.smithi167.stdout:(1/156): ceph-19.0.0-3601.g30850680.el9.x86_64. 37 kB/s | 6.6 kB 00:00 2024-05-11T08:03:28.760 INFO:teuthology.orchestra.run.smithi167.stdout:(2/156): ceph-fuse-19.0.0-3601.g30850680.el9.x8 3.3 MB/s | 890 kB 00:00 2024-05-11T08:03:28.888 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction check 2024-05-11T08:03:28.900 INFO:teuthology.orchestra.run.smithi167.stdout:(3/156): ceph-base-19.0.0-3601.g30850680.el9.x8 9.2 MB/s | 5.4 MB 00:00 2024-05-11T08:03:29.041 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction check succeeded. 2024-05-11T08:03:29.041 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction test 2024-05-11T08:03:29.043 INFO:teuthology.orchestra.run.smithi167.stdout:(4/156): ceph-immutable-object-cache-19.0.0-360 510 kB/s | 145 kB 00:00 2024-05-11T08:03:29.094 INFO:teuthology.orchestra.run.smithi167.stdout:(5/156): ceph-mds-19.0.0-3601.g30850680.el9.x86 12 MB/s | 2.4 MB 00:00 2024-05-11T08:03:29.211 INFO:teuthology.orchestra.run.smithi167.stdout:(6/156): ceph-mgr-19.0.0-3601.g30850680.el9.x86 9.1 MB/s | 1.5 MB 00:00 2024-05-11T08:03:29.395 INFO:teuthology.orchestra.run.smithi167.stdout:(7/156): ceph-mon-19.0.0-3601.g30850680.el9.x86 16 MB/s | 4.7 MB 00:00 2024-05-11T08:03:30.120 INFO:teuthology.orchestra.run.smithi167.stdout:(8/156): ceph-radosgw-19.0.0-3601.g30850680.el9 15 MB/s | 11 MB 00:00 2024-05-11T08:03:30.153 INFO:teuthology.orchestra.run.smithi167.stdout:(9/156): ceph-selinux-19.0.0-3601.g30850680.el9 758 kB/s | 25 kB 00:00 2024-05-11T08:03:31.230 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction test succeeded. 2024-05-11T08:03:31.230 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction 2024-05-11T08:03:31.287 INFO:teuthology.orchestra.run.smithi167.stdout:(10/156): ceph-osd-19.0.0-3601.g30850680.el9.x8 8.1 MB/s | 17 MB 00:02 2024-05-11T08:03:31.346 INFO:teuthology.orchestra.run.smithi167.stdout:(11/156): cephfs-mirror-19.0.0-3601.g30850680.e 3.8 MB/s | 225 kB 00:00 2024-05-11T08:03:31.404 INFO:teuthology.orchestra.run.smithi167.stdout:(12/156): cephfs-shell-19.0.0-3601.g30850680.el 464 kB/s | 27 kB 00:00 2024-05-11T08:03:31.471 INFO:teuthology.orchestra.run.smithi167.stdout:(13/156): libcephfs-devel-19.0.0-3601.g30850680 476 kB/s | 32 kB 00:00 2024-05-11T08:03:31.621 INFO:teuthology.orchestra.run.smithi167.stdout:(14/156): libcephfs2-19.0.0-3601.g30850680.el9. 4.9 MB/s | 746 kB 00:00 2024-05-11T08:03:31.705 INFO:teuthology.orchestra.run.smithi167.stdout:(15/156): libcephsqlite-19.0.0-3601.g30850680.e 1.9 MB/s | 163 kB 00:00 2024-05-11T08:03:31.780 INFO:teuthology.orchestra.run.smithi167.stdout:(16/156): librados-devel-19.0.0-3601.g30850680. 1.7 MB/s | 127 kB 00:00 2024-05-11T08:03:31.881 INFO:teuthology.orchestra.run.smithi167.stdout:(17/156): libradosstriper1-19.0.0-3601.g3085068 4.9 MB/s | 501 kB 00:00 2024-05-11T08:03:32.656 INFO:teuthology.orchestra.run.smithi167.stdout:(18/156): librgw2-19.0.0-3601.g30850680.el9.x86 6.9 MB/s | 5.4 MB 00:00 2024-05-11T08:03:32.731 INFO:teuthology.orchestra.run.smithi167.stdout:(19/156): python3-ceph-argparse-19.0.0-3601.g30 603 kB/s | 45 kB 00:00 2024-05-11T08:03:33.215 INFO:teuthology.orchestra.run.smithi167.stdout:(20/156): ceph-test-19.0.0-3601.g30850680.el9.x 16 MB/s | 49 MB 00:03 2024-05-11T08:03:33.240 INFO:teuthology.orchestra.run.smithi167.stdout:(21/156): python3-ceph-common-19.0.0-3601.g3085 266 kB/s | 135 kB 00:00 2024-05-11T08:03:33.266 INFO:teuthology.orchestra.run.smithi167.stdout:(22/156): python3-cephfs-19.0.0-3601.g30850680. 3.2 MB/s | 161 kB 00:00 2024-05-11T08:03:33.308 INFO:teuthology.orchestra.run.smithi167.stdout:(23/156): python3-rados-19.0.0-3601.g30850680.e 4.7 MB/s | 321 kB 00:00 2024-05-11T08:03:33.333 INFO:teuthology.orchestra.run.smithi167.stdout:(24/156): python3-rbd-19.0.0-3601.g30850680.el9 4.4 MB/s | 300 kB 00:00 2024-05-11T08:03:33.383 INFO:teuthology.orchestra.run.smithi167.stdout:(25/156): rbd-fuse-19.0.0-3601.g30850680.el9.x8 1.7 MB/s | 86 kB 00:00 2024-05-11T08:03:33.592 INFO:teuthology.orchestra.run.smithi167.stdout:(26/156): rbd-mirror-19.0.0-3601.g30850680.el9. 15 MB/s | 3.1 MB 00:00 2024-05-11T08:03:33.617 INFO:teuthology.orchestra.run.smithi167.stdout:(27/156): python3-rgw-19.0.0-3601.g30850680.el9 322 kB/s | 100 kB 00:00 2024-05-11T08:03:33.634 INFO:teuthology.orchestra.run.smithi167.stdout:(28/156): rbd-nbd-19.0.0-3601.g30850680.el9.x86 4.0 MB/s | 171 kB 00:00 2024-05-11T08:03:33.659 INFO:teuthology.orchestra.run.smithi167.stdout:(29/156): ceph-grafana-dashboards-19.0.0-3601.g 818 kB/s | 34 kB 00:00 2024-05-11T08:03:33.685 INFO:teuthology.orchestra.run.smithi167.stdout:(30/156): ceph-mgr-cephadm-19.0.0-3601.g3085068 2.8 MB/s | 142 kB 00:00 2024-05-11T08:03:34.144 INFO:teuthology.orchestra.run.smithi167.stdout:(31/156): ceph-mgr-dashboard-19.0.0-3601.g30850 7.3 MB/s | 3.5 MB 00:00 2024-05-11T08:03:34.252 INFO:teuthology.orchestra.run.smithi167.stdout:(32/156): ceph-mgr-diskprediction-local-19.0.0- 13 MB/s | 7.4 MB 00:00 2024-05-11T08:03:34.277 INFO:teuthology.orchestra.run.smithi167.stdout:(33/156): ceph-mgr-modules-core-19.0.0-3601.g30 2.0 MB/s | 274 kB 00:00 2024-05-11T08:03:34.294 INFO:teuthology.orchestra.run.smithi167.stdout:(34/156): ceph-mgr-rook-19.0.0-3601.g30850680.e 1.2 MB/s | 49 kB 00:00 2024-05-11T08:03:34.503 INFO:teuthology.orchestra.run.smithi167.stdout:(35/156): ceph-common-19.0.0-3601.g30850680.el9 3.4 MB/s | 21 MB 00:06 2024-05-11T08:03:34.520 INFO:teuthology.orchestra.run.smithi167.stdout:(36/156): ceph-prometheus-alerts-19.0.0-3601.g3 68 kB/s | 16 kB 00:00 2024-05-11T08:03:34.545 INFO:teuthology.orchestra.run.smithi167.stdout:(37/156): ceph-volume-19.0.0-3601.g30850680.el9 1.1 MB/s | 273 kB 00:00 2024-05-11T08:03:34.562 INFO:teuthology.orchestra.run.smithi167.stdout:(38/156): cephfs-top-19.0.0-3601.g30850680.el9. 436 kB/s | 18 kB 00:00 2024-05-11T08:03:34.687 INFO:teuthology.orchestra.run.smithi167.stdout:(39/156): cephadm-19.0.0-3601.g30850680.el9.noa 4.1 MB/s | 764 kB 00:00 2024-05-11T08:03:34.988 INFO:teuthology.orchestra.run.smithi167.stdout:(40/156): fuse-2.9.9-15.el9.x86_64.rpm 182 kB/s | 80 kB 00:00 2024-05-11T08:03:35.188 INFO:teuthology.orchestra.run.smithi167.stdout:(41/156): libquadmath-11.4.1-3.el9.x86_64.rpm 390 kB/s | 195 kB 00:00 2024-05-11T08:03:35.322 INFO:teuthology.orchestra.run.smithi167.stdout:(42/156): libgfortran-11.4.1-3.el9.x86_64.rpm 1.0 MB/s | 803 kB 00:00 2024-05-11T08:03:35.347 INFO:teuthology.orchestra.run.smithi167.stdout:(43/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 705 kB/s | 253 kB 00:00 2024-05-11T08:03:35.447 INFO:teuthology.orchestra.run.smithi167.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 851 kB/s | 106 kB 00:00 2024-05-11T08:03:35.473 INFO:teuthology.orchestra.run.smithi167.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 1.1 MB/s | 135 kB 00:00 2024-05-11T08:03:35.556 INFO:teuthology.orchestra.run.smithi167.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 1.1 MB/s | 125 kB 00:00 2024-05-11T08:03:35.581 INFO:teuthology.orchestra.run.smithi167.stdout:(47/156): python3-urllib3-1.26.5-5.el9.noarch.r 1.9 MB/s | 215 kB 00:00 2024-05-11T08:03:35.723 INFO:teuthology.orchestra.run.smithi167.stdout:(48/156): python3-cryptography-36.0.1-4.el9.x86 2.3 MB/s | 1.2 MB 00:00 2024-05-11T08:03:36.457 INFO:teuthology.orchestra.run.smithi167.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 146 kB/s | 107 kB 00:00 2024-05-11T08:03:36.499 INFO:teuthology.orchestra.run.smithi167.stdout:(50/156): automake-1.16.2-8.el9.noarch.rpm 731 kB/s | 689 kB 00:00 2024-05-11T08:03:36.608 INFO:teuthology.orchestra.run.smithi167.stdout:(51/156): bison-3.7.4-5.el9.x86_64.rpm 919 kB/s | 943 kB 00:01 2024-05-11T08:03:37.058 INFO:teuthology.orchestra.run.smithi167.stdout:(52/156): elfutils-libelf-devel-0.190-2.el9.x86 36 kB/s | 21 kB 00:00 2024-05-11T08:03:37.233 INFO:teuthology.orchestra.run.smithi167.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 52 kB/s | 33 kB 00:00 2024-05-11T08:03:37.367 INFO:teuthology.orchestra.run.smithi167.stdout:(54/156): flex-2.6.4-9.el9.x86_64.rpm 362 kB/s | 313 kB 00:00 2024-05-11T08:03:37.560 INFO:teuthology.orchestra.run.smithi204.stdout: Preparing : 1/1 2024-05-11T08:03:37.636 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/164 2024-05-11T08:03:37.694 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-05-11T08:03:37.769 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-05-11T08:03:37.803 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-05-11T08:03:37.817 INFO:teuthology.orchestra.run.smithi167.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 31 kB/s | 18 kB 00:00 2024-05-11T08:03:37.829 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-05-11T08:03:37.876 INFO:teuthology.orchestra.run.smithi167.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 22 kB/s | 11 kB 00:00 2024-05-11T08:03:37.883 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-05-11T08:03:38.184 INFO:teuthology.orchestra.run.smithi167.stdout:(57/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 2.7 MB/s | 3.0 MB 00:01 2024-05-11T08:03:38.211 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-05-11T08:03:38.321 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : librados2-2:19.0.0-3601.g30850680.el9.x86_64 8/164 2024-05-11T08:03:38.451 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: librados2-2:19.0.0-3601.g30850680.el9.x86_64 8/164 2024-05-11T08:03:38.472 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libcephfs2-2:19.0.0-3601.g30850680.el9.x86_64 9/164 2024-05-11T08:03:38.493 INFO:teuthology.orchestra.run.smithi167.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 73 kB/s | 45 kB 00:00 2024-05-11T08:03:38.564 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: libcephfs2-2:19.0.0-3601.g30850680.el9.x86_64 9/164 2024-05-11T08:03:38.594 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-rados-2:19.0.0-3601.g30850680.el9.x86_64 10/164 2024-05-11T08:03:38.643 INFO:teuthology.orchestra.run.smithi167.stdout:(59/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 194 kB/s | 160 kB 00:00 2024-05-11T08:03:38.769 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/164 2024-05-11T08:03:38.825 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/164 2024-05-11T08:03:38.852 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libcephsqlite-2:19.0.0-3601.g30850680.el9.x86_64 13/164 2024-05-11T08:03:38.940 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3601.g30850680.el9.x86_64 13/164 2024-05-11T08:03:38.962 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libradosstriper1-2:19.0.0-3601.g30850680.el9.x86 14/164 2024-05-11T08:03:39.025 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3601.g30850680.el9.x86 14/164 2024-05-11T08:03:39.054 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-11T08:03:39.074 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-11T08:03:39.110 INFO:teuthology.orchestra.run.smithi167.stdout:(60/156): librdkafka-1.6.1-102.el9.x86_64.rpm 717 kB/s | 662 kB 00:00 2024-05-11T08:03:39.265 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-11T08:03:39.268 INFO:teuthology.orchestra.run.smithi167.stdout:(61/156): libuuid-devel-2.37.4-18.el9.x86_64.rp 41 kB/s | 25 kB 00:00 2024-05-11T08:03:39.402 INFO:teuthology.orchestra.run.smithi167.stdout:(62/156): libtool-2.4.6-45.el9.x86_64.rpm 641 kB/s | 582 kB 00:00 2024-05-11T08:03:39.440 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-11T08:03:39.594 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-11T08:03:39.637 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-11T08:03:39.765 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-11T08:03:39.769 INFO:teuthology.orchestra.run.smithi167.stdout:(63/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 72 kB/s | 47 kB 00:00 2024-05-11T08:03:39.839 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-11T08:03:40.053 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-11T08:03:40.189 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : re2-1:20211101-3.el9.x86_64 19/164 2024-05-11T08:03:40.211 INFO:teuthology.orchestra.run.smithi167.stdout:(64/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 310 kB/s | 292 kB 00:00 2024-05-11T08:03:40.218 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-05-11T08:03:40.245 INFO:teuthology.orchestra.run.smithi167.stdout:(65/156): lua-5.4.4-4.el9.x86_64.rpm 224 kB/s | 188 kB 00:00 2024-05-11T08:03:40.253 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : liboath-2.6.7-2.el9.x86_64 21/164 2024-05-11T08:03:40.335 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 22/164 2024-05-11T08:03:40.378 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 23/164 2024-05-11T08:03:40.412 INFO:teuthology.orchestra.run.smithi167.stdout:(66/156): openblas-0.3.21-2.el9.x86_64.rpm 55 kB/s | 35 kB 00:00 2024-05-11T08:03:40.422 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 24/164 2024-05-11T08:03:40.454 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 25/164 2024-05-11T08:03:40.482 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : lua-5.4.4-4.el9.x86_64 26/164 2024-05-11T08:03:40.568 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 27/164 2024-05-11T08:03:40.621 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 28/164 2024-05-11T08:03:40.652 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 29/164 2024-05-11T08:03:40.696 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 30/164 2024-05-11T08:03:40.772 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 31/164 2024-05-11T08:03:40.815 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-ceph-common-2:19.0.0-3601.g30850680.el9. 32/164 2024-05-11T08:03:40.859 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-ceph-argparse-2:19.0.0-3601.g30850680.el 33/164 2024-05-11T08:03:40.886 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-cephfs-2:19.0.0-3601.g30850680.el9.x86_6 34/164 2024-05-11T08:03:40.955 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 35/164 2024-05-11T08:03:41.001 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : luarocks-3.9.2-1.el9.noarch 36/164 2024-05-11T08:03:41.076 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 37/164 2024-05-11T08:03:41.210 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 38/164 2024-05-11T08:03:41.270 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 39/164 2024-05-11T08:03:41.412 INFO:teuthology.orchestra.run.smithi167.stdout:(67/156): openblas-openmp-0.3.21-2.el9.x86_64.r 4.0 MB/s | 4.7 MB 00:01 2024-05-11T08:03:41.454 INFO:teuthology.orchestra.run.smithi167.stdout:(68/156): postgresql-13.14-1.el9.x86_64.rpm 1.5 MB/s | 1.6 MB 00:01 2024-05-11T08:03:41.521 INFO:teuthology.orchestra.run.smithi167.stdout:(69/156): openssl-devel-3.2.1-1.el9.x86_64.rpm 3.5 MB/s | 4.4 MB 00:01 2024-05-11T08:03:41.796 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-rsa-4.9-2.el9.noarch 40/164 2024-05-11T08:03:41.875 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 41/164 2024-05-11T08:03:41.904 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 42/164 2024-05-11T08:03:41.946 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 43/164 2024-05-11T08:03:41.985 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : librados-devel-2:19.0.0-3601.g30850680.el9.x86_6 44/164 2024-05-11T08:03:42.027 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 45/164 2024-05-11T08:03:42.175 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 46/164 2024-05-11T08:03:42.213 INFO:teuthology.orchestra.run.smithi167.stdout:(70/156): postgresql-private-libs-13.14-1.el9.x 176 kB/s | 133 kB 00:00 2024-05-11T08:03:42.229 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 47/164 2024-05-11T08:03:42.320 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 48/164 2024-05-11T08:03:42.385 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 49/164 2024-05-11T08:03:42.413 INFO:teuthology.orchestra.run.smithi167.stdout:(71/156): postgresql-contrib-13.14-1.el9.x86_64 911 kB/s | 911 kB 00:01 2024-05-11T08:03:42.466 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 50/164 2024-05-11T08:03:42.498 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 51/164 2024-05-11T08:03:42.534 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 52/164 2024-05-11T08:03:42.581 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 53/164 2024-05-11T08:03:42.642 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 54/164 2024-05-11T08:03:42.685 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 55/164 2024-05-11T08:03:42.727 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 56/164 2024-05-11T08:03:42.738 INFO:teuthology.orchestra.run.smithi167.stdout:(72/156): postgresql-server-13.14-1.el9.x86_64. 4.7 MB/s | 5.8 MB 00:01 2024-05-11T08:03:42.863 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 57/164 2024-05-11T08:03:42.908 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 58/164 2024-05-11T08:03:42.938 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 59/164 2024-05-11T08:03:42.980 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 60/164 2024-05-11T08:03:43.074 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-05-11T08:03:43.097 INFO:teuthology.orchestra.run.smithi167.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 128 kB/s | 87 kB 00:00 2024-05-11T08:03:43.105 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 62/164 2024-05-11T08:03:43.155 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-05-11T08:03:43.222 INFO:teuthology.orchestra.run.smithi167.stdout:(74/156): protobuf-3.14.0-13.el9.x86_64.rpm 1.0 MB/s | 1.0 MB 00:01 2024-05-11T08:03:43.304 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-05-11T08:03:43.446 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-05-11T08:03:43.554 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-05-11T08:03:43.692 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-05-11T08:03:43.736 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-05-11T08:03:43.764 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-05-11T08:03:43.799 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-05-11T08:03:43.838 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-05-11T08:03:43.869 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-05-11T08:03:43.899 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-05-11T08:03:43.934 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-05-11T08:03:43.939 INFO:teuthology.orchestra.run.smithi167.stdout:(75/156): python3-babel-2.9.1-2.el9.noarch.rpm 5.0 MB/s | 6.0 MB 00:01 2024-05-11T08:03:43.964 INFO:teuthology.orchestra.run.smithi167.stdout:(76/156): python3-devel-3.9.18-3.el9.x86_64.rpm 282 kB/s | 244 kB 00:00 2024-05-11T08:03:43.967 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-05-11T08:03:44.002 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-05-11T08:03:44.035 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-05-11T08:03:44.064 INFO:teuthology.orchestra.run.smithi167.stdout:(77/156): python3-jinja2-2.11.3-5.el9.noarch.rp 294 kB/s | 248 kB 00:00 2024-05-11T08:03:44.122 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-05-11T08:03:44.171 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-05-11T08:03:44.205 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-05-11T08:03:44.252 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-05-11T08:03:44.350 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-packaging-20.9-5.el9.noarch 82/164 2024-05-11T08:03:44.407 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/164 2024-05-11T08:03:44.514 INFO:teuthology.orchestra.run.smithi167.stdout:(78/156): python3-jsonpatch-1.21-16.el9.noarch. 45 kB/s | 26 kB 00:00 2024-05-11T08:03:44.540 INFO:teuthology.orchestra.run.smithi167.stdout:(79/156): python3-jsonpointer-2.0-4.el9.noarch. 33 kB/s | 19 kB 00:00 2024-05-11T08:03:44.840 INFO:teuthology.orchestra.run.smithi167.stdout:(80/156): python3-mako-1.1.4-6.el9.noarch.rpm 222 kB/s | 172 kB 00:00 2024-05-11T08:03:45.124 INFO:teuthology.orchestra.run.smithi167.stdout:(81/156): python3-markupsafe-1.1.1-12.el9.x86_6 57 kB/s | 35 kB 00:00 2024-05-11T08:03:45.126 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/164 2024-05-11T08:03:45.170 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/164 2024-05-11T08:03:45.204 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/164 2024-05-11T08:03:45.302 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/164 2024-05-11T08:03:45.362 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 88/164 2024-05-11T08:03:45.448 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-05-11T08:03:45.482 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-05-11T08:03:45.650 INFO:teuthology.orchestra.run.smithi167.stdout:(82/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 310 kB/s | 250 kB 00:00 2024-05-11T08:03:45.659 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : openblas-0.3.21-2.el9.x86_64 91/164 2024-05-11T08:03:45.687 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 92/164 2024-05-11T08:03:45.725 INFO:teuthology.orchestra.run.smithi167.stdout:(83/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 4.3 MB/s | 5.1 MB 00:01 2024-05-11T08:03:45.766 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-05-11T08:03:45.967 INFO:teuthology.orchestra.run.smithi167.stdout:(84/156): python3-oauthlib-3.1.1-5.el9.noarch.r 264 kB/s | 222 kB 00:00 2024-05-11T08:03:46.350 INFO:teuthology.orchestra.run.smithi167.stdout:(85/156): python3-packaging-20.9-5.el9.noarch.r 110 kB/s | 77 kB 00:00 2024-05-11T08:03:46.376 INFO:teuthology.orchestra.run.smithi167.stdout:(86/156): python3-prettytable-0.7.2-27.el9.noar 65 kB/s | 42 kB 00:00 2024-05-11T08:03:46.428 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-05-11T08:03:46.536 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-05-11T08:03:46.793 INFO:teuthology.orchestra.run.smithi167.stdout:(87/156): python3-protobuf-3.14.0-13.el9.noarch 326 kB/s | 269 kB 00:00 2024-05-11T08:03:47.101 INFO:teuthology.orchestra.run.smithi167.stdout:(88/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 211 kB/s | 159 kB 00:00 2024-05-11T08:03:47.210 INFO:teuthology.orchestra.run.smithi167.stdout:(89/156): python3-pyasn1-modules-0.4.8-6.el9.no 334 kB/s | 279 kB 00:00 2024-05-11T08:03:47.477 INFO:teuthology.orchestra.run.smithi167.stdout:(90/156): python3-pytz-2021.1-5.el9.noarch.rpm 75 kB/s | 51 kB 00:00 2024-05-11T08:03:47.744 INFO:teuthology.orchestra.run.smithi167.stdout:(91/156): python3-requests-oauthlib-1.3.0-12.el 83 kB/s | 54 kB 00:00 2024-05-11T08:03:47.786 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-05-11T08:03:47.859 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-05-11T08:03:47.879 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-05-11T08:03:47.904 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 99/164 2024-05-11T08:03:48.089 INFO:teuthology.orchestra.run.smithi167.stdout:(92/156): python3-toml-0.10.2-6.el9.noarch.rpm 68 kB/s | 42 kB 00:00 2024-05-11T08:03:48.211 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-05-11T08:03:48.259 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : librbd1-2:19.0.0-3601.g30850680.el9.x86_64 101/164 2024-05-11T08:03:48.333 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: librbd1-2:19.0.0-3601.g30850680.el9.x86_64 101/164 2024-05-11T08:03:48.363 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-rbd-2:19.0.0-3601.g30850680.el9.x86_64 102/164 2024-05-11T08:03:48.386 INFO:teuthology.orchestra.run.smithi167.stdout:(93/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 69 kB/s | 44 kB 00:00 2024-05-11T08:03:48.400 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-05-11T08:03:48.736 INFO:teuthology.orchestra.run.smithi167.stdout:(94/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 11 MB/s | 16 MB 00:01 2024-05-11T08:03:48.882 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-05-11T08:03:48.912 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : librgw2-2:19.0.0-3601.g30850680.el9.x86_64 105/164 2024-05-11T08:03:48.983 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: librgw2-2:19.0.0-3601.g30850680.el9.x86_64 105/164 2024-05-11T08:03:49.014 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-rgw-2:19.0.0-3601.g30850680.el9.x86_64 106/164 2024-05-11T08:03:49.028 INFO:teuthology.orchestra.run.smithi167.stdout:(95/156): qatlib-24.02.0-1.el9.x86_64.rpm 235 kB/s | 221 kB 00:00 2024-05-11T08:03:49.145 INFO:teuthology.orchestra.run.smithi167.stdout:(96/156): qatlib-service-24.02.0-1.el9.x86_64.r 47 kB/s | 36 kB 00:00 2024-05-11T08:03:49.379 INFO:teuthology.orchestra.run.smithi167.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 77 kB/s | 47 kB 00:00 2024-05-11T08:03:49.804 INFO:teuthology.orchestra.run.smithi167.stdout:(98/156): uuid-1.6.2-55.el9.x86_64.rpm 88 kB/s | 58 kB 00:00 2024-05-11T08:03:49.896 INFO:teuthology.orchestra.run.smithi167.stdout:(99/156): socat-1.7.4.1-5.el9.x86_64.rpm 351 kB/s | 305 kB 00:00 2024-05-11T08:03:50.113 INFO:teuthology.orchestra.run.smithi167.stdout:(100/156): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 132 kB/s | 94 kB 00:00 2024-05-11T08:03:50.205 INFO:teuthology.orchestra.run.smithi167.stdout:(101/156): lua-devel-5.4.4-4.el9.x86_64.rpm 243 kB/s | 22 kB 00:00 2024-05-11T08:03:50.347 INFO:teuthology.orchestra.run.smithi167.stdout:(102/156): protobuf-compiler-3.14.0-13.el9.x86_ 6.0 MB/s | 863 kB 00:00 2024-05-11T08:03:50.497 INFO:teuthology.orchestra.run.smithi167.stdout:(103/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 92 kB/s | 64 kB 00:00 2024-05-11T08:03:50.522 INFO:teuthology.orchestra.run.smithi167.stdout:(104/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 72 kB/s | 45 kB 00:00 2024-05-11T08:03:50.681 INFO:teuthology.orchestra.run.smithi167.stdout:(105/156): grpc-data-1.46.7-10.el9.noarch.rpm 123 kB/s | 19 kB 00:00 2024-05-11T08:03:50.706 INFO:teuthology.orchestra.run.smithi167.stdout:(106/156): abseil-cpp-20211102.0-3.el9.x86_64.r 1.5 MB/s | 548 kB 00:00 2024-05-11T08:03:50.765 INFO:teuthology.orchestra.run.smithi167.stdout:(107/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 444 kB/s | 26 kB 00:00 2024-05-11T08:03:50.799 INFO:teuthology.orchestra.run.smithi167.stdout:(108/156): gperftools-libs-2.9.1-2.el9.x86_64.r 1.0 MB/s | 309 kB 00:00 2024-05-11T08:03:50.824 INFO:teuthology.orchestra.run.smithi167.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 831 kB/s | 49 kB 00:00 2024-05-11T08:03:50.866 INFO:teuthology.orchestra.run.smithi167.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 1.0 MB/s | 67 kB 00:00 2024-05-11T08:03:50.900 INFO:teuthology.orchestra.run.smithi167.stdout:(111/156): luarocks-3.9.2-1.el9.noarch.rpm 2.0 MB/s | 151 kB 00:00 2024-05-11T08:03:50.983 INFO:teuthology.orchestra.run.smithi167.stdout:(112/156): python3-asyncssh-2.13.2-5.el9.noarch 6.4 MB/s | 548 kB 00:00 2024-05-11T08:03:51.025 INFO:teuthology.orchestra.run.smithi167.stdout:(113/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 5.2 MB/s | 839 kB 00:00 2024-05-11T08:03:51.059 INFO:teuthology.orchestra.run.smithi167.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 576 kB/s | 43 kB 00:00 2024-05-11T08:03:51.092 INFO:teuthology.orchestra.run.smithi167.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 480 kB/s | 32 kB 00:00 2024-05-11T08:03:51.126 INFO:teuthology.orchestra.run.smithi167.stdout:(116/156): python3-certifi-2021.10.8-2.el9.noar 224 kB/s | 15 kB 00:00 2024-05-11T08:03:51.168 INFO:teuthology.orchestra.run.smithi167.stdout:(117/156): python3-cheroot-8.6.0-4.el9.noarch.r 2.2 MB/s | 172 kB 00:00 2024-05-11T08:03:51.243 INFO:teuthology.orchestra.run.smithi167.stdout:(118/156): libarrow-9.0.0-11.el9.x86_64.rpm 7.9 MB/s | 4.4 MB 00:00 2024-05-11T08:03:51.269 INFO:teuthology.orchestra.run.smithi167.stdout:(119/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.0 MB/s | 303 kB 00:00 2024-05-11T08:03:51.294 INFO:teuthology.orchestra.run.smithi167.stdout:(120/156): python3-cherrypy-18.6.1-2.el9.noarch 2.1 MB/s | 358 kB 00:00 2024-05-11T08:03:51.304 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-common-2:19.0.0-3601.g30850680.el9.x86_64 107/164 2024-05-11T08:03:51.319 INFO:teuthology.orchestra.run.smithi167.stdout:(121/156): python3-colorama-0.4.4-12.el9.noarch 466 kB/s | 35 kB 00:00 2024-05-11T08:03:51.335 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-common-2:19.0.0-3601.g30850680.el9.x86_64 107/164 2024-05-11T08:03:51.344 INFO:teuthology.orchestra.run.smithi167.stdout:(122/156): python3-google-auth-2.29.0-1.el9.noa 2.7 MB/s | 206 kB 00:00 2024-05-11T08:03:51.386 INFO:teuthology.orchestra.run.smithi167.stdout:(123/156): python3-grpcio-tools-1.46.7-10.el9.x 2.1 MB/s | 144 kB 00:00 2024-05-11T08:03:51.437 INFO:teuthology.orchestra.run.smithi167.stdout:(124/156): python3-grpcio-1.46.7-10.el9.x86_64. 14 MB/s | 2.0 MB 00:00 2024-05-11T08:03:51.454 INFO:teuthology.orchestra.run.smithi167.stdout:(125/156): python3-jaraco-8.2.1-3.el9.noarch.rp 98 kB/s | 11 kB 00:00 2024-05-11T08:03:51.470 INFO:teuthology.orchestra.run.smithi167.stdout:(126/156): python3-jaraco-classes-3.2.1-5.el9.n 210 kB/s | 18 kB 00:00 2024-05-11T08:03:51.496 INFO:teuthology.orchestra.run.smithi167.stdout:(127/156): python3-jaraco-collections-3.0.0-8.e 393 kB/s | 23 kB 00:00 2024-05-11T08:03:51.521 INFO:teuthology.orchestra.run.smithi167.stdout:(128/156): python3-jaraco-functools-3.5.0-2.el9 289 kB/s | 19 kB 00:00 2024-05-11T08:03:51.538 INFO:teuthology.orchestra.run.smithi167.stdout:(129/156): python3-jaraco-text-3.2.0-6.el9.noar 291 kB/s | 20 kB 00:00 2024-05-11T08:03:51.580 INFO:teuthology.orchestra.run.smithi167.stdout:(130/156): python3-kubernetes-26.1.0-2.el9.noar 12 MB/s | 1.0 MB 00:00 2024-05-11T08:03:51.597 INFO:teuthology.orchestra.run.smithi167.stdout:(131/156): python3-logutils-0.3.5-21.el9.noarch 613 kB/s | 46 kB 00:00 2024-05-11T08:03:51.622 INFO:teuthology.orchestra.run.smithi167.stdout:(132/156): python3-more-itertools-8.12.0-2.el9. 940 kB/s | 79 kB 00:00 2024-05-11T08:03:51.639 INFO:teuthology.orchestra.run.smithi167.stdout:(133/156): python3-natsort-7.1.1-5.el9.noarch.r 983 kB/s | 58 kB 00:00 2024-05-11T08:03:51.673 INFO:teuthology.orchestra.run.smithi167.stdout:(134/156): python3-pecan-1.4.2-3.el9.noarch.rpm 3.5 MB/s | 272 kB 00:00 2024-05-11T08:03:51.690 INFO:teuthology.orchestra.run.smithi167.stdout:(135/156): python3-portend-3.1.0-2.el9.noarch.r 243 kB/s | 16 kB 00:00 2024-05-11T08:03:51.715 INFO:teuthology.orchestra.run.smithi167.stdout:(136/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-05-11T08:03:51.732 INFO:teuthology.orchestra.run.smithi167.stdout:(137/156): python3-pyperclip-1.8.2-2.el9.noarch 479 kB/s | 28 kB 00:00 2024-05-11T08:03:51.826 INFO:teuthology.orchestra.run.smithi167.stdout:(138/156): python3-repoze-lru-0.7-16.el9.noarch 520 kB/s | 31 kB 00:00 2024-05-11T08:03:51.827 INFO:teuthology.orchestra.run.smithi167.stdout:(139/156): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-05-11T08:03:51.827 INFO:teuthology.orchestra.run.smithi167.stdout:(140/156): python3-rsa-4.9-2.el9.noarch.rpm 782 kB/s | 59 kB 00:00 2024-05-11T08:03:51.827 INFO:teuthology.orchestra.run.smithi167.stdout:(141/156): python3-tempora-5.0.0-2.el9.noarch.r 474 kB/s | 36 kB 00:00 2024-05-11T08:03:51.852 INFO:teuthology.orchestra.run.smithi167.stdout:(142/156): python3-typing-extensions-4.4.0-2.el 864 kB/s | 51 kB 00:00 2024-05-11T08:03:51.885 INFO:teuthology.orchestra.run.smithi167.stdout:(143/156): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-05-11T08:03:51.891 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-common-2:19.0.0-3601.g30850680.el9.x86_64 107/164 2024-05-11T08:03:51.910 INFO:teuthology.orchestra.run.smithi167.stdout:(144/156): python3-websocket-client-1.2.3-2.el9 1.0 MB/s | 90 kB 00:00 2024-05-11T08:03:51.931 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-base-2:19.0.0-3601.g30850680.el9.x86_64 108/164 2024-05-11T08:03:51.935 INFO:teuthology.orchestra.run.smithi167.stdout:(145/156): python3-werkzeug-2.0.3-3.el9.1.noarc 5.0 MB/s | 427 kB 00:00 2024-05-11T08:03:51.995 INFO:teuthology.orchestra.run.smithi167.stdout:(146/156): python3-zc-lockfile-2.0-10.el9.noarc 181 kB/s | 20 kB 00:00 2024-05-11T08:03:52.019 INFO:teuthology.orchestra.run.smithi167.stdout:(147/156): re2-20211101-3.el9.x86_64.rpm 1.7 MB/s | 192 kB 00:00 2024-05-11T08:03:52.022 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-base-2:19.0.0-3601.g30850680.el9.x86_64 108/164 2024-05-11T08:03:52.022 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-11T08:03:52.022 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-11T08:03:52.022 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T08:03:52.061 INFO:teuthology.orchestra.run.smithi167.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 13 MB/s | 1.6 MB 00:00 2024-05-11T08:03:52.065 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-selinux-2:19.0.0-3601.g30850680.el9.x86_64 109/164 2024-05-11T08:03:52.361 INFO:teuthology.orchestra.run.smithi167.stdout:(149/156): librbd1-19.0.0-3601.g30850680.el9.x8 9.2 MB/s | 3.1 MB 00:00 2024-05-11T08:03:52.420 INFO:teuthology.orchestra.run.smithi167.stdout:(150/156): librados2-19.0.0-3601.g30850680.el9. 8.0 MB/s | 3.4 MB 00:00 2024-05-11T08:03:52.762 INFO:teuthology.orchestra.run.smithi167.stdout:(151/156): NetworkManager-team-1.47.5-1.el9.x86 122 kB/s | 42 kB 00:00 2024-05-11T08:03:52.987 INFO:teuthology.orchestra.run.smithi167.stdout:(152/156): NetworkManager-1.47.5-1.el9.x86_64.r 2.5 MB/s | 2.3 MB 00:00 2024-05-11T08:03:53.137 INFO:teuthology.orchestra.run.smithi167.stdout:(153/156): openssl-3.2.1-1.el9.x86_64.rpm 9.0 MB/s | 1.3 MB 00:00 2024-05-11T08:03:53.162 INFO:teuthology.orchestra.run.smithi167.stdout:(154/156): NetworkManager-tui-1.47.5-1.el9.x86_ 620 kB/s | 248 kB 00:00 2024-05-11T08:03:53.338 INFO:teuthology.orchestra.run.smithi167.stdout:(155/156): NetworkManager-libnm-1.47.5-1.el9.x8 1.9 MB/s | 1.8 MB 00:00 2024-05-11T08:03:53.929 INFO:teuthology.orchestra.run.smithi167.stdout:(156/156): openssl-libs-3.2.1-1.el9.x86_64.rpm 3.0 MB/s | 2.4 MB 00:00 2024-05-11T08:03:53.932 INFO:teuthology.orchestra.run.smithi167.stdout:-------------------------------------------------------------------------------- 2024-05-11T08:03:53.933 INFO:teuthology.orchestra.run.smithi167.stdout:Total 8.7 MB/s | 226 MB 00:25 2024-05-11T08:03:55.074 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction check 2024-05-11T08:03:55.226 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction check succeeded. 2024-05-11T08:03:55.226 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction test 2024-05-11T08:03:57.383 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction test succeeded. 2024-05-11T08:03:57.383 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction 2024-05-11T08:04:03.464 INFO:teuthology.orchestra.run.smithi167.stdout: Preparing : 1/1 2024-05-11T08:04:03.551 INFO:teuthology.orchestra.run.smithi167.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/164 2024-05-11T08:04:03.589 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-05-11T08:04:03.656 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-05-11T08:04:03.693 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-05-11T08:04:03.719 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-05-11T08:04:03.774 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-05-11T08:04:03.774 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3601.g30850680.el9.x86_64 109/164 2024-05-11T08:04:03.774 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /sys 2024-05-11T08:04:03.775 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /proc 2024-05-11T08:04:03.775 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /mnt 2024-05-11T08:04:03.775 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /var/tmp 2024-05-11T08:04:03.775 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /home 2024-05-11T08:04:03.775 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /root 2024-05-11T08:04:03.775 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /tmp 2024-05-11T08:04:03.775 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T08:04:03.950 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-volume-2:19.0.0-3601.g30850680.el9.noarch 110/164 2024-05-11T08:04:03.967 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-volume-2:19.0.0-3601.g30850680.el9.noarch 110/164 2024-05-11T08:04:03.967 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-11T08:04:03.967 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-11T08:04:03.967 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T08:04:04.104 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-05-11T08:04:04.220 INFO:teuthology.orchestra.run.smithi167.stdout: Upgrading : librados2-2:19.0.0-3601.g30850680.el9.x86_64 8/164 2024-05-11T08:04:04.342 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: librados2-2:19.0.0-3601.g30850680.el9.x86_64 8/164 2024-05-11T08:04:04.371 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libcephfs2-2:19.0.0-3601.g30850680.el9.x86_64 9/164 2024-05-11T08:04:04.445 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: libcephfs2-2:19.0.0-3601.g30850680.el9.x86_64 9/164 2024-05-11T08:04:04.475 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-rados-2:19.0.0-3601.g30850680.el9.x86_64 10/164 2024-05-11T08:04:04.647 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/164 2024-05-11T08:04:04.706 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/164 2024-05-11T08:04:04.733 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libcephsqlite-2:19.0.0-3601.g30850680.el9.x86_64 13/164 2024-05-11T08:04:04.820 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3601.g30850680.el9.x86_64 13/164 2024-05-11T08:04:04.843 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libradosstriper1-2:19.0.0-3601.g30850680.el9.x86 14/164 2024-05-11T08:04:04.889 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3601.g30850680.el9.x86 14/164 2024-05-11T08:04:04.911 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-11T08:04:04.934 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-11T08:04:05.129 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-11T08:04:05.312 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-11T08:04:05.450 INFO:teuthology.orchestra.run.smithi167.stdout: Upgrading : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-11T08:04:05.461 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-osd-2:19.0.0-3601.g30850680.el9.x86_64 111/164 2024-05-11T08:04:05.489 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-osd-2:19.0.0-3601.g30850680.el9.x86_64 111/164 2024-05-11T08:04:05.489 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-11T08:04:05.489 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-11T08:04:05.489 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-11T08:04:05.489 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-11T08:04:05.489 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T08:04:05.511 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-11T08:04:05.639 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-11T08:04:05.703 INFO:teuthology.orchestra.run.smithi167.stdout: Upgrading : NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-11T08:04:05.712 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mds-2:19.0.0-3601.g30850680.el9.x86_64 112/164 2024-05-11T08:04:05.736 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-mds-2:19.0.0-3601.g30850680.el9.x86_64 112/164 2024-05-11T08:04:05.736 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-11T08:04:05.736 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-11T08:04:05.736 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-11T08:04:05.736 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-11T08:04:05.736 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T08:04:05.920 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-11T08:04:06.025 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : re2-1:20211101-3.el9.x86_64 19/164 2024-05-11T08:04:06.058 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-05-11T08:04:06.092 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : liboath-2.6.7-2.el9.x86_64 21/164 2024-05-11T08:04:06.174 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 22/164 2024-05-11T08:04:06.188 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mon-2:19.0.0-3601.g30850680.el9.x86_64 113/164 2024-05-11T08:04:06.211 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-mon-2:19.0.0-3601.g30850680.el9.x86_64 113/164 2024-05-11T08:04:06.212 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-11T08:04:06.212 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-11T08:04:06.212 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-11T08:04:06.212 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-11T08:04:06.212 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T08:04:06.217 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 23/164 2024-05-11T08:04:06.260 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 24/164 2024-05-11T08:04:06.293 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 25/164 2024-05-11T08:04:06.321 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : lua-5.4.4-4.el9.x86_64 26/164 2024-05-11T08:04:06.326 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-05-11T08:04:06.389 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 27/164 2024-05-11T08:04:06.400 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-05-11T08:04:06.443 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 28/164 2024-05-11T08:04:06.456 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-05-11T08:04:06.474 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 29/164 2024-05-11T08:04:06.509 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 30/164 2024-05-11T08:04:06.576 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 31/164 2024-05-11T08:04:06.611 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-ceph-common-2:19.0.0-3601.g30850680.el9. 32/164 2024-05-11T08:04:06.642 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-05-11T08:04:06.654 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-ceph-argparse-2:19.0.0-3601.g30850680.el 33/164 2024-05-11T08:04:06.683 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-cephfs-2:19.0.0-3601.g30850680.el9.x86_6 34/164 2024-05-11T08:04:06.697 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-05-11T08:04:06.753 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 35/164 2024-05-11T08:04:06.775 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-05-11T08:04:06.798 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : luarocks-3.9.2-1.el9.noarch 36/164 2024-05-11T08:04:06.864 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-05-11T08:04:06.873 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 37/164 2024-05-11T08:04:07.008 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 38/164 2024-05-11T08:04:07.011 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-05-11T08:04:07.050 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-05-11T08:04:07.067 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 39/164 2024-05-11T08:04:07.080 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-05-11T08:04:07.115 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : fuse-2.9.9-15.el9.x86_64 124/164 2024-05-11T08:04:07.562 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: cephadm-2:19.0.0-3601.g30850680.el9.noarch 125/164 2024-05-11T08:04:07.590 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : cephadm-2:19.0.0-3601.g30850680.el9.noarch 125/164 2024-05-11T08:04:07.597 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-rsa-4.9-2.el9.noarch 40/164 2024-05-11T08:04:07.621 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3601.g30850680.e 126/164 2024-05-11T08:04:07.680 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 41/164 2024-05-11T08:04:07.682 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3601.g30850680. 127/164 2024-05-11T08:04:07.712 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3601.g30850680.el9.noa 128/164 2024-05-11T08:04:07.718 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 42/164 2024-05-11T08:04:07.760 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 43/164 2024-05-11T08:04:07.799 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : librados-devel-2:19.0.0-3601.g30850680.el9.x86_6 44/164 2024-05-11T08:04:07.842 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 45/164 2024-05-11T08:04:08.002 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 46/164 2024-05-11T08:04:08.043 INFO:teuthology.orchestra.run.smithi167.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 47/164 2024-05-11T08:04:08.135 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 48/164 2024-05-11T08:04:08.199 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 49/164 2024-05-11T08:04:08.273 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 50/164 2024-05-11T08:04:08.303 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 51/164 2024-05-11T08:04:08.339 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 52/164 2024-05-11T08:04:08.387 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 53/164 2024-05-11T08:04:08.430 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 54/164 2024-05-11T08:04:08.474 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 55/164 2024-05-11T08:04:08.516 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 56/164 2024-05-11T08:04:08.653 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 57/164 2024-05-11T08:04:08.705 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 58/164 2024-05-11T08:04:08.735 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 59/164 2024-05-11T08:04:08.756 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3601.g30850680.el9.noa 128/164 2024-05-11T08:04:08.777 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 60/164 2024-05-11T08:04:08.798 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3601.g30850680.el9.n 129/164 2024-05-11T08:04:08.880 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-05-11T08:04:08.911 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 62/164 2024-05-11T08:04:08.953 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-05-11T08:04:08.993 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-05-11T08:04:09.018 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-05-11T08:04:09.100 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-05-11T08:04:09.228 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-05-11T08:04:09.267 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-05-11T08:04:09.294 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-05-11T08:04:09.329 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-05-11T08:04:09.377 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-05-11T08:04:09.408 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-05-11T08:04:09.438 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-05-11T08:04:09.472 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-05-11T08:04:09.505 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-05-11T08:04:09.547 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-05-11T08:04:09.582 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-05-11T08:04:09.660 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-05-11T08:04:09.704 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-05-11T08:04:09.735 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-05-11T08:04:09.762 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3601.g30850680.el9.n 129/164 2024-05-11T08:04:09.782 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-05-11T08:04:09.799 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3601.g308 130/164 2024-05-11T08:04:09.880 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-packaging-20.9-5.el9.noarch 82/164 2024-05-11T08:04:09.909 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3601.g308 130/164 2024-05-11T08:04:09.928 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/164 2024-05-11T08:04:10.068 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3601.g30850680.el 131/164 2024-05-11T08:04:10.100 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mgr-2:19.0.0-3601.g30850680.el9.x86_64 132/164 2024-05-11T08:04:10.124 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3601.g30850680.el9.x86_64 132/164 2024-05-11T08:04:10.124 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-11T08:04:10.124 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-11T08:04:10.124 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-11T08:04:10.124 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-11T08:04:10.125 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T08:04:10.176 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mgr-rook-2:19.0.0-3601.g30850680.el9.noarch 133/164 2024-05-11T08:04:10.188 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3601.g30850680.el9.noarch 133/164 2024-05-11T08:04:10.295 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-2:19.0.0-3601.g30850680.el9.x86_64 134/164 2024-05-11T08:04:10.355 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-fuse-2:19.0.0-3601.g30850680.el9.x86_64 135/164 2024-05-11T08:04:10.671 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/164 2024-05-11T08:04:10.717 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/164 2024-05-11T08:04:10.751 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/164 2024-05-11T08:04:10.851 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/164 2024-05-11T08:04:10.908 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 88/164 2024-05-11T08:04:11.005 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-05-11T08:04:11.036 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-05-11T08:04:11.213 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : openblas-0.3.21-2.el9.x86_64 91/164 2024-05-11T08:04:11.243 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 92/164 2024-05-11T08:04:11.290 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libtool-2.4.6-45.el9.x86_64 136/164 2024-05-11T08:04:11.322 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-05-11T08:04:11.328 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-radosgw-2:19.0.0-3601.g30850680.el9.x86_64 137/164 2024-05-11T08:04:11.352 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3601.g30850680.el9.x86_64 137/164 2024-05-11T08:04:11.353 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-11T08:04:11.353 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-11T08:04:11.353 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-11T08:04:11.353 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-11T08:04:11.353 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T08:04:11.396 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3601.g30850 138/164 2024-05-11T08:04:11.418 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3601.g30850 138/164 2024-05-11T08:04:11.418 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-11T08:04:11.419 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-11T08:04:11.419 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T08:04:11.463 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : cephfs-mirror-2:19.0.0-3601.g30850680.el9.x86_64 139/164 2024-05-11T08:04:11.487 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3601.g30850680.el9.x86_64 139/164 2024-05-11T08:04:11.487 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-11T08:04:11.487 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-11T08:04:11.487 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-11T08:04:11.487 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-11T08:04:11.487 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T08:04:11.789 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : rbd-mirror-2:19.0.0-3601.g30850680.el9.x86_64 140/164 2024-05-11T08:04:11.812 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3601.g30850680.el9.x86_64 140/164 2024-05-11T08:04:11.813 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-11T08:04:11.813 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-11T08:04:11.813 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-11T08:04:11.813 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-11T08:04:11.813 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T08:04:11.978 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-05-11T08:04:12.082 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-05-11T08:04:13.348 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-05-11T08:04:13.432 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-05-11T08:04:13.460 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-05-11T08:04:13.485 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 99/164 2024-05-11T08:04:13.788 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-05-11T08:04:13.831 INFO:teuthology.orchestra.run.smithi167.stdout: Upgrading : librbd1-2:19.0.0-3601.g30850680.el9.x86_64 101/164 2024-05-11T08:04:13.905 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: librbd1-2:19.0.0-3601.g30850680.el9.x86_64 101/164 2024-05-11T08:04:13.936 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-rbd-2:19.0.0-3601.g30850680.el9.x86_64 102/164 2024-05-11T08:04:13.973 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-05-11T08:04:14.449 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-05-11T08:04:14.484 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : librgw2-2:19.0.0-3601.g30850680.el9.x86_64 105/164 2024-05-11T08:04:14.556 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: librgw2-2:19.0.0-3601.g30850680.el9.x86_64 105/164 2024-05-11T08:04:14.586 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-rgw-2:19.0.0-3601.g30850680.el9.x86_64 106/164 2024-05-11T08:04:16.426 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-test-2:19.0.0-3601.g30850680.el9.x86_64 141/164 2024-05-11T08:04:16.476 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : rbd-fuse-2:19.0.0-3601.g30850680.el9.x86_64 142/164 2024-05-11T08:04:16.517 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : rbd-nbd-2:19.0.0-3601.g30850680.el9.x86_64 143/164 2024-05-11T08:04:16.548 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 144/164 2024-05-11T08:04:16.576 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 145/164 2024-05-11T08:04:16.768 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : cephfs-shell-2:19.0.0-3601.g30850680.el9.x86_64 146/164 2024-05-11T08:04:16.769 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-common-2:19.0.0-3601.g30850680.el9.x86_64 107/164 2024-05-11T08:04:16.807 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-common-2:19.0.0-3601.g30850680.el9.x86_64 107/164 2024-05-11T08:04:16.814 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 147/164 2024-05-11T08:04:16.844 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libcephfs-devel-2:19.0.0-3601.g30850680.el9.x86_ 148/164 2024-05-11T08:04:16.874 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : NetworkManager-team-1:1.47.5-1.el9.x86_64 149/164 2024-05-11T08:04:16.912 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : NetworkManager-tui-1:1.47.5-1.el9.x86_64 150/164 2024-05-11T08:04:17.367 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-common-2:19.0.0-3601.g30850680.el9.x86_64 107/164 2024-05-11T08:04:17.402 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-base-2:19.0.0-3601.g30850680.el9.x86_64 108/164 2024-05-11T08:04:17.483 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-11T08:04:17.489 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-base-2:19.0.0-3601.g30850680.el9.x86_64 108/164 2024-05-11T08:04:17.490 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-11T08:04:17.490 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-11T08:04:17.490 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T08:04:17.527 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-11T08:04:17.529 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-selinux-2:19.0.0-3601.g30850680.el9.x86_64 109/164 2024-05-11T08:04:17.547 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-11T08:04:18.381 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : cephfs-top-2:19.0.0-3601.g30850680.el9.noarch 152/164 2024-05-11T08:04:18.465 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 153/164 2024-05-11T08:04:18.514 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-05-11T08:04:18.598 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-05-11T08:04:18.627 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-05-11T08:04:18.653 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 157/164 2024-05-11T08:04:18.653 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-11T08:04:18.746 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-11T08:04:18.770 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 159/164 2024-05-11T08:04:18.770 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-11T08:04:18.855 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-11T08:04:18.880 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 161/164 2024-05-11T08:04:18.885 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-11T08:04:18.885 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-11T08:04:19.065 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-11T08:04:19.065 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-11T08:04:19.158 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-11T08:04:19.231 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-11T08:04:24.863 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-11T08:04:24.864 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-2:19.0.0-3601.g30850680.el9.x86_64 1/164 2024-05-11T08:04:24.864 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-base-2:19.0.0-3601.g30850680.el9.x86_64 2/164 2024-05-11T08:04:24.864 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-common-2:19.0.0-3601.g30850680.el9.x86_64 3/164 2024-05-11T08:04:24.864 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-fuse-2:19.0.0-3601.g30850680.el9.x86_64 4/164 2024-05-11T08:04:24.864 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3601.g30850 5/164 2024-05-11T08:04:24.864 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mds-2:19.0.0-3601.g30850680.el9.x86_64 6/164 2024-05-11T08:04:24.864 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mgr-2:19.0.0-3601.g30850680.el9.x86_64 7/164 2024-05-11T08:04:24.865 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mon-2:19.0.0-3601.g30850680.el9.x86_64 8/164 2024-05-11T08:04:24.865 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-osd-2:19.0.0-3601.g30850680.el9.x86_64 9/164 2024-05-11T08:04:24.865 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-radosgw-2:19.0.0-3601.g30850680.el9.x86_64 10/164 2024-05-11T08:04:24.865 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-selinux-2:19.0.0-3601.g30850680.el9.x86_64 11/164 2024-05-11T08:04:24.865 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-test-2:19.0.0-3601.g30850680.el9.x86_64 12/164 2024-05-11T08:04:24.865 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : cephfs-mirror-2:19.0.0-3601.g30850680.el9.x86_64 13/164 2024-05-11T08:04:24.865 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : cephfs-shell-2:19.0.0-3601.g30850680.el9.x86_64 14/164 2024-05-11T08:04:24.865 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libcephfs-devel-2:19.0.0-3601.g30850680.el9.x86_ 15/164 2024-05-11T08:04:24.865 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libcephfs2-2:19.0.0-3601.g30850680.el9.x86_64 16/164 2024-05-11T08:04:24.865 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libcephsqlite-2:19.0.0-3601.g30850680.el9.x86_64 17/164 2024-05-11T08:04:24.865 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librados-devel-2:19.0.0-3601.g30850680.el9.x86_6 18/164 2024-05-11T08:04:24.865 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libradosstriper1-2:19.0.0-3601.g30850680.el9.x86 19/164 2024-05-11T08:04:24.865 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librgw2-2:19.0.0-3601.g30850680.el9.x86_64 20/164 2024-05-11T08:04:24.866 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3601.g30850680.el 21/164 2024-05-11T08:04:24.866 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-ceph-common-2:19.0.0-3601.g30850680.el9. 22/164 2024-05-11T08:04:24.866 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-cephfs-2:19.0.0-3601.g30850680.el9.x86_6 23/164 2024-05-11T08:04:24.866 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-rados-2:19.0.0-3601.g30850680.el9.x86_64 24/164 2024-05-11T08:04:24.866 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-rbd-2:19.0.0-3601.g30850680.el9.x86_64 25/164 2024-05-11T08:04:24.866 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-rgw-2:19.0.0-3601.g30850680.el9.x86_64 26/164 2024-05-11T08:04:24.866 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : rbd-fuse-2:19.0.0-3601.g30850680.el9.x86_64 27/164 2024-05-11T08:04:24.880 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : rbd-mirror-2:19.0.0-3601.g30850680.el9.x86_64 28/164 2024-05-11T08:04:24.880 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : rbd-nbd-2:19.0.0-3601.g30850680.el9.x86_64 29/164 2024-05-11T08:04:24.880 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3601.g30850680. 30/164 2024-05-11T08:04:24.880 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3601.g30850680.el9.noa 31/164 2024-05-11T08:04:24.880 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3601.g30850680.el9.n 32/164 2024-05-11T08:04:24.880 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3601.g308 33/164 2024-05-11T08:04:24.880 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3601.g30850680.el 34/164 2024-05-11T08:04:24.880 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3601.g30850680.el9.noarch 35/164 2024-05-11T08:04:24.880 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3601.g30850680.e 36/164 2024-05-11T08:04:24.881 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-volume-2:19.0.0-3601.g30850680.el9.noarch 37/164 2024-05-11T08:04:24.881 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : cephadm-2:19.0.0-3601.g30850680.el9.noarch 38/164 2024-05-11T08:04:24.881 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : cephfs-top-2:19.0.0-3601.g30850680.el9.noarch 39/164 2024-05-11T08:04:24.881 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/164 2024-05-11T08:04:24.881 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/164 2024-05-11T08:04:24.881 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/164 2024-05-11T08:04:24.881 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-05-11T08:04:24.881 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-05-11T08:04:24.881 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-05-11T08:04:24.881 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-05-11T08:04:24.881 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-05-11T08:04:24.881 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/164 2024-05-11T08:04:24.881 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-05-11T08:04:24.881 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-05-11T08:04:24.882 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-05-11T08:04:24.882 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/164 2024-05-11T08:04:24.882 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-05-11T08:04:24.882 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-05-11T08:04:24.882 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-05-11T08:04:24.882 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-05-11T08:04:24.882 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-05-11T08:04:24.882 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-05-11T08:04:24.882 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-05-11T08:04:24.882 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-05-11T08:04:24.882 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/164 2024-05-11T08:04:24.882 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/164 2024-05-11T08:04:24.882 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-05-11T08:04:24.883 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-05-11T08:04:24.883 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-05-11T08:04:24.883 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/164 2024-05-11T08:04:24.883 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/164 2024-05-11T08:04:24.883 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/164 2024-05-11T08:04:24.883 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/164 2024-05-11T08:04:24.883 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/164 2024-05-11T08:04:24.883 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/164 2024-05-11T08:04:24.883 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/164 2024-05-11T08:04:24.883 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-05-11T08:04:24.883 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-05-11T08:04:24.883 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-05-11T08:04:24.883 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/164 2024-05-11T08:04:24.883 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/164 2024-05-11T08:04:24.884 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/164 2024-05-11T08:04:24.884 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/164 2024-05-11T08:04:24.884 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/164 2024-05-11T08:04:24.884 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/164 2024-05-11T08:04:24.884 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/164 2024-05-11T08:04:24.884 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/164 2024-05-11T08:04:24.884 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/164 2024-05-11T08:04:24.884 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/164 2024-05-11T08:04:24.884 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/164 2024-05-11T08:04:24.884 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/164 2024-05-11T08:04:24.884 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/164 2024-05-11T08:04:24.885 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/164 2024-05-11T08:04:24.885 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/164 2024-05-11T08:04:24.885 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/164 2024-05-11T08:04:24.885 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/164 2024-05-11T08:04:24.885 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/164 2024-05-11T08:04:24.885 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/164 2024-05-11T08:04:24.885 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/164 2024-05-11T08:04:24.885 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/164 2024-05-11T08:04:24.885 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/164 2024-05-11T08:04:24.885 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/164 2024-05-11T08:04:24.885 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/164 2024-05-11T08:04:24.885 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/164 2024-05-11T08:04:24.885 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/164 2024-05-11T08:04:24.885 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/164 2024-05-11T08:04:24.885 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/164 2024-05-11T08:04:24.886 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/164 2024-05-11T08:04:24.886 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/164 2024-05-11T08:04:24.886 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/164 2024-05-11T08:04:24.886 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/164 2024-05-11T08:04:24.886 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/164 2024-05-11T08:04:24.886 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/164 2024-05-11T08:04:24.886 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/164 2024-05-11T08:04:24.886 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/164 2024-05-11T08:04:24.886 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/164 2024-05-11T08:04:24.886 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/164 2024-05-11T08:04:24.886 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/164 2024-05-11T08:04:24.886 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/164 2024-05-11T08:04:24.886 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/164 2024-05-11T08:04:24.887 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/164 2024-05-11T08:04:24.887 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/164 2024-05-11T08:04:24.887 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/164 2024-05-11T08:04:24.887 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/164 2024-05-11T08:04:24.887 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/164 2024-05-11T08:04:24.887 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/164 2024-05-11T08:04:24.887 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-05-11T08:04:24.887 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-05-11T08:04:24.887 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-05-11T08:04:24.887 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-05-11T08:04:24.887 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-05-11T08:04:24.887 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-05-11T08:04:24.887 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-05-11T08:04:24.887 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-05-11T08:04:24.888 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-05-11T08:04:24.888 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-05-11T08:04:24.888 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-05-11T08:04:24.888 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-05-11T08:04:24.888 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-05-11T08:04:24.888 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-05-11T08:04:24.888 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-05-11T08:04:24.888 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-05-11T08:04:24.888 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-05-11T08:04:24.888 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-05-11T08:04:24.888 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-05-11T08:04:24.888 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/164 2024-05-11T08:04:24.888 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/164 2024-05-11T08:04:24.888 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-05-11T08:04:24.889 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-05-11T08:04:24.889 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-05-11T08:04:24.889 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/164 2024-05-11T08:04:24.889 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-05-11T08:04:24.889 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librados2-2:19.0.0-3601.g30850680.el9.x86_64 149/164 2024-05-11T08:04:24.889 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-05-11T08:04:24.889 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librbd1-2:19.0.0-3601.g30850680.el9.x86_64 151/164 2024-05-11T08:04:24.889 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-05-11T08:04:24.889 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : NetworkManager-1:1.47.5-1.el9.x86_64 153/164 2024-05-11T08:04:24.889 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/164 2024-05-11T08:04:24.889 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 155/164 2024-05-11T08:04:24.889 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/164 2024-05-11T08:04:24.889 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : NetworkManager-team-1:1.47.5-1.el9.x86_64 157/164 2024-05-11T08:04:24.890 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/164 2024-05-11T08:04:24.890 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : NetworkManager-tui-1:1.47.5-1.el9.x86_64 159/164 2024-05-11T08:04:24.890 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/164 2024-05-11T08:04:24.890 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 161/164 2024-05-11T08:04:24.890 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 162/164 2024-05-11T08:04:24.890 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 163/164 2024-05-11T08:04:27.734 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-11T08:04:27.734 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T08:04:27.734 INFO:teuthology.orchestra.run.smithi204.stdout:Upgraded: 2024-05-11T08:04:27.734 INFO:teuthology.orchestra.run.smithi204.stdout: NetworkManager-1:1.47.5-1.el9.x86_64 2024-05-11T08:04:27.734 INFO:teuthology.orchestra.run.smithi204.stdout: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 2024-05-11T08:04:27.734 INFO:teuthology.orchestra.run.smithi204.stdout: NetworkManager-team-1:1.47.5-1.el9.x86_64 2024-05-11T08:04:27.734 INFO:teuthology.orchestra.run.smithi204.stdout: NetworkManager-tui-1:1.47.5-1.el9.x86_64 2024-05-11T08:04:27.735 INFO:teuthology.orchestra.run.smithi204.stdout: librados2-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T08:04:27.735 INFO:teuthology.orchestra.run.smithi204.stdout: librbd1-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T08:04:27.735 INFO:teuthology.orchestra.run.smithi204.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-11T08:04:27.735 INFO:teuthology.orchestra.run.smithi204.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-11T08:04:27.735 INFO:teuthology.orchestra.run.smithi204.stdout:Installed: 2024-05-11T08:04:27.735 INFO:teuthology.orchestra.run.smithi204.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-11T08:04:27.735 INFO:teuthology.orchestra.run.smithi204.stdout: automake-1.16.2-8.el9.noarch 2024-05-11T08:04:27.735 INFO:teuthology.orchestra.run.smithi204.stdout: bison-3.7.4-5.el9.x86_64 2024-05-11T08:04:27.735 INFO:teuthology.orchestra.run.smithi204.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-11T08:04:27.735 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T08:04:27.735 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-base-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T08:04:27.735 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-common-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T08:04:27.735 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-fuse-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T08:04:27.735 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-grafana-dashboards-2:19.0.0-3601.g30850680.el9.noarch 2024-05-11T08:04:27.736 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-immutable-object-cache-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T08:04:27.736 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mds-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T08:04:27.736 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T08:04:27.736 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-cephadm-2:19.0.0-3601.g30850680.el9.noarch 2024-05-11T08:04:27.736 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-dashboard-2:19.0.0-3601.g30850680.el9.noarch 2024-05-11T08:04:27.736 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3601.g30850680.el9.noarch 2024-05-11T08:04:27.736 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-modules-core-2:19.0.0-3601.g30850680.el9.noarch 2024-05-11T08:04:27.736 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-rook-2:19.0.0-3601.g30850680.el9.noarch 2024-05-11T08:04:27.736 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mon-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T08:04:27.736 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-osd-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T08:04:27.736 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-prometheus-alerts-2:19.0.0-3601.g30850680.el9.noarch 2024-05-11T08:04:27.736 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-radosgw-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T08:04:27.736 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-selinux-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T08:04:27.737 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-test-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T08:04:27.737 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-volume-2:19.0.0-3601.g30850680.el9.noarch 2024-05-11T08:04:27.737 INFO:teuthology.orchestra.run.smithi204.stdout: cephadm-2:19.0.0-3601.g30850680.el9.noarch 2024-05-11T08:04:27.737 INFO:teuthology.orchestra.run.smithi204.stdout: cephfs-mirror-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T08:04:27.737 INFO:teuthology.orchestra.run.smithi204.stdout: cephfs-shell-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T08:04:27.737 INFO:teuthology.orchestra.run.smithi204.stdout: cephfs-top-2:19.0.0-3601.g30850680.el9.noarch 2024-05-11T08:04:27.737 INFO:teuthology.orchestra.run.smithi204.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-05-11T08:04:27.737 INFO:teuthology.orchestra.run.smithi204.stdout: flex-2.6.4-9.el9.x86_64 2024-05-11T08:04:27.737 INFO:teuthology.orchestra.run.smithi204.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-11T08:04:27.737 INFO:teuthology.orchestra.run.smithi204.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-11T08:04:27.737 INFO:teuthology.orchestra.run.smithi204.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-11T08:04:27.737 INFO:teuthology.orchestra.run.smithi204.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-11T08:04:27.737 INFO:teuthology.orchestra.run.smithi204.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-11T08:04:27.737 INFO:teuthology.orchestra.run.smithi204.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-11T08:04:27.738 INFO:teuthology.orchestra.run.smithi204.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-11T08:04:27.738 INFO:teuthology.orchestra.run.smithi204.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-11T08:04:27.738 INFO:teuthology.orchestra.run.smithi204.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-11T08:04:27.738 INFO:teuthology.orchestra.run.smithi204.stdout: libcephfs-devel-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T08:04:27.738 INFO:teuthology.orchestra.run.smithi204.stdout: libcephfs2-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T08:04:27.738 INFO:teuthology.orchestra.run.smithi204.stdout: libcephsqlite-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T08:04:27.738 INFO:teuthology.orchestra.run.smithi204.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-11T08:04:27.738 INFO:teuthology.orchestra.run.smithi204.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-11T08:04:27.738 INFO:teuthology.orchestra.run.smithi204.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-11T08:04:27.738 INFO:teuthology.orchestra.run.smithi204.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-11T08:04:27.738 INFO:teuthology.orchestra.run.smithi204.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-11T08:04:27.738 INFO:teuthology.orchestra.run.smithi204.stdout: librados-devel-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T08:04:27.738 INFO:teuthology.orchestra.run.smithi204.stdout: libradosstriper1-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T08:04:27.739 INFO:teuthology.orchestra.run.smithi204.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-11T08:04:27.739 INFO:teuthology.orchestra.run.smithi204.stdout: librgw2-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T08:04:27.739 INFO:teuthology.orchestra.run.smithi204.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-11T08:04:27.739 INFO:teuthology.orchestra.run.smithi204.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-11T08:04:27.739 INFO:teuthology.orchestra.run.smithi204.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-11T08:04:27.739 INFO:teuthology.orchestra.run.smithi204.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-11T08:04:27.739 INFO:teuthology.orchestra.run.smithi204.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-11T08:04:27.739 INFO:teuthology.orchestra.run.smithi204.stdout: lua-5.4.4-4.el9.x86_64 2024-05-11T08:04:27.739 INFO:teuthology.orchestra.run.smithi204.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-11T08:04:27.739 INFO:teuthology.orchestra.run.smithi204.stdout: luarocks-3.9.2-1.el9.noarch 2024-05-11T08:04:27.739 INFO:teuthology.orchestra.run.smithi204.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-11T08:04:27.739 INFO:teuthology.orchestra.run.smithi204.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-11T08:04:27.739 INFO:teuthology.orchestra.run.smithi204.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-11T08:04:27.739 INFO:teuthology.orchestra.run.smithi204.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-11T08:04:27.740 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql-13.14-1.el9.x86_64 2024-05-11T08:04:27.740 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-11T08:04:27.740 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-11T08:04:27.740 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-11T08:04:27.740 INFO:teuthology.orchestra.run.smithi204.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-11T08:04:27.740 INFO:teuthology.orchestra.run.smithi204.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-11T08:04:27.740 INFO:teuthology.orchestra.run.smithi204.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-11T08:04:27.740 INFO:teuthology.orchestra.run.smithi204.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-11T08:04:27.740 INFO:teuthology.orchestra.run.smithi204.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-11T08:04:27.740 INFO:teuthology.orchestra.run.smithi204.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-11T08:04:27.740 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-11T08:04:27.740 INFO:teuthology.orchestra.run.smithi204.stdout: python3-ceph-argparse-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T08:04:27.740 INFO:teuthology.orchestra.run.smithi204.stdout: python3-ceph-common-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T08:04:27.740 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cephfs-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T08:04:27.741 INFO:teuthology.orchestra.run.smithi204.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-11T08:04:27.741 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-11T08:04:27.741 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-11T08:04:27.741 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-11T08:04:27.741 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-11T08:04:27.741 INFO:teuthology.orchestra.run.smithi204.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-05-11T08:04:27.741 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-11T08:04:27.741 INFO:teuthology.orchestra.run.smithi204.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-11T08:04:27.741 INFO:teuthology.orchestra.run.smithi204.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-11T08:04:27.741 INFO:teuthology.orchestra.run.smithi204.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-11T08:04:27.741 INFO:teuthology.orchestra.run.smithi204.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-11T08:04:27.741 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-11T08:04:27.741 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-11T08:04:27.741 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-11T08:04:27.741 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-11T08:04:27.742 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-11T08:04:27.742 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-11T08:04:27.742 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-11T08:04:27.742 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-11T08:04:27.742 INFO:teuthology.orchestra.run.smithi204.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-11T08:04:27.742 INFO:teuthology.orchestra.run.smithi204.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-11T08:04:27.742 INFO:teuthology.orchestra.run.smithi204.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-11T08:04:27.742 INFO:teuthology.orchestra.run.smithi204.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-11T08:04:27.742 INFO:teuthology.orchestra.run.smithi204.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-11T08:04:27.742 INFO:teuthology.orchestra.run.smithi204.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-11T08:04:27.742 INFO:teuthology.orchestra.run.smithi204.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-11T08:04:27.742 INFO:teuthology.orchestra.run.smithi204.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-11T08:04:27.742 INFO:teuthology.orchestra.run.smithi204.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-11T08:04:27.743 INFO:teuthology.orchestra.run.smithi204.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-11T08:04:27.743 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-11T08:04:27.743 INFO:teuthology.orchestra.run.smithi204.stdout: python3-ply-3.11-14.el9.noarch 2024-05-11T08:04:27.743 INFO:teuthology.orchestra.run.smithi204.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-11T08:04:27.743 INFO:teuthology.orchestra.run.smithi204.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-11T08:04:27.743 INFO:teuthology.orchestra.run.smithi204.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-11T08:04:27.743 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-11T08:04:27.743 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-11T08:04:27.743 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-11T08:04:27.743 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-11T08:04:27.743 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-11T08:04:27.743 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-11T08:04:27.743 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rados-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T08:04:27.743 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rbd-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T08:04:27.743 INFO:teuthology.orchestra.run.smithi204.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-11T08:04:27.743 INFO:teuthology.orchestra.run.smithi204.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-11T08:04:27.744 INFO:teuthology.orchestra.run.smithi204.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-11T08:04:27.744 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rgw-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T08:04:27.744 INFO:teuthology.orchestra.run.smithi204.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-11T08:04:27.744 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-11T08:04:27.744 INFO:teuthology.orchestra.run.smithi204.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-11T08:04:27.744 INFO:teuthology.orchestra.run.smithi204.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-11T08:04:27.744 INFO:teuthology.orchestra.run.smithi204.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-11T08:04:27.744 INFO:teuthology.orchestra.run.smithi204.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-11T08:04:27.744 INFO:teuthology.orchestra.run.smithi204.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-11T08:04:27.744 INFO:teuthology.orchestra.run.smithi204.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-11T08:04:27.744 INFO:teuthology.orchestra.run.smithi204.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-11T08:04:27.744 INFO:teuthology.orchestra.run.smithi204.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-11T08:04:27.744 INFO:teuthology.orchestra.run.smithi204.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-11T08:04:27.744 INFO:teuthology.orchestra.run.smithi204.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-11T08:04:27.745 INFO:teuthology.orchestra.run.smithi204.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-11T08:04:27.745 INFO:teuthology.orchestra.run.smithi204.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-11T08:04:27.745 INFO:teuthology.orchestra.run.smithi204.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-11T08:04:27.745 INFO:teuthology.orchestra.run.smithi204.stdout: rbd-fuse-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T08:04:27.745 INFO:teuthology.orchestra.run.smithi204.stdout: rbd-mirror-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T08:04:27.745 INFO:teuthology.orchestra.run.smithi204.stdout: rbd-nbd-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T08:04:27.745 INFO:teuthology.orchestra.run.smithi204.stdout: re2-1:20211101-3.el9.x86_64 2024-05-11T08:04:27.745 INFO:teuthology.orchestra.run.smithi204.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-11T08:04:27.745 INFO:teuthology.orchestra.run.smithi204.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-11T08:04:27.745 INFO:teuthology.orchestra.run.smithi204.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-11T08:04:27.745 INFO:teuthology.orchestra.run.smithi204.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-11T08:04:27.745 INFO:teuthology.orchestra.run.smithi204.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-11T08:04:27.746 INFO:teuthology.orchestra.run.smithi204.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-11T08:04:27.746 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T08:04:27.746 INFO:teuthology.orchestra.run.smithi204.stdout:Complete! 2024-05-11T08:04:28.217 DEBUG:teuthology.parallel:result is None 2024-05-11T08:04:29.285 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3601.g30850680.el9.x86_64 109/164 2024-05-11T08:04:29.285 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /sys 2024-05-11T08:04:29.285 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /proc 2024-05-11T08:04:29.285 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /mnt 2024-05-11T08:04:29.285 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /var/tmp 2024-05-11T08:04:29.285 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /home 2024-05-11T08:04:29.285 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /root 2024-05-11T08:04:29.285 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /tmp 2024-05-11T08:04:29.286 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T08:04:29.460 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-volume-2:19.0.0-3601.g30850680.el9.noarch 110/164 2024-05-11T08:04:29.475 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-volume-2:19.0.0-3601.g30850680.el9.noarch 110/164 2024-05-11T08:04:29.475 INFO:teuthology.orchestra.run.smithi167.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-11T08:04:29.476 INFO:teuthology.orchestra.run.smithi167.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-11T08:04:29.476 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T08:04:30.988 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-osd-2:19.0.0-3601.g30850680.el9.x86_64 111/164 2024-05-11T08:04:31.014 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-osd-2:19.0.0-3601.g30850680.el9.x86_64 111/164 2024-05-11T08:04:31.015 INFO:teuthology.orchestra.run.smithi167.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-11T08:04:31.015 INFO:teuthology.orchestra.run.smithi167.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-11T08:04:31.015 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-11T08:04:31.015 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-11T08:04:31.015 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T08:04:31.238 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mds-2:19.0.0-3601.g30850680.el9.x86_64 112/164 2024-05-11T08:04:31.261 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-mds-2:19.0.0-3601.g30850680.el9.x86_64 112/164 2024-05-11T08:04:31.262 INFO:teuthology.orchestra.run.smithi167.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-11T08:04:31.262 INFO:teuthology.orchestra.run.smithi167.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-11T08:04:31.262 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-11T08:04:31.262 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-11T08:04:31.262 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T08:04:31.717 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mon-2:19.0.0-3601.g30850680.el9.x86_64 113/164 2024-05-11T08:04:31.740 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-mon-2:19.0.0-3601.g30850680.el9.x86_64 113/164 2024-05-11T08:04:31.740 INFO:teuthology.orchestra.run.smithi167.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-11T08:04:31.740 INFO:teuthology.orchestra.run.smithi167.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-11T08:04:31.740 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-11T08:04:31.740 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-11T08:04:31.740 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T08:04:31.844 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-05-11T08:04:31.902 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-05-11T08:04:31.957 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-05-11T08:04:32.144 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-05-11T08:04:32.190 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-05-11T08:04:32.268 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-05-11T08:04:32.357 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-05-11T08:04:32.507 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-05-11T08:04:32.553 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-05-11T08:04:32.589 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-05-11T08:04:32.617 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : fuse-2.9.9-15.el9.x86_64 124/164 2024-05-11T08:04:33.004 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: cephadm-2:19.0.0-3601.g30850680.el9.noarch 125/164 2024-05-11T08:04:33.032 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : cephadm-2:19.0.0-3601.g30850680.el9.noarch 125/164 2024-05-11T08:04:33.064 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3601.g30850680.e 126/164 2024-05-11T08:04:33.126 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3601.g30850680. 127/164 2024-05-11T08:04:33.155 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3601.g30850680.el9.noa 128/164 2024-05-11T08:04:34.191 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3601.g30850680.el9.noa 128/164 2024-05-11T08:04:34.224 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3601.g30850680.el9.n 129/164 2024-05-11T08:04:35.182 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3601.g30850680.el9.n 129/164 2024-05-11T08:04:35.207 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3601.g308 130/164 2024-05-11T08:04:35.318 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3601.g308 130/164 2024-05-11T08:04:35.469 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3601.g30850680.el 131/164 2024-05-11T08:04:35.509 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mgr-2:19.0.0-3601.g30850680.el9.x86_64 132/164 2024-05-11T08:04:35.533 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3601.g30850680.el9.x86_64 132/164 2024-05-11T08:04:35.533 INFO:teuthology.orchestra.run.smithi167.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-11T08:04:35.534 INFO:teuthology.orchestra.run.smithi167.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-11T08:04:35.534 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-11T08:04:35.534 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-11T08:04:35.534 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T08:04:35.577 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mgr-rook-2:19.0.0-3601.g30850680.el9.noarch 133/164 2024-05-11T08:04:35.588 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3601.g30850680.el9.noarch 133/164 2024-05-11T08:04:35.694 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-2:19.0.0-3601.g30850680.el9.x86_64 134/164 2024-05-11T08:04:35.749 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-fuse-2:19.0.0-3601.g30850680.el9.x86_64 135/164 2024-05-11T08:04:36.673 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libtool-2.4.6-45.el9.x86_64 136/164 2024-05-11T08:04:36.704 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-radosgw-2:19.0.0-3601.g30850680.el9.x86_64 137/164 2024-05-11T08:04:36.728 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3601.g30850680.el9.x86_64 137/164 2024-05-11T08:04:36.728 INFO:teuthology.orchestra.run.smithi167.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-11T08:04:36.728 INFO:teuthology.orchestra.run.smithi167.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-11T08:04:36.728 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-11T08:04:36.728 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-11T08:04:36.728 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T08:04:36.772 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3601.g30850 138/164 2024-05-11T08:04:36.794 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3601.g30850 138/164 2024-05-11T08:04:36.794 INFO:teuthology.orchestra.run.smithi167.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-11T08:04:36.794 INFO:teuthology.orchestra.run.smithi167.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-11T08:04:36.794 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T08:04:36.839 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : cephfs-mirror-2:19.0.0-3601.g30850680.el9.x86_64 139/164 2024-05-11T08:04:36.862 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3601.g30850680.el9.x86_64 139/164 2024-05-11T08:04:36.862 INFO:teuthology.orchestra.run.smithi167.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-11T08:04:36.863 INFO:teuthology.orchestra.run.smithi167.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-11T08:04:36.863 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-11T08:04:36.863 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-11T08:04:36.863 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T08:04:37.165 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : rbd-mirror-2:19.0.0-3601.g30850680.el9.x86_64 140/164 2024-05-11T08:04:37.187 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3601.g30850680.el9.x86_64 140/164 2024-05-11T08:04:37.188 INFO:teuthology.orchestra.run.smithi167.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-11T08:04:37.188 INFO:teuthology.orchestra.run.smithi167.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-11T08:04:37.188 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-11T08:04:37.188 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-11T08:04:37.188 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T08:04:41.819 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-test-2:19.0.0-3601.g30850680.el9.x86_64 141/164 2024-05-11T08:04:41.862 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : rbd-fuse-2:19.0.0-3601.g30850680.el9.x86_64 142/164 2024-05-11T08:04:41.901 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : rbd-nbd-2:19.0.0-3601.g30850680.el9.x86_64 143/164 2024-05-11T08:04:41.933 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 144/164 2024-05-11T08:04:41.960 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 145/164 2024-05-11T08:04:42.156 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : cephfs-shell-2:19.0.0-3601.g30850680.el9.x86_64 146/164 2024-05-11T08:04:42.207 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 147/164 2024-05-11T08:04:42.236 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libcephfs-devel-2:19.0.0-3601.g30850680.el9.x86_ 148/164 2024-05-11T08:04:42.266 INFO:teuthology.orchestra.run.smithi167.stdout: Upgrading : NetworkManager-team-1:1.47.5-1.el9.x86_64 149/164 2024-05-11T08:04:42.312 INFO:teuthology.orchestra.run.smithi167.stdout: Upgrading : NetworkManager-tui-1:1.47.5-1.el9.x86_64 150/164 2024-05-11T08:04:42.989 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-11T08:04:43.027 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-11T08:04:43.047 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-11T08:04:43.890 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : cephfs-top-2:19.0.0-3601.g30850680.el9.noarch 152/164 2024-05-11T08:04:43.982 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 153/164 2024-05-11T08:04:44.033 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-05-11T08:04:44.133 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-05-11T08:04:44.168 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-05-11T08:04:44.194 INFO:teuthology.orchestra.run.smithi167.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 157/164 2024-05-11T08:04:44.194 INFO:teuthology.orchestra.run.smithi167.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-11T08:04:44.270 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-11T08:04:44.303 INFO:teuthology.orchestra.run.smithi167.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 159/164 2024-05-11T08:04:44.303 INFO:teuthology.orchestra.run.smithi167.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-11T08:04:44.388 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-11T08:04:44.412 INFO:teuthology.orchestra.run.smithi167.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 161/164 2024-05-11T08:04:44.417 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-11T08:04:44.417 INFO:teuthology.orchestra.run.smithi167.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-11T08:04:44.606 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-11T08:04:44.607 INFO:teuthology.orchestra.run.smithi167.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-11T08:04:44.699 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-11T08:04:44.764 INFO:teuthology.orchestra.run.smithi167.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-11T08:04:50.429 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-11T08:04:50.430 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-2:19.0.0-3601.g30850680.el9.x86_64 1/164 2024-05-11T08:04:50.430 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-base-2:19.0.0-3601.g30850680.el9.x86_64 2/164 2024-05-11T08:04:50.430 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-common-2:19.0.0-3601.g30850680.el9.x86_64 3/164 2024-05-11T08:04:50.430 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-fuse-2:19.0.0-3601.g30850680.el9.x86_64 4/164 2024-05-11T08:04:50.430 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3601.g30850 5/164 2024-05-11T08:04:50.430 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mds-2:19.0.0-3601.g30850680.el9.x86_64 6/164 2024-05-11T08:04:50.430 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mgr-2:19.0.0-3601.g30850680.el9.x86_64 7/164 2024-05-11T08:04:50.430 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mon-2:19.0.0-3601.g30850680.el9.x86_64 8/164 2024-05-11T08:04:50.430 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-osd-2:19.0.0-3601.g30850680.el9.x86_64 9/164 2024-05-11T08:04:50.431 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-radosgw-2:19.0.0-3601.g30850680.el9.x86_64 10/164 2024-05-11T08:04:50.431 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-selinux-2:19.0.0-3601.g30850680.el9.x86_64 11/164 2024-05-11T08:04:50.431 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-test-2:19.0.0-3601.g30850680.el9.x86_64 12/164 2024-05-11T08:04:50.431 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : cephfs-mirror-2:19.0.0-3601.g30850680.el9.x86_64 13/164 2024-05-11T08:04:50.431 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : cephfs-shell-2:19.0.0-3601.g30850680.el9.x86_64 14/164 2024-05-11T08:04:50.431 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libcephfs-devel-2:19.0.0-3601.g30850680.el9.x86_ 15/164 2024-05-11T08:04:50.431 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libcephfs2-2:19.0.0-3601.g30850680.el9.x86_64 16/164 2024-05-11T08:04:50.431 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libcephsqlite-2:19.0.0-3601.g30850680.el9.x86_64 17/164 2024-05-11T08:04:50.431 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librados-devel-2:19.0.0-3601.g30850680.el9.x86_6 18/164 2024-05-11T08:04:50.431 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libradosstriper1-2:19.0.0-3601.g30850680.el9.x86 19/164 2024-05-11T08:04:50.431 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librgw2-2:19.0.0-3601.g30850680.el9.x86_64 20/164 2024-05-11T08:04:50.431 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3601.g30850680.el 21/164 2024-05-11T08:04:50.431 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-ceph-common-2:19.0.0-3601.g30850680.el9. 22/164 2024-05-11T08:04:50.431 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-cephfs-2:19.0.0-3601.g30850680.el9.x86_6 23/164 2024-05-11T08:04:50.432 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-rados-2:19.0.0-3601.g30850680.el9.x86_64 24/164 2024-05-11T08:04:50.432 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-rbd-2:19.0.0-3601.g30850680.el9.x86_64 25/164 2024-05-11T08:04:50.432 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-rgw-2:19.0.0-3601.g30850680.el9.x86_64 26/164 2024-05-11T08:04:50.432 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : rbd-fuse-2:19.0.0-3601.g30850680.el9.x86_64 27/164 2024-05-11T08:04:50.444 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : rbd-mirror-2:19.0.0-3601.g30850680.el9.x86_64 28/164 2024-05-11T08:04:50.444 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : rbd-nbd-2:19.0.0-3601.g30850680.el9.x86_64 29/164 2024-05-11T08:04:50.444 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3601.g30850680. 30/164 2024-05-11T08:04:50.444 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3601.g30850680.el9.noa 31/164 2024-05-11T08:04:50.444 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3601.g30850680.el9.n 32/164 2024-05-11T08:04:50.444 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3601.g308 33/164 2024-05-11T08:04:50.444 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3601.g30850680.el 34/164 2024-05-11T08:04:50.444 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3601.g30850680.el9.noarch 35/164 2024-05-11T08:04:50.445 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3601.g30850680.e 36/164 2024-05-11T08:04:50.445 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-volume-2:19.0.0-3601.g30850680.el9.noarch 37/164 2024-05-11T08:04:50.445 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : cephadm-2:19.0.0-3601.g30850680.el9.noarch 38/164 2024-05-11T08:04:50.445 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : cephfs-top-2:19.0.0-3601.g30850680.el9.noarch 39/164 2024-05-11T08:04:50.445 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/164 2024-05-11T08:04:50.445 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/164 2024-05-11T08:04:50.445 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/164 2024-05-11T08:04:50.445 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-05-11T08:04:50.445 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-05-11T08:04:50.445 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-05-11T08:04:50.445 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-05-11T08:04:50.445 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-05-11T08:04:50.445 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/164 2024-05-11T08:04:50.445 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-05-11T08:04:50.446 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-05-11T08:04:50.446 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-05-11T08:04:50.446 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/164 2024-05-11T08:04:50.446 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-05-11T08:04:50.446 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-05-11T08:04:50.446 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-05-11T08:04:50.446 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-05-11T08:04:50.446 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-05-11T08:04:50.446 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-05-11T08:04:50.446 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-05-11T08:04:50.446 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-05-11T08:04:50.446 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/164 2024-05-11T08:04:50.446 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/164 2024-05-11T08:04:50.447 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-05-11T08:04:50.447 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-05-11T08:04:50.447 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-05-11T08:04:50.447 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/164 2024-05-11T08:04:50.447 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/164 2024-05-11T08:04:50.447 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/164 2024-05-11T08:04:50.447 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/164 2024-05-11T08:04:50.447 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/164 2024-05-11T08:04:50.447 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/164 2024-05-11T08:04:50.447 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/164 2024-05-11T08:04:50.447 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-05-11T08:04:50.447 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-05-11T08:04:50.447 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-05-11T08:04:50.447 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/164 2024-05-11T08:04:50.447 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/164 2024-05-11T08:04:50.448 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/164 2024-05-11T08:04:50.448 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/164 2024-05-11T08:04:50.448 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/164 2024-05-11T08:04:50.448 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/164 2024-05-11T08:04:50.448 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/164 2024-05-11T08:04:50.448 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/164 2024-05-11T08:04:50.448 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/164 2024-05-11T08:04:50.448 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/164 2024-05-11T08:04:50.448 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/164 2024-05-11T08:04:50.448 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/164 2024-05-11T08:04:50.448 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/164 2024-05-11T08:04:50.448 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/164 2024-05-11T08:04:50.448 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/164 2024-05-11T08:04:50.448 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/164 2024-05-11T08:04:50.448 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/164 2024-05-11T08:04:50.448 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/164 2024-05-11T08:04:50.449 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/164 2024-05-11T08:04:50.449 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/164 2024-05-11T08:04:50.449 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/164 2024-05-11T08:04:50.449 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/164 2024-05-11T08:04:50.449 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/164 2024-05-11T08:04:50.449 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/164 2024-05-11T08:04:50.449 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/164 2024-05-11T08:04:50.449 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/164 2024-05-11T08:04:50.449 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/164 2024-05-11T08:04:50.449 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/164 2024-05-11T08:04:50.449 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/164 2024-05-11T08:04:50.449 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/164 2024-05-11T08:04:50.449 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/164 2024-05-11T08:04:50.449 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/164 2024-05-11T08:04:50.450 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/164 2024-05-11T08:04:50.450 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/164 2024-05-11T08:04:50.450 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/164 2024-05-11T08:04:50.450 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/164 2024-05-11T08:04:50.450 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/164 2024-05-11T08:04:50.450 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/164 2024-05-11T08:04:50.450 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/164 2024-05-11T08:04:50.450 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/164 2024-05-11T08:04:50.450 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/164 2024-05-11T08:04:50.450 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/164 2024-05-11T08:04:50.450 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/164 2024-05-11T08:04:50.450 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/164 2024-05-11T08:04:50.450 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/164 2024-05-11T08:04:50.450 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/164 2024-05-11T08:04:50.451 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/164 2024-05-11T08:04:50.451 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-05-11T08:04:50.451 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-05-11T08:04:50.451 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-05-11T08:04:50.451 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-05-11T08:04:50.451 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-05-11T08:04:50.451 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-05-11T08:04:50.451 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-05-11T08:04:50.451 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-05-11T08:04:50.451 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-05-11T08:04:50.451 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-05-11T08:04:50.451 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-05-11T08:04:50.451 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-05-11T08:04:50.451 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-05-11T08:04:50.451 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-05-11T08:04:50.452 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-05-11T08:04:50.452 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-05-11T08:04:50.452 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-05-11T08:04:50.452 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-05-11T08:04:50.452 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-05-11T08:04:50.452 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/164 2024-05-11T08:04:50.452 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/164 2024-05-11T08:04:50.452 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-05-11T08:04:50.452 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-05-11T08:04:50.452 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-05-11T08:04:50.452 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/164 2024-05-11T08:04:50.452 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-05-11T08:04:50.452 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librados2-2:19.0.0-3601.g30850680.el9.x86_64 149/164 2024-05-11T08:04:50.452 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-05-11T08:04:50.453 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librbd1-2:19.0.0-3601.g30850680.el9.x86_64 151/164 2024-05-11T08:04:50.453 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-05-11T08:04:50.453 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : NetworkManager-1:1.47.5-1.el9.x86_64 153/164 2024-05-11T08:04:50.453 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/164 2024-05-11T08:04:50.453 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 155/164 2024-05-11T08:04:50.453 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/164 2024-05-11T08:04:50.453 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : NetworkManager-team-1:1.47.5-1.el9.x86_64 157/164 2024-05-11T08:04:50.453 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/164 2024-05-11T08:04:50.453 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : NetworkManager-tui-1:1.47.5-1.el9.x86_64 159/164 2024-05-11T08:04:50.453 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/164 2024-05-11T08:04:50.453 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 161/164 2024-05-11T08:04:50.453 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 162/164 2024-05-11T08:04:50.453 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 163/164 2024-05-11T08:04:53.290 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-11T08:04:53.290 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T08:04:53.290 INFO:teuthology.orchestra.run.smithi167.stdout:Upgraded: 2024-05-11T08:04:53.291 INFO:teuthology.orchestra.run.smithi167.stdout: NetworkManager-1:1.47.5-1.el9.x86_64 2024-05-11T08:04:53.291 INFO:teuthology.orchestra.run.smithi167.stdout: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 2024-05-11T08:04:53.291 INFO:teuthology.orchestra.run.smithi167.stdout: NetworkManager-team-1:1.47.5-1.el9.x86_64 2024-05-11T08:04:53.291 INFO:teuthology.orchestra.run.smithi167.stdout: NetworkManager-tui-1:1.47.5-1.el9.x86_64 2024-05-11T08:04:53.291 INFO:teuthology.orchestra.run.smithi167.stdout: librados2-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T08:04:53.291 INFO:teuthology.orchestra.run.smithi167.stdout: librbd1-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T08:04:53.291 INFO:teuthology.orchestra.run.smithi167.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-11T08:04:53.291 INFO:teuthology.orchestra.run.smithi167.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-11T08:04:53.291 INFO:teuthology.orchestra.run.smithi167.stdout:Installed: 2024-05-11T08:04:53.291 INFO:teuthology.orchestra.run.smithi167.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-11T08:04:53.291 INFO:teuthology.orchestra.run.smithi167.stdout: automake-1.16.2-8.el9.noarch 2024-05-11T08:04:53.291 INFO:teuthology.orchestra.run.smithi167.stdout: bison-3.7.4-5.el9.x86_64 2024-05-11T08:04:53.291 INFO:teuthology.orchestra.run.smithi167.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-11T08:04:53.291 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T08:04:53.292 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-base-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T08:04:53.292 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-common-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T08:04:53.292 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-fuse-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T08:04:53.292 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-grafana-dashboards-2:19.0.0-3601.g30850680.el9.noarch 2024-05-11T08:04:53.292 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-immutable-object-cache-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T08:04:53.292 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mds-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T08:04:53.292 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T08:04:53.292 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-cephadm-2:19.0.0-3601.g30850680.el9.noarch 2024-05-11T08:04:53.292 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-dashboard-2:19.0.0-3601.g30850680.el9.noarch 2024-05-11T08:04:53.292 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3601.g30850680.el9.noarch 2024-05-11T08:04:53.292 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-modules-core-2:19.0.0-3601.g30850680.el9.noarch 2024-05-11T08:04:53.292 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-rook-2:19.0.0-3601.g30850680.el9.noarch 2024-05-11T08:04:53.293 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mon-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T08:04:53.293 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-osd-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T08:04:53.293 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-prometheus-alerts-2:19.0.0-3601.g30850680.el9.noarch 2024-05-11T08:04:53.293 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-radosgw-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T08:04:53.293 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-selinux-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T08:04:53.293 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-test-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T08:04:53.293 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-volume-2:19.0.0-3601.g30850680.el9.noarch 2024-05-11T08:04:53.293 INFO:teuthology.orchestra.run.smithi167.stdout: cephadm-2:19.0.0-3601.g30850680.el9.noarch 2024-05-11T08:04:53.293 INFO:teuthology.orchestra.run.smithi167.stdout: cephfs-mirror-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T08:04:53.293 INFO:teuthology.orchestra.run.smithi167.stdout: cephfs-shell-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T08:04:53.293 INFO:teuthology.orchestra.run.smithi167.stdout: cephfs-top-2:19.0.0-3601.g30850680.el9.noarch 2024-05-11T08:04:53.293 INFO:teuthology.orchestra.run.smithi167.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-05-11T08:04:53.293 INFO:teuthology.orchestra.run.smithi167.stdout: flex-2.6.4-9.el9.x86_64 2024-05-11T08:04:53.294 INFO:teuthology.orchestra.run.smithi167.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-11T08:04:53.294 INFO:teuthology.orchestra.run.smithi167.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-11T08:04:53.294 INFO:teuthology.orchestra.run.smithi167.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-11T08:04:53.294 INFO:teuthology.orchestra.run.smithi167.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-11T08:04:53.294 INFO:teuthology.orchestra.run.smithi167.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-11T08:04:53.294 INFO:teuthology.orchestra.run.smithi167.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-11T08:04:53.294 INFO:teuthology.orchestra.run.smithi167.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-11T08:04:53.294 INFO:teuthology.orchestra.run.smithi167.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-11T08:04:53.294 INFO:teuthology.orchestra.run.smithi167.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-11T08:04:53.294 INFO:teuthology.orchestra.run.smithi167.stdout: libcephfs-devel-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T08:04:53.294 INFO:teuthology.orchestra.run.smithi167.stdout: libcephfs2-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T08:04:53.294 INFO:teuthology.orchestra.run.smithi167.stdout: libcephsqlite-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T08:04:53.294 INFO:teuthology.orchestra.run.smithi167.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-11T08:04:53.294 INFO:teuthology.orchestra.run.smithi167.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-11T08:04:53.294 INFO:teuthology.orchestra.run.smithi167.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-11T08:04:53.295 INFO:teuthology.orchestra.run.smithi167.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-11T08:04:53.295 INFO:teuthology.orchestra.run.smithi167.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-11T08:04:53.295 INFO:teuthology.orchestra.run.smithi167.stdout: librados-devel-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T08:04:53.295 INFO:teuthology.orchestra.run.smithi167.stdout: libradosstriper1-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T08:04:53.295 INFO:teuthology.orchestra.run.smithi167.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-11T08:04:53.295 INFO:teuthology.orchestra.run.smithi167.stdout: librgw2-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T08:04:53.295 INFO:teuthology.orchestra.run.smithi167.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-11T08:04:53.295 INFO:teuthology.orchestra.run.smithi167.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-11T08:04:53.295 INFO:teuthology.orchestra.run.smithi167.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-11T08:04:53.295 INFO:teuthology.orchestra.run.smithi167.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-11T08:04:53.295 INFO:teuthology.orchestra.run.smithi167.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-11T08:04:53.295 INFO:teuthology.orchestra.run.smithi167.stdout: lua-5.4.4-4.el9.x86_64 2024-05-11T08:04:53.295 INFO:teuthology.orchestra.run.smithi167.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-11T08:04:53.295 INFO:teuthology.orchestra.run.smithi167.stdout: luarocks-3.9.2-1.el9.noarch 2024-05-11T08:04:53.296 INFO:teuthology.orchestra.run.smithi167.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-11T08:04:53.296 INFO:teuthology.orchestra.run.smithi167.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-11T08:04:53.296 INFO:teuthology.orchestra.run.smithi167.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-11T08:04:53.296 INFO:teuthology.orchestra.run.smithi167.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-11T08:04:53.296 INFO:teuthology.orchestra.run.smithi167.stdout: postgresql-13.14-1.el9.x86_64 2024-05-11T08:04:53.296 INFO:teuthology.orchestra.run.smithi167.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-11T08:04:53.296 INFO:teuthology.orchestra.run.smithi167.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-11T08:04:53.296 INFO:teuthology.orchestra.run.smithi167.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-11T08:04:53.296 INFO:teuthology.orchestra.run.smithi167.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-11T08:04:53.296 INFO:teuthology.orchestra.run.smithi167.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-11T08:04:53.296 INFO:teuthology.orchestra.run.smithi167.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-11T08:04:53.296 INFO:teuthology.orchestra.run.smithi167.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-11T08:04:53.296 INFO:teuthology.orchestra.run.smithi167.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-11T08:04:53.297 INFO:teuthology.orchestra.run.smithi167.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-11T08:04:53.297 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-11T08:04:53.297 INFO:teuthology.orchestra.run.smithi167.stdout: python3-ceph-argparse-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T08:04:53.297 INFO:teuthology.orchestra.run.smithi167.stdout: python3-ceph-common-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T08:04:53.297 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cephfs-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T08:04:53.297 INFO:teuthology.orchestra.run.smithi167.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-11T08:04:53.297 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-11T08:04:53.297 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-11T08:04:53.297 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-11T08:04:53.297 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-11T08:04:53.297 INFO:teuthology.orchestra.run.smithi167.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-05-11T08:04:53.297 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-11T08:04:53.297 INFO:teuthology.orchestra.run.smithi167.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-11T08:04:53.297 INFO:teuthology.orchestra.run.smithi167.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-11T08:04:53.297 INFO:teuthology.orchestra.run.smithi167.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-11T08:04:53.298 INFO:teuthology.orchestra.run.smithi167.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-11T08:04:53.298 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-11T08:04:53.298 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-11T08:04:53.298 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-11T08:04:53.298 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-11T08:04:53.298 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-11T08:04:53.298 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-11T08:04:53.298 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-11T08:04:53.298 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-11T08:04:53.298 INFO:teuthology.orchestra.run.smithi167.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-11T08:04:53.298 INFO:teuthology.orchestra.run.smithi167.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-11T08:04:53.298 INFO:teuthology.orchestra.run.smithi167.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-11T08:04:53.298 INFO:teuthology.orchestra.run.smithi167.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-11T08:04:53.299 INFO:teuthology.orchestra.run.smithi167.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-11T08:04:53.299 INFO:teuthology.orchestra.run.smithi167.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-11T08:04:53.299 INFO:teuthology.orchestra.run.smithi167.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-11T08:04:53.299 INFO:teuthology.orchestra.run.smithi167.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-11T08:04:53.299 INFO:teuthology.orchestra.run.smithi167.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-11T08:04:53.299 INFO:teuthology.orchestra.run.smithi167.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-11T08:04:53.299 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-11T08:04:53.299 INFO:teuthology.orchestra.run.smithi167.stdout: python3-ply-3.11-14.el9.noarch 2024-05-11T08:04:53.299 INFO:teuthology.orchestra.run.smithi167.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-11T08:04:53.299 INFO:teuthology.orchestra.run.smithi167.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-11T08:04:53.299 INFO:teuthology.orchestra.run.smithi167.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-11T08:04:53.299 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-11T08:04:53.299 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-11T08:04:53.299 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-11T08:04:53.300 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-11T08:04:53.300 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-11T08:04:53.300 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-11T08:04:53.300 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rados-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T08:04:53.300 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rbd-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T08:04:53.300 INFO:teuthology.orchestra.run.smithi167.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-11T08:04:53.300 INFO:teuthology.orchestra.run.smithi167.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-11T08:04:53.300 INFO:teuthology.orchestra.run.smithi167.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-11T08:04:53.300 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rgw-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T08:04:53.300 INFO:teuthology.orchestra.run.smithi167.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-11T08:04:53.300 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-11T08:04:53.300 INFO:teuthology.orchestra.run.smithi167.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-11T08:04:53.300 INFO:teuthology.orchestra.run.smithi167.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-11T08:04:53.300 INFO:teuthology.orchestra.run.smithi167.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-11T08:04:53.300 INFO:teuthology.orchestra.run.smithi167.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-11T08:04:53.301 INFO:teuthology.orchestra.run.smithi167.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-11T08:04:53.301 INFO:teuthology.orchestra.run.smithi167.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-11T08:04:53.301 INFO:teuthology.orchestra.run.smithi167.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-11T08:04:53.301 INFO:teuthology.orchestra.run.smithi167.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-11T08:04:53.301 INFO:teuthology.orchestra.run.smithi167.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-11T08:04:53.301 INFO:teuthology.orchestra.run.smithi167.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-11T08:04:53.301 INFO:teuthology.orchestra.run.smithi167.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-11T08:04:53.301 INFO:teuthology.orchestra.run.smithi167.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-11T08:04:53.301 INFO:teuthology.orchestra.run.smithi167.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-11T08:04:53.301 INFO:teuthology.orchestra.run.smithi167.stdout: rbd-fuse-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T08:04:53.301 INFO:teuthology.orchestra.run.smithi167.stdout: rbd-mirror-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T08:04:53.302 INFO:teuthology.orchestra.run.smithi167.stdout: rbd-nbd-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T08:04:53.302 INFO:teuthology.orchestra.run.smithi167.stdout: re2-1:20211101-3.el9.x86_64 2024-05-11T08:04:53.302 INFO:teuthology.orchestra.run.smithi167.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-11T08:04:53.302 INFO:teuthology.orchestra.run.smithi167.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-11T08:04:53.302 INFO:teuthology.orchestra.run.smithi167.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-11T08:04:53.302 INFO:teuthology.orchestra.run.smithi167.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-11T08:04:53.302 INFO:teuthology.orchestra.run.smithi167.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-11T08:04:53.302 INFO:teuthology.orchestra.run.smithi167.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-11T08:04:53.302 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T08:04:53.303 INFO:teuthology.orchestra.run.smithi167.stdout:Complete! 2024-05-11T08:04:53.738 DEBUG:teuthology.parallel:result is None 2024-05-11T08:04:53.739 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=308506802b0bdea4df7050cfc1fcbdc04fea6a6e 2024-05-11T08:04:53.959 DEBUG:teuthology.orchestra.run.smithi167:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-11T08:04:53.983 INFO:teuthology.orchestra.run.smithi167.stdout:19.0.0-3601.g30850680.el9 2024-05-11T08:04:53.983 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3601.g30850680.el9 2024-05-11T08:04:53.983 INFO:teuthology.task.install:The correct ceph version 19.0.0-3601.g30850680 is installed. 2024-05-11T08:04:53.986 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=308506802b0bdea4df7050cfc1fcbdc04fea6a6e 2024-05-11T08:04:54.199 DEBUG:teuthology.orchestra.run.smithi204:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-11T08:04:54.226 INFO:teuthology.orchestra.run.smithi204.stdout:19.0.0-3601.g30850680.el9 2024-05-11T08:04:54.226 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3601.g30850680.el9 2024-05-11T08:04:54.227 INFO:teuthology.task.install:The correct ceph version 19.0.0-3601.g30850680 is installed. 2024-05-11T08:04:54.228 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-05-11T08:04:54.229 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-05-11T08:04:54.229 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-11T08:04:54.257 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-11T08:04:54.257 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-11T08:04:54.298 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-05-11T08:04:54.298 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-05-11T08:04:54.298 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/usr/bin/daemon-helper 2024-05-11T08:04:54.324 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-11T08:04:54.390 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-11T08:04:54.390 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/usr/bin/daemon-helper 2024-05-11T08:04:54.417 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-11T08:04:54.484 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-05-11T08:04:54.484 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-05-11T08:04:54.484 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-11T08:04:54.509 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-11T08:04:54.574 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-11T08:04:54.574 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-11T08:04:54.600 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-11T08:04:54.667 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-05-11T08:04:54.667 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-05-11T08:04:54.667 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/usr/bin/stdin-killer 2024-05-11T08:04:54.691 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-11T08:04:54.756 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-11T08:04:54.756 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/usr/bin/stdin-killer 2024-05-11T08:04:54.781 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-11T08:04:54.847 INFO:teuthology.run_tasks:Running task ceph... 2024-05-11T08:04:54.930 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-05-11T08:04:54.930 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod 777 /var/log/ceph 2024-05-11T08:04:54.933 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod 777 /var/log/ceph 2024-05-11T08:04:54.958 INFO:tasks.ceph:Disabling ceph logrotate... 2024-05-11T08:04:54.958 DEBUG:teuthology.orchestra.run.smithi167:> sudo rm -f -- /etc/logrotate.d/ceph 2024-05-11T08:04:54.997 DEBUG:teuthology.orchestra.run.smithi204:> sudo rm -f -- /etc/logrotate.d/ceph 2024-05-11T08:04:55.023 INFO:tasks.ceph:Creating extra log directories... 2024-05-11T08:04:55.023 DEBUG:teuthology.orchestra.run.smithi167:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-05-11T08:04:55.061 DEBUG:teuthology.orchestra.run.smithi204:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-05-11T08:04:55.095 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-05-11T08:04:55.095 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-05-11T08:04:55.095 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-05-11T08:04:55.095 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-05-11T08:04:55.095 DEBUG:teuthology.orchestra.run.smithi167:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-05-11T08:04:55.139 DEBUG:teuthology.orchestra.run.smithi167:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-05-11T08:04:55.250 DEBUG:teuthology.orchestra.run.smithi167:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-05-11T08:04:55.275 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-11T08:04:55.275 DEBUG:teuthology.orchestra.run.smithi204:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-05-11T08:04:55.291 DEBUG:teuthology.orchestra.run.smithi204:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-05-11T08:04:55.401 DEBUG:teuthology.orchestra.run.smithi204:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-05-11T08:04:55.432 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-05-11T08:04:55.432 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-05-11T08:04:55.432 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/rishabh-2024-05-11_05:57:22-fs-wip-rishabh-testing-20240510.053205-testing-default-smithi/7702178', 'branch': 'wip-rishabh-testing-20240510.053205', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/backtrace}', 'email': None, 'first_in_suite': False, 'job_id': '7702178', 'kernel': {'client': {'sha1': 'distro'}, 'flavor': 'default', 'kdb': True, 'sha1': 'fb9a6a44107e0b9f9ca2d9a78f2308f5dd49911e'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'mgrmodules': {'sequential': [{'print': 'Enabling mgr modules'}]}, 'name': 'rishabh-2024-05-11_05:57:22-fs-wip-rishabh-testing-20240510.053205-testing-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-rishabh-testing-20240510.053205'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy'], 'sha1': '308506802b0bdea4df7050cfc1fcbdc04fea6a6e'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '308506802b0bdea4df7050cfc1fcbdc04fea6a6e'}}, 'kclient': {'syntax': 'v1'}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']}, 'subvolume_version': 2, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-rishabh-testing-20240510.053205', 'sha1': '308506802b0bdea4df7050cfc1fcbdc04fea6a6e'}}, 'owner': 'scheduled_rishabh@teuthology', 'priority': 75, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 4085, 'sha1': '308506802b0bdea4df7050cfc1fcbdc04fea6a6e', 'sleep_before_teardown': 0, 'subset': '100/512', 'suite': 'fs', 'suite_branch': 'wip-rishabh-testing-20240510.053205', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_308506802b0bdea4df7050cfc1fcbdc04fea6a6e/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '308506802b0bdea4df7050cfc1fcbdc04fea6a6e', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi167.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBK0JOctBAw4YxRwVXlCl/LZiTxFjn9XYNRMgJBHFDKxCGhC/FlEfWF9jjJ1rw+Eky8U41KhvsOQw5nWR/Aw7zyc=', 'smithi204.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPfxZSuUgNSMOyo/zhhjozhvdbiffHNZPmOu9r4pL2pJdOF/GmDg947fBKRdCppJba0CuOJjGYyUrua3hf+tOgE='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'sha1': 'distro'}, 'flavor': 'default', 'kdb': True, 'sha1': 'fb9a6a44107e0b9f9ca2d9a78f2308f5dd49911e'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '308506802b0bdea4df7050cfc1fcbdc04fea6a6e'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy'], 'sha1': '308506802b0bdea4df7050cfc1fcbdc04fea6a6e', 'cluster': 'ceph'}}, {'sequential': ['mgrmodules']}, {'kclient': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_backtrace']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': '1ae7ad82388e92a475afff437d49054826c019a1', 'timestamp': '2024-05-11_05:57:22', 'tube': 'smithi', 'user': 'rishabh', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-05-11T08:04:55.433 DEBUG:teuthology.orchestra.run.smithi167:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-05-11T08:04:55.436 DEBUG:teuthology.orchestra.run.smithi204:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-05-11T08:04:55.482 DEBUG:teuthology.orchestra.run.smithi167:> sudo install -d -m0777 -- /var/run/ceph 2024-05-11T08:04:55.491 DEBUG:teuthology.orchestra.run.smithi204:> sudo install -d -m0777 -- /var/run/ceph 2024-05-11T08:04:55.547 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-05-11T08:04:55.547 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/scratch_devs of=/dev/stdout 2024-05-11T08:04:55.570 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-11T08:04:55.570 DEBUG:teuthology.orchestra.run.smithi167:> stat /dev/vg_nvme/lv_1 2024-05-11T08:04:55.625 INFO:teuthology.orchestra.run.smithi167.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-11T08:04:55.625 INFO:teuthology.orchestra.run.smithi167.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-11T08:04:55.626 INFO:teuthology.orchestra.run.smithi167.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-05-11T08:04:55.626 INFO:teuthology.orchestra.run.smithi167.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-11T08:04:55.626 INFO:teuthology.orchestra.run.smithi167.stdout:Context: system_u:object_r:device_t:s0 2024-05-11T08:04:55.626 INFO:teuthology.orchestra.run.smithi167.stdout:Access: 2024-05-11 08:04:45.119491389 +0000 2024-05-11T08:04:55.626 INFO:teuthology.orchestra.run.smithi167.stdout:Modify: 2024-05-11 08:02:29.123381522 +0000 2024-05-11T08:04:55.626 INFO:teuthology.orchestra.run.smithi167.stdout:Change: 2024-05-11 08:02:29.123381522 +0000 2024-05-11T08:04:55.626 INFO:teuthology.orchestra.run.smithi167.stdout: Birth: 2024-05-11 08:02:29.123381522 +0000 2024-05-11T08:04:55.626 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-11T08:04:55.689 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records in 2024-05-11T08:04:55.689 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records out 2024-05-11T08:04:55.689 INFO:teuthology.orchestra.run.smithi167.stderr:512 bytes copied, 0.000282683 s, 1.8 MB/s 2024-05-11T08:04:55.690 DEBUG:teuthology.orchestra.run.smithi167:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-11T08:04:55.747 DEBUG:teuthology.orchestra.run.smithi167:> stat /dev/vg_nvme/lv_2 2024-05-11T08:04:55.803 INFO:teuthology.orchestra.run.smithi167.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-11T08:04:55.803 INFO:teuthology.orchestra.run.smithi167.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-11T08:04:55.803 INFO:teuthology.orchestra.run.smithi167.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-05-11T08:04:55.803 INFO:teuthology.orchestra.run.smithi167.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-11T08:04:55.803 INFO:teuthology.orchestra.run.smithi167.stdout:Context: system_u:object_r:device_t:s0 2024-05-11T08:04:55.803 INFO:teuthology.orchestra.run.smithi167.stdout:Access: 2024-05-11 08:04:45.120491372 +0000 2024-05-11T08:04:55.803 INFO:teuthology.orchestra.run.smithi167.stdout:Modify: 2024-05-11 08:02:29.466375744 +0000 2024-05-11T08:04:55.803 INFO:teuthology.orchestra.run.smithi167.stdout:Change: 2024-05-11 08:02:29.466375744 +0000 2024-05-11T08:04:55.803 INFO:teuthology.orchestra.run.smithi167.stdout: Birth: 2024-05-11 08:02:29.466375744 +0000 2024-05-11T08:04:55.804 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-11T08:04:55.866 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records in 2024-05-11T08:04:55.866 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records out 2024-05-11T08:04:55.866 INFO:teuthology.orchestra.run.smithi167.stderr:512 bytes copied, 0.000189696 s, 2.7 MB/s 2024-05-11T08:04:55.867 DEBUG:teuthology.orchestra.run.smithi167:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-11T08:04:55.924 DEBUG:teuthology.orchestra.run.smithi167:> stat /dev/vg_nvme/lv_3 2024-05-11T08:04:55.979 INFO:teuthology.orchestra.run.smithi167.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-11T08:04:55.979 INFO:teuthology.orchestra.run.smithi167.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-11T08:04:55.979 INFO:teuthology.orchestra.run.smithi167.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-05-11T08:04:55.979 INFO:teuthology.orchestra.run.smithi167.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-11T08:04:55.979 INFO:teuthology.orchestra.run.smithi167.stdout:Context: system_u:object_r:device_t:s0 2024-05-11T08:04:55.979 INFO:teuthology.orchestra.run.smithi167.stdout:Access: 2024-05-11 08:04:45.120491372 +0000 2024-05-11T08:04:55.979 INFO:teuthology.orchestra.run.smithi167.stdout:Modify: 2024-05-11 08:02:29.814369882 +0000 2024-05-11T08:04:55.979 INFO:teuthology.orchestra.run.smithi167.stdout:Change: 2024-05-11 08:02:29.814369882 +0000 2024-05-11T08:04:55.979 INFO:teuthology.orchestra.run.smithi167.stdout: Birth: 2024-05-11 08:02:29.814369882 +0000 2024-05-11T08:04:55.980 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-11T08:04:56.042 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records in 2024-05-11T08:04:56.042 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records out 2024-05-11T08:04:56.042 INFO:teuthology.orchestra.run.smithi167.stderr:512 bytes copied, 0.000185635 s, 2.8 MB/s 2024-05-11T08:04:56.043 DEBUG:teuthology.orchestra.run.smithi167:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-11T08:04:56.100 DEBUG:teuthology.orchestra.run.smithi167:> stat /dev/vg_nvme/lv_4 2024-05-11T08:04:56.156 INFO:teuthology.orchestra.run.smithi167.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-11T08:04:56.156 INFO:teuthology.orchestra.run.smithi167.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-11T08:04:56.156 INFO:teuthology.orchestra.run.smithi167.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-05-11T08:04:56.156 INFO:teuthology.orchestra.run.smithi167.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-11T08:04:56.156 INFO:teuthology.orchestra.run.smithi167.stdout:Context: system_u:object_r:device_t:s0 2024-05-11T08:04:56.156 INFO:teuthology.orchestra.run.smithi167.stdout:Access: 2024-05-11 08:04:45.120491372 +0000 2024-05-11T08:04:56.156 INFO:teuthology.orchestra.run.smithi167.stdout:Modify: 2024-05-11 08:02:30.175363801 +0000 2024-05-11T08:04:56.156 INFO:teuthology.orchestra.run.smithi167.stdout:Change: 2024-05-11 08:02:30.175363801 +0000 2024-05-11T08:04:56.156 INFO:teuthology.orchestra.run.smithi167.stdout: Birth: 2024-05-11 08:02:30.175363801 +0000 2024-05-11T08:04:56.156 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-11T08:04:56.219 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records in 2024-05-11T08:04:56.219 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records out 2024-05-11T08:04:56.219 INFO:teuthology.orchestra.run.smithi167.stderr:512 bytes copied, 0.000194444 s, 2.6 MB/s 2024-05-11T08:04:56.220 DEBUG:teuthology.orchestra.run.smithi167:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-11T08:04:56.277 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-11T08:04:56.277 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-11T08:04:56.277 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/scratch_devs of=/dev/stdout 2024-05-11T08:04:56.295 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-11T08:04:56.295 DEBUG:teuthology.orchestra.run.smithi204:> stat /dev/vg_nvme/lv_1 2024-05-11T08:04:56.350 INFO:teuthology.orchestra.run.smithi204.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-11T08:04:56.351 INFO:teuthology.orchestra.run.smithi204.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-11T08:04:56.351 INFO:teuthology.orchestra.run.smithi204.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-05-11T08:04:56.351 INFO:teuthology.orchestra.run.smithi204.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-11T08:04:56.351 INFO:teuthology.orchestra.run.smithi204.stdout:Context: system_u:object_r:device_t:s0 2024-05-11T08:04:56.351 INFO:teuthology.orchestra.run.smithi204.stdout:Access: 2024-05-11 08:04:19.614221689 +0000 2024-05-11T08:04:56.351 INFO:teuthology.orchestra.run.smithi204.stdout:Modify: 2024-05-11 08:02:23.453721908 +0000 2024-05-11T08:04:56.351 INFO:teuthology.orchestra.run.smithi204.stdout:Change: 2024-05-11 08:02:23.453721908 +0000 2024-05-11T08:04:56.351 INFO:teuthology.orchestra.run.smithi204.stdout: Birth: 2024-05-11 08:02:23.453721908 +0000 2024-05-11T08:04:56.351 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-11T08:04:56.414 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records in 2024-05-11T08:04:56.414 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records out 2024-05-11T08:04:56.415 INFO:teuthology.orchestra.run.smithi204.stderr:512 bytes copied, 0.000189035 s, 2.7 MB/s 2024-05-11T08:04:56.416 DEBUG:teuthology.orchestra.run.smithi204:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-11T08:04:56.472 DEBUG:teuthology.orchestra.run.smithi204:> stat /dev/vg_nvme/lv_2 2024-05-11T08:04:56.528 INFO:teuthology.orchestra.run.smithi204.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-11T08:04:56.529 INFO:teuthology.orchestra.run.smithi204.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-11T08:04:56.529 INFO:teuthology.orchestra.run.smithi204.stdout:Device: 5h/5d Inode: 687 Links: 1 2024-05-11T08:04:56.529 INFO:teuthology.orchestra.run.smithi204.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-11T08:04:56.529 INFO:teuthology.orchestra.run.smithi204.stdout:Context: system_u:object_r:device_t:s0 2024-05-11T08:04:56.529 INFO:teuthology.orchestra.run.smithi204.stdout:Access: 2024-05-11 08:04:19.615221677 +0000 2024-05-11T08:04:56.529 INFO:teuthology.orchestra.run.smithi204.stdout:Modify: 2024-05-11 08:02:23.829717306 +0000 2024-05-11T08:04:56.529 INFO:teuthology.orchestra.run.smithi204.stdout:Change: 2024-05-11 08:02:23.829717306 +0000 2024-05-11T08:04:56.529 INFO:teuthology.orchestra.run.smithi204.stdout: Birth: 2024-05-11 08:02:23.829717306 +0000 2024-05-11T08:04:56.529 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-11T08:04:56.592 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records in 2024-05-11T08:04:56.592 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records out 2024-05-11T08:04:56.592 INFO:teuthology.orchestra.run.smithi204.stderr:512 bytes copied, 0.000209944 s, 2.4 MB/s 2024-05-11T08:04:56.594 DEBUG:teuthology.orchestra.run.smithi204:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-11T08:04:56.650 DEBUG:teuthology.orchestra.run.smithi204:> stat /dev/vg_nvme/lv_3 2024-05-11T08:04:56.705 INFO:teuthology.orchestra.run.smithi204.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-11T08:04:56.705 INFO:teuthology.orchestra.run.smithi204.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-11T08:04:56.705 INFO:teuthology.orchestra.run.smithi204.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-05-11T08:04:56.705 INFO:teuthology.orchestra.run.smithi204.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-11T08:04:56.705 INFO:teuthology.orchestra.run.smithi204.stdout:Context: system_u:object_r:device_t:s0 2024-05-11T08:04:56.705 INFO:teuthology.orchestra.run.smithi204.stdout:Access: 2024-05-11 08:04:19.615221677 +0000 2024-05-11T08:04:56.705 INFO:teuthology.orchestra.run.smithi204.stdout:Modify: 2024-05-11 08:02:24.170713132 +0000 2024-05-11T08:04:56.705 INFO:teuthology.orchestra.run.smithi204.stdout:Change: 2024-05-11 08:02:24.170713132 +0000 2024-05-11T08:04:56.705 INFO:teuthology.orchestra.run.smithi204.stdout: Birth: 2024-05-11 08:02:24.170713132 +0000 2024-05-11T08:04:56.706 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-11T08:04:56.767 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records in 2024-05-11T08:04:56.768 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records out 2024-05-11T08:04:56.768 INFO:teuthology.orchestra.run.smithi204.stderr:512 bytes copied, 0.00017119 s, 3.0 MB/s 2024-05-11T08:04:56.769 DEBUG:teuthology.orchestra.run.smithi204:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-11T08:04:56.824 DEBUG:teuthology.orchestra.run.smithi204:> stat /dev/vg_nvme/lv_4 2024-05-11T08:04:56.880 INFO:teuthology.orchestra.run.smithi204.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-11T08:04:56.880 INFO:teuthology.orchestra.run.smithi204.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-11T08:04:56.880 INFO:teuthology.orchestra.run.smithi204.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-05-11T08:04:56.880 INFO:teuthology.orchestra.run.smithi204.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-11T08:04:56.880 INFO:teuthology.orchestra.run.smithi204.stdout:Context: system_u:object_r:device_t:s0 2024-05-11T08:04:56.880 INFO:teuthology.orchestra.run.smithi204.stdout:Access: 2024-05-11 08:04:19.615221677 +0000 2024-05-11T08:04:56.880 INFO:teuthology.orchestra.run.smithi204.stdout:Modify: 2024-05-11 08:02:24.504709044 +0000 2024-05-11T08:04:56.880 INFO:teuthology.orchestra.run.smithi204.stdout:Change: 2024-05-11 08:02:24.504709044 +0000 2024-05-11T08:04:56.880 INFO:teuthology.orchestra.run.smithi204.stdout: Birth: 2024-05-11 08:02:24.504709044 +0000 2024-05-11T08:04:56.881 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-11T08:04:56.943 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records in 2024-05-11T08:04:56.943 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records out 2024-05-11T08:04:56.943 INFO:teuthology.orchestra.run.smithi204.stderr:512 bytes copied, 0.000180311 s, 2.8 MB/s 2024-05-11T08:04:56.944 DEBUG:teuthology.orchestra.run.smithi204:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-11T08:04:57.001 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-11T08:04:57.001 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi167.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi204.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-05-11T08:04:57.001 INFO:tasks.ceph:Generating config... 2024-05-11T08:04:57.002 INFO:tasks.ceph:[client] client mount timeout = 600 2024-05-11T08:04:57.002 INFO:tasks.ceph:[client] debug client = 20 2024-05-11T08:04:57.002 INFO:tasks.ceph:[client] debug ms = 1 2024-05-11T08:04:57.002 INFO:tasks.ceph:[client] pid file = 2024-05-11T08:04:57.002 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-05-11T08:04:57.003 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-05-11T08:04:57.003 INFO:tasks.ceph:[global] ms die on skipped message = False 2024-05-11T08:04:57.003 INFO:tasks.ceph:[mds] debug mds = 20 2024-05-11T08:04:57.003 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-05-11T08:04:57.003 INFO:tasks.ceph:[mds] debug ms = 1 2024-05-11T08:04:57.003 INFO:tasks.ceph:[mds] mds debug frag = True 2024-05-11T08:04:57.003 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-05-11T08:04:57.003 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-05-11T08:04:57.003 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-05-11T08:04:57.003 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-05-11T08:04:57.003 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-05-11T08:04:57.003 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-05-11T08:04:57.003 INFO:tasks.ceph:[mgr] debug client = 20 2024-05-11T08:04:57.003 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-05-11T08:04:57.004 INFO:tasks.ceph:[mgr] debug ms = 1 2024-05-11T08:04:57.004 INFO:tasks.ceph:[mon] debug mon = 20 2024-05-11T08:04:57.004 INFO:tasks.ceph:[mon] debug ms = 1 2024-05-11T08:04:57.004 INFO:tasks.ceph:[mon] debug paxos = 20 2024-05-11T08:04:57.004 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-05-11T08:04:57.004 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-05-11T08:04:57.004 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-05-11T08:04:57.004 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-05-11T08:04:57.004 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-05-11T08:04:57.004 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-05-11T08:04:57.004 INFO:tasks.ceph:[osd] debug ms = 1 2024-05-11T08:04:57.004 INFO:tasks.ceph:[osd] debug osd = 20 2024-05-11T08:04:57.004 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-05-11T08:04:57.004 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-05-11T08:04:57.005 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-05-11T08:04:57.005 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-05-11T08:04:57.005 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-05-11T08:04:57.005 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-05-11T08:04:57.005 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-05-11T08:04:57.005 INFO:tasks.ceph:Setting up mon.a... 2024-05-11T08:04:57.005 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-05-11T08:04:57.056 INFO:teuthology.orchestra.run.smithi167.stdout:creating /etc/ceph/ceph.keyring 2024-05-11T08:04:57.059 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-05-11T08:04:57.104 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-05-11T08:04:57.128 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.167'), ('mon.b', '172.21.15.204'), ('mon.c', '[v2:172.21.15.204:3301,v1:172.21.15.204:6790]')] 2024-05-11T08:04:57.129 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.167,172.21.15.204,[v2:172.21.15.204:3301,v1:172.21.15.204:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-05-11T08:04:57.130 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-05-11T08:04:57.130 DEBUG:teuthology.orchestra.run.smithi167:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-05-11T08:04:57.184 DEBUG:teuthology.orchestra.run.smithi167:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.167 --add b 172.21.15.204 --addv c '[v2:172.21.15.204:3301,v1:172.21.15.204:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-05-11T08:04:57.271 INFO:teuthology.orchestra.run.smithi167.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-05-11T08:04:57.271 INFO:teuthology.orchestra.run.smithi167.stdout:monmaptool: generated fsid 2034778d-8ff1-4823-aa44-ded32b518241 2024-05-11T08:04:57.271 INFO:teuthology.orchestra.run.smithi167.stdout:setting min_mon_release = quincy 2024-05-11T08:04:57.271 INFO:teuthology.orchestra.run.smithi167.stdout:epoch 0 2024-05-11T08:04:57.271 INFO:teuthology.orchestra.run.smithi167.stdout:fsid 2034778d-8ff1-4823-aa44-ded32b518241 2024-05-11T08:04:57.271 INFO:teuthology.orchestra.run.smithi167.stdout:last_changed 2024-05-11T08:04:57.272031+0000 2024-05-11T08:04:57.271 INFO:teuthology.orchestra.run.smithi167.stdout:created 2024-05-11T08:04:57.272031+0000 2024-05-11T08:04:57.271 INFO:teuthology.orchestra.run.smithi167.stdout:min_mon_release 17 (quincy) 2024-05-11T08:04:57.271 INFO:teuthology.orchestra.run.smithi167.stdout:election_strategy: 1 2024-05-11T08:04:57.271 INFO:teuthology.orchestra.run.smithi167.stdout:0: [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] mon.a 2024-05-11T08:04:57.271 INFO:teuthology.orchestra.run.smithi167.stdout:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.b 2024-05-11T08:04:57.271 INFO:teuthology.orchestra.run.smithi167.stdout:2: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.c 2024-05-11T08:04:57.272 INFO:teuthology.orchestra.run.smithi167.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-05-11T08:04:57.273 DEBUG:teuthology.orchestra.run.smithi167:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-05-11T08:04:57.288 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 2034778d-8ff1-4823-aa44-ded32b518241... 2024-05-11T08:04:57.289 DEBUG:teuthology.orchestra.run.smithi167:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-11T08:04:57.330 DEBUG:teuthology.orchestra.run.smithi204:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-11T08:04:57.370 INFO:teuthology.orchestra.run.smithi167.stdout:[global] 2024-05-11T08:04:57.370 INFO:teuthology.orchestra.run.smithi167.stdout: chdir = "" 2024-05-11T08:04:57.371 INFO:teuthology.orchestra.run.smithi167.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-11T08:04:57.371 INFO:teuthology.orchestra.run.smithi167.stdout: auth supported = cephx 2024-05-11T08:04:57.371 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T08:04:57.371 INFO:teuthology.orchestra.run.smithi167.stdout: filestore xattr use omap = true 2024-05-11T08:04:57.371 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T08:04:57.371 INFO:teuthology.orchestra.run.smithi167.stdout: mon clock drift allowed = 1.000 2024-05-11T08:04:57.371 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T08:04:57.371 INFO:teuthology.orchestra.run.smithi167.stdout: osd crush chooseleaf type = 0 2024-05-11T08:04:57.371 INFO:teuthology.orchestra.run.smithi167.stdout: auth debug = true 2024-05-11T08:04:57.371 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T08:04:57.371 INFO:teuthology.orchestra.run.smithi167.stdout: ms die on old message = true 2024-05-11T08:04:57.372 INFO:teuthology.orchestra.run.smithi167.stdout: ms die on bug = true 2024-05-11T08:04:57.372 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T08:04:57.372 INFO:teuthology.orchestra.run.smithi167.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-11T08:04:57.372 INFO:teuthology.orchestra.run.smithi167.stdout: mon pg warn max object skew = 0 2024-05-11T08:04:57.372 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T08:04:57.372 INFO:teuthology.orchestra.run.smithi167.stdout: # disable pg_autoscaler by default for new pools 2024-05-11T08:04:57.372 INFO:teuthology.orchestra.run.smithi167.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-11T08:04:57.372 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T08:04:57.372 INFO:teuthology.orchestra.run.smithi167.stdout: osd pool default size = 2 2024-05-11T08:04:57.372 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T08:04:57.372 INFO:teuthology.orchestra.run.smithi167.stdout: mon osd allow primary affinity = true 2024-05-11T08:04:57.372 INFO:teuthology.orchestra.run.smithi167.stdout: mon osd allow pg remap = true 2024-05-11T08:04:57.372 INFO:teuthology.orchestra.run.smithi167.stdout: mon warn on legacy crush tunables = false 2024-05-11T08:04:57.372 INFO:teuthology.orchestra.run.smithi167.stdout: mon warn on crush straw calc version zero = false 2024-05-11T08:04:57.373 INFO:teuthology.orchestra.run.smithi167.stdout: mon warn on no sortbitwise = false 2024-05-11T08:04:57.373 INFO:teuthology.orchestra.run.smithi167.stdout: mon warn on osd down out interval zero = false 2024-05-11T08:04:57.373 INFO:teuthology.orchestra.run.smithi167.stdout: mon warn on too few osds = false 2024-05-11T08:04:57.373 INFO:teuthology.orchestra.run.smithi167.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-11T08:04:57.373 INFO:teuthology.orchestra.run.smithi167.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-11T08:04:57.373 INFO:teuthology.orchestra.run.smithi167.stdout: mon_allow_pool_size_one = true 2024-05-11T08:04:57.373 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T08:04:57.373 INFO:teuthology.orchestra.run.smithi167.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-11T08:04:57.373 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T08:04:57.373 INFO:teuthology.orchestra.run.smithi167.stdout: osd default data pool replay window = 5 2024-05-11T08:04:57.373 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T08:04:57.373 INFO:teuthology.orchestra.run.smithi167.stdout: mon allow pool delete = true 2024-05-11T08:04:57.374 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T08:04:57.374 INFO:teuthology.orchestra.run.smithi167.stdout: mon cluster log file level = debug 2024-05-11T08:04:57.374 INFO:teuthology.orchestra.run.smithi167.stdout: debug asserts on shutdown = true 2024-05-11T08:04:57.374 INFO:teuthology.orchestra.run.smithi167.stdout: mon health detail to clog = false 2024-05-11T08:04:57.374 INFO:teuthology.orchestra.run.smithi167.stdout: mon host = "172.21.15.167,172.21.15.204,[v2:172.21.15.204:3301,v1:172.21.15.204:6790]" 2024-05-11T08:04:57.374 INFO:teuthology.orchestra.run.smithi167.stdout: ms die on skipped message = False 2024-05-11T08:04:57.374 INFO:teuthology.orchestra.run.smithi167.stdout: fsid = 2034778d-8ff1-4823-aa44-ded32b518241 2024-05-11T08:04:57.374 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T08:04:57.374 INFO:teuthology.orchestra.run.smithi167.stdout:[osd] 2024-05-11T08:04:57.374 INFO:teuthology.orchestra.run.smithi167.stdout: osd journal size = 100 2024-05-11T08:04:57.374 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T08:04:57.375 INFO:teuthology.orchestra.run.smithi167.stdout: osd scrub load threshold = 5.0 2024-05-11T08:04:57.375 INFO:teuthology.orchestra.run.smithi167.stdout: osd scrub max interval = 600 2024-05-11T08:04:57.375 INFO:teuthology.orchestra.run.smithi167.stdout: osd mclock profile = high_recovery_ops 2024-05-11T08:04:57.375 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T08:04:57.375 INFO:teuthology.orchestra.run.smithi167.stdout: osd recover clone overlap = true 2024-05-11T08:04:57.375 INFO:teuthology.orchestra.run.smithi167.stdout: osd recovery max chunk = 1048576 2024-05-11T08:04:57.375 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T08:04:57.375 INFO:teuthology.orchestra.run.smithi167.stdout: osd debug shutdown = true 2024-05-11T08:04:57.375 INFO:teuthology.orchestra.run.smithi167.stdout: osd debug op order = true 2024-05-11T08:04:57.375 INFO:teuthology.orchestra.run.smithi167.stdout: osd debug verify stray on activate = true 2024-05-11T08:04:57.375 INFO:teuthology.orchestra.run.smithi167.stdout: osd debug trim objects = true 2024-05-11T08:04:57.375 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T08:04:57.375 INFO:teuthology.orchestra.run.smithi167.stdout: osd open classes on start = true 2024-05-11T08:04:57.375 INFO:teuthology.orchestra.run.smithi167.stdout: osd debug pg log writeout = true 2024-05-11T08:04:57.375 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T08:04:57.376 INFO:teuthology.orchestra.run.smithi167.stdout: osd deep scrub update digest min age = 30 2024-05-11T08:04:57.376 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T08:04:57.376 INFO:teuthology.orchestra.run.smithi167.stdout: osd map max advance = 10 2024-05-11T08:04:57.376 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T08:04:57.376 INFO:teuthology.orchestra.run.smithi167.stdout: journal zero on create = true 2024-05-11T08:04:57.376 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T08:04:57.376 INFO:teuthology.orchestra.run.smithi167.stdout: filestore ondisk finisher threads = 3 2024-05-11T08:04:57.376 INFO:teuthology.orchestra.run.smithi167.stdout: filestore apply finisher threads = 3 2024-05-11T08:04:57.376 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T08:04:57.376 INFO:teuthology.orchestra.run.smithi167.stdout: bdev debug aio = true 2024-05-11T08:04:57.376 INFO:teuthology.orchestra.run.smithi167.stdout: osd debug misdirected ops = true 2024-05-11T08:04:57.376 INFO:teuthology.orchestra.run.smithi167.stdout: bluestore block size = 96636764160 2024-05-11T08:04:57.377 INFO:teuthology.orchestra.run.smithi167.stdout: bluestore fsck on mount = True 2024-05-11T08:04:57.377 INFO:teuthology.orchestra.run.smithi167.stdout: debug bluefs = 20 2024-05-11T08:04:57.377 INFO:teuthology.orchestra.run.smithi167.stdout: debug bluestore = 20 2024-05-11T08:04:57.377 INFO:teuthology.orchestra.run.smithi167.stdout: debug ms = 1 2024-05-11T08:04:57.377 INFO:teuthology.orchestra.run.smithi167.stdout: debug osd = 20 2024-05-11T08:04:57.377 INFO:teuthology.orchestra.run.smithi167.stdout: debug rocksdb = 10 2024-05-11T08:04:57.377 INFO:teuthology.orchestra.run.smithi167.stdout: mon osd backfillfull_ratio = 0.85 2024-05-11T08:04:57.377 INFO:teuthology.orchestra.run.smithi167.stdout: mon osd full ratio = 0.9 2024-05-11T08:04:57.377 INFO:teuthology.orchestra.run.smithi167.stdout: mon osd nearfull ratio = 0.8 2024-05-11T08:04:57.377 INFO:teuthology.orchestra.run.smithi167.stdout: osd failsafe full ratio = 0.95 2024-05-11T08:04:57.377 INFO:teuthology.orchestra.run.smithi167.stdout: osd objectstore = bluestore 2024-05-11T08:04:57.377 INFO:teuthology.orchestra.run.smithi167.stdout: osd op complaint time = 180 2024-05-11T08:04:57.377 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T08:04:57.378 INFO:teuthology.orchestra.run.smithi167.stdout:[mgr] 2024-05-11T08:04:57.378 INFO:teuthology.orchestra.run.smithi167.stdout: debug ms = 1 2024-05-11T08:04:57.378 INFO:teuthology.orchestra.run.smithi167.stdout: debug mgr = 20 2024-05-11T08:04:57.378 INFO:teuthology.orchestra.run.smithi167.stdout: debug mon = 20 2024-05-11T08:04:57.378 INFO:teuthology.orchestra.run.smithi167.stdout: debug auth = 20 2024-05-11T08:04:57.378 INFO:teuthology.orchestra.run.smithi167.stdout: mon reweight min pgs per osd = 4 2024-05-11T08:04:57.378 INFO:teuthology.orchestra.run.smithi167.stdout: mon reweight min bytes per osd = 10 2024-05-11T08:04:57.378 INFO:teuthology.orchestra.run.smithi167.stdout: mgr/telemetry/nag = false 2024-05-11T08:04:57.378 INFO:teuthology.orchestra.run.smithi167.stdout: debug client = 20 2024-05-11T08:04:57.378 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T08:04:57.378 INFO:teuthology.orchestra.run.smithi167.stdout:[mon] 2024-05-11T08:04:57.378 INFO:teuthology.orchestra.run.smithi167.stdout: debug ms = 1 2024-05-11T08:04:57.379 INFO:teuthology.orchestra.run.smithi167.stdout: debug mon = 20 2024-05-11T08:04:57.379 INFO:teuthology.orchestra.run.smithi167.stdout: debug paxos = 20 2024-05-11T08:04:57.379 INFO:teuthology.orchestra.run.smithi167.stdout: debug auth = 20 2024-05-11T08:04:57.379 INFO:teuthology.orchestra.run.smithi167.stdout: mon data avail warn = 5 2024-05-11T08:04:57.379 INFO:teuthology.orchestra.run.smithi167.stdout: mon mgr mkfs grace = 240 2024-05-11T08:04:57.379 INFO:teuthology.orchestra.run.smithi167.stdout: mon reweight min pgs per osd = 4 2024-05-11T08:04:57.379 INFO:teuthology.orchestra.run.smithi167.stdout: mon osd reporter subtree level = osd 2024-05-11T08:04:57.379 INFO:teuthology.orchestra.run.smithi167.stdout: mon osd prime pg temp = true 2024-05-11T08:04:57.379 INFO:teuthology.orchestra.run.smithi167.stdout: mon reweight min bytes per osd = 10 2024-05-11T08:04:57.379 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T08:04:57.379 INFO:teuthology.orchestra.run.smithi167.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-11T08:04:57.379 INFO:teuthology.orchestra.run.smithi167.stdout: auth mon ticket ttl = 660 # 11m 2024-05-11T08:04:57.380 INFO:teuthology.orchestra.run.smithi167.stdout: auth service ticket ttl = 240 # 4m 2024-05-11T08:04:57.380 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T08:04:57.380 INFO:teuthology.orchestra.run.smithi167.stdout: # don't complain about insecure global_id in the test suite 2024-05-11T08:04:57.380 INFO:teuthology.orchestra.run.smithi167.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-11T08:04:57.380 INFO:teuthology.orchestra.run.smithi167.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-11T08:04:57.380 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T08:04:57.380 INFO:teuthology.orchestra.run.smithi167.stdout: # 1m isn't quite enough 2024-05-11T08:04:57.380 INFO:teuthology.orchestra.run.smithi167.stdout: mon_down_mkfs_grace = 2m 2024-05-11T08:04:57.380 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T08:04:57.380 INFO:teuthology.orchestra.run.smithi167.stdout: mon_warn_on_filestore_osds = false 2024-05-11T08:04:57.380 INFO:teuthology.orchestra.run.smithi167.stdout: mon down mkfs grace = 300 2024-05-11T08:04:57.380 INFO:teuthology.orchestra.run.smithi167.stdout: mon op complaint time = 120 2024-05-11T08:04:57.380 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T08:04:57.380 INFO:teuthology.orchestra.run.smithi167.stdout:[client] 2024-05-11T08:04:57.381 INFO:teuthology.orchestra.run.smithi167.stdout: rgw cache enabled = true 2024-05-11T08:04:57.381 INFO:teuthology.orchestra.run.smithi167.stdout: rgw enable ops log = true 2024-05-11T08:04:57.381 INFO:teuthology.orchestra.run.smithi167.stdout: rgw enable usage log = true 2024-05-11T08:04:57.381 INFO:teuthology.orchestra.run.smithi167.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-11T08:04:57.381 INFO:teuthology.orchestra.run.smithi167.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-11T08:04:57.381 INFO:teuthology.orchestra.run.smithi167.stdout: client mount timeout = 600 2024-05-11T08:04:57.381 INFO:teuthology.orchestra.run.smithi167.stdout: debug client = 20 2024-05-11T08:04:57.381 INFO:teuthology.orchestra.run.smithi167.stdout: debug ms = 1 2024-05-11T08:04:57.381 INFO:teuthology.orchestra.run.smithi167.stdout: pid file = "" 2024-05-11T08:04:57.381 INFO:teuthology.orchestra.run.smithi167.stdout: rados mon op timeout = 900 2024-05-11T08:04:57.381 INFO:teuthology.orchestra.run.smithi167.stdout: rados osd op timeout = 900 2024-05-11T08:04:57.381 INFO:teuthology.orchestra.run.smithi167.stdout:[mon.a] 2024-05-11T08:04:57.382 INFO:teuthology.orchestra.run.smithi167.stdout:[mon.b] 2024-05-11T08:04:57.382 INFO:teuthology.orchestra.run.smithi167.stdout:[mon.c] 2024-05-11T08:04:57.382 INFO:teuthology.orchestra.run.smithi167.stdout:[mds.a] 2024-05-11T08:04:57.382 INFO:teuthology.orchestra.run.smithi167.stdout:[mds.c] 2024-05-11T08:04:57.382 INFO:teuthology.orchestra.run.smithi167.stdout:[mds.b] 2024-05-11T08:04:57.382 INFO:teuthology.orchestra.run.smithi167.stdout:[mds.d] 2024-05-11T08:04:57.382 INFO:teuthology.orchestra.run.smithi167.stdout:[mds] 2024-05-11T08:04:57.382 INFO:teuthology.orchestra.run.smithi167.stdout: debug mds = 20 2024-05-11T08:04:57.382 INFO:teuthology.orchestra.run.smithi167.stdout: debug mds balancer = 20 2024-05-11T08:04:57.382 INFO:teuthology.orchestra.run.smithi167.stdout: debug ms = 1 2024-05-11T08:04:57.382 INFO:teuthology.orchestra.run.smithi167.stdout: mds debug frag = True 2024-05-11T08:04:57.382 INFO:teuthology.orchestra.run.smithi167.stdout: mds debug scatterstat = True 2024-05-11T08:04:57.382 INFO:teuthology.orchestra.run.smithi167.stdout: mds op complaint time = 180 2024-05-11T08:04:57.382 INFO:teuthology.orchestra.run.smithi167.stdout: mds verify scatter = True 2024-05-11T08:04:57.382 INFO:teuthology.orchestra.run.smithi167.stdout: osd op complaint time = 180 2024-05-11T08:04:57.383 INFO:teuthology.orchestra.run.smithi167.stdout: rados mon op timeout = 900 2024-05-11T08:04:57.383 INFO:teuthology.orchestra.run.smithi167.stdout: rados osd op timeout = 900 2024-05-11T08:04:57.383 INFO:teuthology.orchestra.run.smithi204.stdout:[global] 2024-05-11T08:04:57.383 INFO:teuthology.orchestra.run.smithi204.stdout: chdir = "" 2024-05-11T08:04:57.383 INFO:teuthology.orchestra.run.smithi204.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-11T08:04:57.384 INFO:teuthology.orchestra.run.smithi204.stdout: auth supported = cephx 2024-05-11T08:04:57.384 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T08:04:57.384 INFO:teuthology.orchestra.run.smithi204.stdout: filestore xattr use omap = true 2024-05-11T08:04:57.384 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T08:04:57.384 INFO:teuthology.orchestra.run.smithi204.stdout: mon clock drift allowed = 1.000 2024-05-11T08:04:57.384 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T08:04:57.384 INFO:teuthology.orchestra.run.smithi204.stdout: osd crush chooseleaf type = 0 2024-05-11T08:04:57.384 INFO:teuthology.orchestra.run.smithi204.stdout: auth debug = true 2024-05-11T08:04:57.384 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T08:04:57.384 INFO:teuthology.orchestra.run.smithi204.stdout: ms die on old message = true 2024-05-11T08:04:57.384 INFO:teuthology.orchestra.run.smithi204.stdout: ms die on bug = true 2024-05-11T08:04:57.384 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T08:04:57.384 INFO:teuthology.orchestra.run.smithi204.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-11T08:04:57.385 INFO:teuthology.orchestra.run.smithi204.stdout: mon pg warn max object skew = 0 2024-05-11T08:04:57.385 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T08:04:57.385 INFO:teuthology.orchestra.run.smithi204.stdout: # disable pg_autoscaler by default for new pools 2024-05-11T08:04:57.385 INFO:teuthology.orchestra.run.smithi204.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-11T08:04:57.385 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T08:04:57.385 INFO:teuthology.orchestra.run.smithi204.stdout: osd pool default size = 2 2024-05-11T08:04:57.385 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T08:04:57.385 INFO:teuthology.orchestra.run.smithi204.stdout: mon osd allow primary affinity = true 2024-05-11T08:04:57.385 INFO:teuthology.orchestra.run.smithi204.stdout: mon osd allow pg remap = true 2024-05-11T08:04:57.385 INFO:teuthology.orchestra.run.smithi204.stdout: mon warn on legacy crush tunables = false 2024-05-11T08:04:57.385 INFO:teuthology.orchestra.run.smithi204.stdout: mon warn on crush straw calc version zero = false 2024-05-11T08:04:57.385 INFO:teuthology.orchestra.run.smithi204.stdout: mon warn on no sortbitwise = false 2024-05-11T08:04:57.385 INFO:teuthology.orchestra.run.smithi204.stdout: mon warn on osd down out interval zero = false 2024-05-11T08:04:57.386 INFO:teuthology.orchestra.run.smithi204.stdout: mon warn on too few osds = false 2024-05-11T08:04:57.386 INFO:teuthology.orchestra.run.smithi204.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-11T08:04:57.386 INFO:teuthology.orchestra.run.smithi204.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-11T08:04:57.386 INFO:teuthology.orchestra.run.smithi204.stdout: mon_allow_pool_size_one = true 2024-05-11T08:04:57.386 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T08:04:57.386 INFO:teuthology.orchestra.run.smithi204.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-11T08:04:57.386 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T08:04:57.386 INFO:teuthology.orchestra.run.smithi204.stdout: osd default data pool replay window = 5 2024-05-11T08:04:57.386 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T08:04:57.386 INFO:teuthology.orchestra.run.smithi204.stdout: mon allow pool delete = true 2024-05-11T08:04:57.386 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T08:04:57.386 INFO:teuthology.orchestra.run.smithi204.stdout: mon cluster log file level = debug 2024-05-11T08:04:57.386 INFO:teuthology.orchestra.run.smithi204.stdout: debug asserts on shutdown = true 2024-05-11T08:04:57.386 INFO:teuthology.orchestra.run.smithi204.stdout: mon health detail to clog = false 2024-05-11T08:04:57.386 INFO:teuthology.orchestra.run.smithi204.stdout: mon host = "172.21.15.167,172.21.15.204,[v2:172.21.15.204:3301,v1:172.21.15.204:6790]" 2024-05-11T08:04:57.387 INFO:teuthology.orchestra.run.smithi204.stdout: ms die on skipped message = False 2024-05-11T08:04:57.387 INFO:teuthology.orchestra.run.smithi204.stdout: fsid = 2034778d-8ff1-4823-aa44-ded32b518241 2024-05-11T08:04:57.387 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T08:04:57.387 INFO:teuthology.orchestra.run.smithi204.stdout:[osd] 2024-05-11T08:04:57.387 INFO:teuthology.orchestra.run.smithi204.stdout: osd journal size = 100 2024-05-11T08:04:57.387 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T08:04:57.387 INFO:teuthology.orchestra.run.smithi204.stdout: osd scrub load threshold = 5.0 2024-05-11T08:04:57.387 INFO:teuthology.orchestra.run.smithi204.stdout: osd scrub max interval = 600 2024-05-11T08:04:57.387 INFO:teuthology.orchestra.run.smithi204.stdout: osd mclock profile = high_recovery_ops 2024-05-11T08:04:57.387 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T08:04:57.387 INFO:teuthology.orchestra.run.smithi204.stdout: osd recover clone overlap = true 2024-05-11T08:04:57.387 INFO:teuthology.orchestra.run.smithi204.stdout: osd recovery max chunk = 1048576 2024-05-11T08:04:57.387 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T08:04:57.387 INFO:teuthology.orchestra.run.smithi204.stdout: osd debug shutdown = true 2024-05-11T08:04:57.387 INFO:teuthology.orchestra.run.smithi204.stdout: osd debug op order = true 2024-05-11T08:04:57.388 INFO:teuthology.orchestra.run.smithi204.stdout: osd debug verify stray on activate = true 2024-05-11T08:04:57.388 INFO:teuthology.orchestra.run.smithi204.stdout: osd debug trim objects = true 2024-05-11T08:04:57.388 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T08:04:57.388 INFO:teuthology.orchestra.run.smithi204.stdout: osd open classes on start = true 2024-05-11T08:04:57.388 INFO:teuthology.orchestra.run.smithi204.stdout: osd debug pg log writeout = true 2024-05-11T08:04:57.388 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T08:04:57.388 INFO:teuthology.orchestra.run.smithi204.stdout: osd deep scrub update digest min age = 30 2024-05-11T08:04:57.388 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T08:04:57.388 INFO:teuthology.orchestra.run.smithi204.stdout: osd map max advance = 10 2024-05-11T08:04:57.388 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T08:04:57.388 INFO:teuthology.orchestra.run.smithi204.stdout: journal zero on create = true 2024-05-11T08:04:57.388 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T08:04:57.388 INFO:teuthology.orchestra.run.smithi204.stdout: filestore ondisk finisher threads = 3 2024-05-11T08:04:57.388 INFO:teuthology.orchestra.run.smithi204.stdout: filestore apply finisher threads = 3 2024-05-11T08:04:57.388 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T08:04:57.388 INFO:teuthology.orchestra.run.smithi204.stdout: bdev debug aio = true 2024-05-11T08:04:57.389 INFO:teuthology.orchestra.run.smithi204.stdout: osd debug misdirected ops = true 2024-05-11T08:04:57.389 INFO:teuthology.orchestra.run.smithi204.stdout: bluestore block size = 96636764160 2024-05-11T08:04:57.389 INFO:teuthology.orchestra.run.smithi204.stdout: bluestore fsck on mount = True 2024-05-11T08:04:57.389 INFO:teuthology.orchestra.run.smithi204.stdout: debug bluefs = 20 2024-05-11T08:04:57.389 INFO:teuthology.orchestra.run.smithi204.stdout: debug bluestore = 20 2024-05-11T08:04:57.389 INFO:teuthology.orchestra.run.smithi204.stdout: debug ms = 1 2024-05-11T08:04:57.389 INFO:teuthology.orchestra.run.smithi204.stdout: debug osd = 20 2024-05-11T08:04:57.389 INFO:teuthology.orchestra.run.smithi204.stdout: debug rocksdb = 10 2024-05-11T08:04:57.389 INFO:teuthology.orchestra.run.smithi204.stdout: mon osd backfillfull_ratio = 0.85 2024-05-11T08:04:57.389 INFO:teuthology.orchestra.run.smithi204.stdout: mon osd full ratio = 0.9 2024-05-11T08:04:57.389 INFO:teuthology.orchestra.run.smithi204.stdout: mon osd nearfull ratio = 0.8 2024-05-11T08:04:57.389 INFO:teuthology.orchestra.run.smithi204.stdout: osd failsafe full ratio = 0.95 2024-05-11T08:04:57.389 INFO:teuthology.orchestra.run.smithi204.stdout: osd objectstore = bluestore 2024-05-11T08:04:57.389 INFO:teuthology.orchestra.run.smithi204.stdout: osd op complaint time = 180 2024-05-11T08:04:57.389 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T08:04:57.390 INFO:teuthology.orchestra.run.smithi204.stdout:[mgr] 2024-05-11T08:04:57.390 INFO:teuthology.orchestra.run.smithi204.stdout: debug ms = 1 2024-05-11T08:04:57.390 INFO:teuthology.orchestra.run.smithi204.stdout: debug mgr = 20 2024-05-11T08:04:57.390 INFO:teuthology.orchestra.run.smithi204.stdout: debug mon = 20 2024-05-11T08:04:57.390 INFO:teuthology.orchestra.run.smithi204.stdout: debug auth = 20 2024-05-11T08:04:57.390 INFO:teuthology.orchestra.run.smithi204.stdout: mon reweight min pgs per osd = 4 2024-05-11T08:04:57.390 INFO:teuthology.orchestra.run.smithi204.stdout: mon reweight min bytes per osd = 10 2024-05-11T08:04:57.390 INFO:teuthology.orchestra.run.smithi204.stdout: mgr/telemetry/nag = false 2024-05-11T08:04:57.390 INFO:teuthology.orchestra.run.smithi204.stdout: debug client = 20 2024-05-11T08:04:57.390 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T08:04:57.390 INFO:teuthology.orchestra.run.smithi204.stdout:[mon] 2024-05-11T08:04:57.390 INFO:teuthology.orchestra.run.smithi204.stdout: debug ms = 1 2024-05-11T08:04:57.390 INFO:teuthology.orchestra.run.smithi204.stdout: debug mon = 20 2024-05-11T08:04:57.390 INFO:teuthology.orchestra.run.smithi204.stdout: debug paxos = 20 2024-05-11T08:04:57.391 INFO:teuthology.orchestra.run.smithi204.stdout: debug auth = 20 2024-05-11T08:04:57.391 INFO:teuthology.orchestra.run.smithi204.stdout: mon data avail warn = 5 2024-05-11T08:04:57.391 INFO:teuthology.orchestra.run.smithi204.stdout: mon mgr mkfs grace = 240 2024-05-11T08:04:57.391 INFO:teuthology.orchestra.run.smithi204.stdout: mon reweight min pgs per osd = 4 2024-05-11T08:04:57.391 INFO:teuthology.orchestra.run.smithi204.stdout: mon osd reporter subtree level = osd 2024-05-11T08:04:57.391 INFO:teuthology.orchestra.run.smithi204.stdout: mon osd prime pg temp = true 2024-05-11T08:04:57.391 INFO:teuthology.orchestra.run.smithi204.stdout: mon reweight min bytes per osd = 10 2024-05-11T08:04:57.391 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T08:04:57.391 INFO:teuthology.orchestra.run.smithi204.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-11T08:04:57.391 INFO:teuthology.orchestra.run.smithi204.stdout: auth mon ticket ttl = 660 # 11m 2024-05-11T08:04:57.391 INFO:teuthology.orchestra.run.smithi204.stdout: auth service ticket ttl = 240 # 4m 2024-05-11T08:04:57.391 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T08:04:57.391 INFO:teuthology.orchestra.run.smithi204.stdout: # don't complain about insecure global_id in the test suite 2024-05-11T08:04:57.391 INFO:teuthology.orchestra.run.smithi204.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-11T08:04:57.391 INFO:teuthology.orchestra.run.smithi204.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-11T08:04:57.392 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T08:04:57.392 INFO:teuthology.orchestra.run.smithi204.stdout: # 1m isn't quite enough 2024-05-11T08:04:57.392 INFO:teuthology.orchestra.run.smithi204.stdout: mon_down_mkfs_grace = 2m 2024-05-11T08:04:57.392 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T08:04:57.392 INFO:teuthology.orchestra.run.smithi204.stdout: mon_warn_on_filestore_osds = false 2024-05-11T08:04:57.392 INFO:teuthology.orchestra.run.smithi204.stdout: mon down mkfs grace = 300 2024-05-11T08:04:57.392 INFO:teuthology.orchestra.run.smithi204.stdout: mon op complaint time = 120 2024-05-11T08:04:57.392 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-05-11T08:04:57.392 INFO:teuthology.orchestra.run.smithi204.stdout:[client] 2024-05-11T08:04:57.392 INFO:teuthology.orchestra.run.smithi204.stdout: rgw cache enabled = true 2024-05-11T08:04:57.392 INFO:teuthology.orchestra.run.smithi204.stdout: rgw enable ops log = true 2024-05-11T08:04:57.392 INFO:teuthology.orchestra.run.smithi204.stdout: rgw enable usage log = true 2024-05-11T08:04:57.392 INFO:teuthology.orchestra.run.smithi204.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-11T08:04:57.392 INFO:teuthology.orchestra.run.smithi204.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-11T08:04:57.393 INFO:teuthology.orchestra.run.smithi204.stdout: client mount timeout = 600 2024-05-11T08:04:57.393 INFO:teuthology.orchestra.run.smithi204.stdout: debug client = 20 2024-05-11T08:04:57.393 INFO:teuthology.orchestra.run.smithi204.stdout: debug ms = 1 2024-05-11T08:04:57.393 INFO:teuthology.orchestra.run.smithi204.stdout: pid file = "" 2024-05-11T08:04:57.393 INFO:teuthology.orchestra.run.smithi204.stdout: rados mon op timeout = 900 2024-05-11T08:04:57.393 INFO:teuthology.orchestra.run.smithi204.stdout: rados osd op timeout = 900 2024-05-11T08:04:57.393 INFO:teuthology.orchestra.run.smithi204.stdout:[mon.a] 2024-05-11T08:04:57.393 INFO:teuthology.orchestra.run.smithi204.stdout:[mon.b] 2024-05-11T08:04:57.393 INFO:teuthology.orchestra.run.smithi204.stdout:[mon.c] 2024-05-11T08:04:57.393 INFO:teuthology.orchestra.run.smithi204.stdout:[mds.a] 2024-05-11T08:04:57.393 INFO:teuthology.orchestra.run.smithi204.stdout:[mds.c] 2024-05-11T08:04:57.393 INFO:teuthology.orchestra.run.smithi204.stdout:[mds.b] 2024-05-11T08:04:57.393 INFO:teuthology.orchestra.run.smithi204.stdout:[mds.d] 2024-05-11T08:04:57.393 INFO:teuthology.orchestra.run.smithi204.stdout:[mds] 2024-05-11T08:04:57.393 INFO:teuthology.orchestra.run.smithi204.stdout: debug mds = 20 2024-05-11T08:04:57.394 INFO:teuthology.orchestra.run.smithi204.stdout: debug mds balancer = 20 2024-05-11T08:04:57.394 INFO:teuthology.orchestra.run.smithi204.stdout: debug ms = 1 2024-05-11T08:04:57.394 INFO:teuthology.orchestra.run.smithi204.stdout: mds debug frag = True 2024-05-11T08:04:57.394 INFO:teuthology.orchestra.run.smithi204.stdout: mds debug scatterstat = True 2024-05-11T08:04:57.394 INFO:teuthology.orchestra.run.smithi204.stdout: mds op complaint time = 180 2024-05-11T08:04:57.394 INFO:teuthology.orchestra.run.smithi204.stdout: mds verify scatter = True 2024-05-11T08:04:57.394 INFO:teuthology.orchestra.run.smithi204.stdout: osd op complaint time = 180 2024-05-11T08:04:57.394 INFO:teuthology.orchestra.run.smithi204.stdout: rados mon op timeout = 900 2024-05-11T08:04:57.394 INFO:teuthology.orchestra.run.smithi204.stdout: rados osd op timeout = 900 2024-05-11T08:04:57.395 INFO:tasks.ceph:Creating admin key on mon.a... 2024-05-11T08:04:57.396 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-05-11T08:04:57.489 INFO:tasks.ceph:Copying monmap to all nodes... 2024-05-11T08:04:57.490 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-05-11T08:04:57.490 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-05-11T08:04:57.505 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-05-11T08:04:57.506 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-05-11T08:04:57.561 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi167.front.sepia.ceph.com 2024-05-11T08:04:57.561 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-05-11T08:04:57.561 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/etc/ceph/ceph.keyring 2024-05-11T08:04:57.561 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-05-11T08:04:57.635 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-05-11T08:04:57.637 DEBUG:teuthology.orchestra.run.smithi167:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-05-11T08:04:57.691 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi204.front.sepia.ceph.com 2024-05-11T08:04:57.692 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-11T08:04:57.692 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/etc/ceph/ceph.keyring 2024-05-11T08:04:57.692 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-05-11T08:04:57.727 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-11T08:04:57.727 DEBUG:teuthology.orchestra.run.smithi204:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-05-11T08:04:57.782 INFO:tasks.ceph:Setting up mon nodes... 2024-05-11T08:04:57.782 INFO:tasks.ceph:Setting up mgr nodes... 2024-05-11T08:04:57.782 DEBUG:teuthology.orchestra.run.smithi167:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-05-11T08:04:57.833 INFO:teuthology.orchestra.run.smithi167.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-05-11T08:04:57.836 DEBUG:teuthology.orchestra.run.smithi204:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-05-11T08:04:57.895 INFO:teuthology.orchestra.run.smithi204.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-05-11T08:04:57.898 INFO:tasks.ceph:Setting up mds nodes... 2024-05-11T08:04:57.898 DEBUG:teuthology.orchestra.run.smithi167:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-05-11T08:04:57.949 INFO:teuthology.orchestra.run.smithi167.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-05-11T08:04:57.952 DEBUG:teuthology.orchestra.run.smithi167:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-05-11T08:04:57.977 DEBUG:teuthology.orchestra.run.smithi167:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-05-11T08:04:58.068 INFO:teuthology.orchestra.run.smithi167.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-05-11T08:04:58.071 DEBUG:teuthology.orchestra.run.smithi167:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-05-11T08:04:58.095 DEBUG:teuthology.orchestra.run.smithi204:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-05-11T08:04:58.146 INFO:teuthology.orchestra.run.smithi204.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-05-11T08:04:58.149 DEBUG:teuthology.orchestra.run.smithi204:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-05-11T08:04:58.174 DEBUG:teuthology.orchestra.run.smithi204:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-05-11T08:04:58.265 INFO:teuthology.orchestra.run.smithi204.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-05-11T08:04:58.268 DEBUG:teuthology.orchestra.run.smithi204:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-05-11T08:04:58.293 INFO:tasks.ceph_client:Setting up client nodes... 2024-05-11T08:04:58.293 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-05-11T08:04:58.336 INFO:teuthology.orchestra.run.smithi167.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-05-11T08:04:58.348 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-05-11T08:04:58.429 INFO:teuthology.orchestra.run.smithi167.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-05-11T08:04:58.442 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-05-11T08:04:58.483 INFO:teuthology.orchestra.run.smithi204.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-05-11T08:04:58.496 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-05-11T08:04:58.576 INFO:teuthology.orchestra.run.smithi204.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-05-11T08:04:58.588 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-05-11T08:04:58.589 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi167.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi204.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-05-11T08:04:58.589 DEBUG:teuthology.orchestra.run.smithi167:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-05-11T08:04:58.613 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-11T08:04:58.613 INFO:tasks.ceph:role: osd.0 2024-05-11T08:04:58.613 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi167.front.sepia.ceph.com 2024-05-11T08:04:58.613 DEBUG:teuthology.orchestra.run.smithi167:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-05-11T08:04:58.686 INFO:teuthology.orchestra.run.smithi167.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-05-11T08:04:58.686 INFO:teuthology.orchestra.run.smithi167.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-11T08:04:58.686 INFO:teuthology.orchestra.run.smithi167.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-11T08:04:58.686 INFO:teuthology.orchestra.run.smithi167.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-11T08:04:58.686 INFO:teuthology.orchestra.run.smithi167.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-11T08:04:58.686 INFO:teuthology.orchestra.run.smithi167.stdout: = sunit=0 swidth=0 blks 2024-05-11T08:04:58.686 INFO:teuthology.orchestra.run.smithi167.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-11T08:04:58.686 INFO:teuthology.orchestra.run.smithi167.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-11T08:04:58.686 INFO:teuthology.orchestra.run.smithi167.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-11T08:04:58.686 INFO:teuthology.orchestra.run.smithi167.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-11T08:04:59.207 INFO:teuthology.orchestra.run.smithi167.stdout:Discarding blocks...Done. 2024-05-11T08:04:59.209 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi167.front.sepia.ceph.com -o noatime 2024-05-11T08:04:59.209 DEBUG:teuthology.orchestra.run.smithi167:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-05-11T08:04:59.353 DEBUG:teuthology.orchestra.run.smithi167:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-05-11T08:04:59.385 DEBUG:teuthology.orchestra.run.smithi167:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-05-11T08:04:59.452 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-11T08:04:59.452 INFO:tasks.ceph:role: osd.1 2024-05-11T08:04:59.452 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi167.front.sepia.ceph.com 2024-05-11T08:04:59.452 DEBUG:teuthology.orchestra.run.smithi167:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-05-11T08:04:59.524 INFO:teuthology.orchestra.run.smithi167.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-05-11T08:04:59.524 INFO:teuthology.orchestra.run.smithi167.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-11T08:04:59.524 INFO:teuthology.orchestra.run.smithi167.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-11T08:04:59.524 INFO:teuthology.orchestra.run.smithi167.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-11T08:04:59.524 INFO:teuthology.orchestra.run.smithi167.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-11T08:04:59.525 INFO:teuthology.orchestra.run.smithi167.stdout: = sunit=0 swidth=0 blks 2024-05-11T08:04:59.525 INFO:teuthology.orchestra.run.smithi167.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-11T08:04:59.525 INFO:teuthology.orchestra.run.smithi167.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-11T08:04:59.525 INFO:teuthology.orchestra.run.smithi167.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-11T08:04:59.525 INFO:teuthology.orchestra.run.smithi167.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-11T08:04:59.973 INFO:teuthology.orchestra.run.smithi167.stdout:Discarding blocks...Done. 2024-05-11T08:04:59.975 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi167.front.sepia.ceph.com -o noatime 2024-05-11T08:04:59.975 DEBUG:teuthology.orchestra.run.smithi167:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-05-11T08:05:00.071 DEBUG:teuthology.orchestra.run.smithi167:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-05-11T08:05:00.101 DEBUG:teuthology.orchestra.run.smithi167:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-05-11T08:05:00.167 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-11T08:05:00.167 INFO:tasks.ceph:role: osd.2 2024-05-11T08:05:00.167 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi167.front.sepia.ceph.com 2024-05-11T08:05:00.167 DEBUG:teuthology.orchestra.run.smithi167:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-05-11T08:05:00.239 INFO:teuthology.orchestra.run.smithi167.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-05-11T08:05:00.239 INFO:teuthology.orchestra.run.smithi167.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-11T08:05:00.240 INFO:teuthology.orchestra.run.smithi167.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-11T08:05:00.240 INFO:teuthology.orchestra.run.smithi167.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-11T08:05:00.240 INFO:teuthology.orchestra.run.smithi167.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-11T08:05:00.240 INFO:teuthology.orchestra.run.smithi167.stdout: = sunit=0 swidth=0 blks 2024-05-11T08:05:00.240 INFO:teuthology.orchestra.run.smithi167.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-11T08:05:00.240 INFO:teuthology.orchestra.run.smithi167.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-11T08:05:00.240 INFO:teuthology.orchestra.run.smithi167.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-11T08:05:00.240 INFO:teuthology.orchestra.run.smithi167.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-11T08:05:00.734 INFO:teuthology.orchestra.run.smithi167.stdout:Discarding blocks...Done. 2024-05-11T08:05:00.736 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi167.front.sepia.ceph.com -o noatime 2024-05-11T08:05:00.736 DEBUG:teuthology.orchestra.run.smithi167:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-05-11T08:05:00.890 DEBUG:teuthology.orchestra.run.smithi167:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-05-11T08:05:00.921 DEBUG:teuthology.orchestra.run.smithi167:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-05-11T08:05:00.987 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-11T08:05:00.987 INFO:tasks.ceph:role: osd.3 2024-05-11T08:05:00.987 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi167.front.sepia.ceph.com 2024-05-11T08:05:00.987 DEBUG:teuthology.orchestra.run.smithi167:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-05-11T08:05:01.059 INFO:teuthology.orchestra.run.smithi167.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-05-11T08:05:01.060 INFO:teuthology.orchestra.run.smithi167.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-11T08:05:01.060 INFO:teuthology.orchestra.run.smithi167.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-11T08:05:01.060 INFO:teuthology.orchestra.run.smithi167.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-11T08:05:01.060 INFO:teuthology.orchestra.run.smithi167.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-11T08:05:01.060 INFO:teuthology.orchestra.run.smithi167.stdout: = sunit=0 swidth=0 blks 2024-05-11T08:05:01.060 INFO:teuthology.orchestra.run.smithi167.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-11T08:05:01.060 INFO:teuthology.orchestra.run.smithi167.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-11T08:05:01.060 INFO:teuthology.orchestra.run.smithi167.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-11T08:05:01.060 INFO:teuthology.orchestra.run.smithi167.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-11T08:05:01.565 INFO:teuthology.orchestra.run.smithi167.stdout:Discarding blocks...Done. 2024-05-11T08:05:01.567 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi167.front.sepia.ceph.com -o noatime 2024-05-11T08:05:01.567 DEBUG:teuthology.orchestra.run.smithi167:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-05-11T08:05:01.717 DEBUG:teuthology.orchestra.run.smithi167:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-05-11T08:05:01.753 DEBUG:teuthology.orchestra.run.smithi167:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-11T08:05:01.851 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:01.851+0000 7fe0758e5640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-05-11T08:05:01.852 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:01.851+0000 7fe0758e5640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-05-11T08:05:01.852 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:01.851+0000 7fe0758e5640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-05-11T08:05:01.852 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:01.851+0000 7fe0758e5640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-05-11T08:05:01.852 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:01.851+0000 7fe0758e5640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-05-11T08:05:01.852 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:01.852+0000 7fe0758e5640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-05-11T08:05:04.700 DEBUG:teuthology.orchestra.run.smithi167:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-05-11T08:05:04.726 DEBUG:teuthology.orchestra.run.smithi167:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-11T08:05:04.813 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:04.812+0000 7f851de11640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-05-11T08:05:04.813 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:04.813+0000 7f851de11640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-05-11T08:05:04.813 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:04.813+0000 7f851de11640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-05-11T08:05:04.813 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:04.813+0000 7f851de11640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-05-11T08:05:04.813 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:04.813+0000 7f851de11640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-05-11T08:05:04.813 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:04.813+0000 7f851de11640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-05-11T08:05:07.657 DEBUG:teuthology.orchestra.run.smithi167:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-05-11T08:05:07.687 DEBUG:teuthology.orchestra.run.smithi167:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-11T08:05:07.777 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:07.777+0000 7f503480f640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-05-11T08:05:07.777 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:07.777+0000 7f503480f640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-05-11T08:05:07.777 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:07.777+0000 7f503480f640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-05-11T08:05:07.777 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:07.777+0000 7f503480f640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-05-11T08:05:07.777 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:07.777+0000 7f503480f640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-05-11T08:05:07.778 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:07.778+0000 7f503480f640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-05-11T08:05:10.631 DEBUG:teuthology.orchestra.run.smithi167:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-05-11T08:05:10.659 DEBUG:teuthology.orchestra.run.smithi167:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-11T08:05:10.748 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:10.747+0000 7fa467c19640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-05-11T08:05:10.748 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:10.748+0000 7fa467c19640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-05-11T08:05:10.748 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:10.748+0000 7fa467c19640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-05-11T08:05:10.748 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:10.748+0000 7fa467c19640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-05-11T08:05:10.748 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:10.748+0000 7fa467c19640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-05-11T08:05:10.748 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:10.748+0000 7fa467c19640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-05-11T08:05:13.601 DEBUG:teuthology.orchestra.run.smithi167:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-05-11T08:05:13.631 DEBUG:teuthology.orchestra.run.smithi204:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-05-11T08:05:13.662 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-11T08:05:13.662 INFO:tasks.ceph:role: osd.4 2024-05-11T08:05:13.662 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi204.front.sepia.ceph.com 2024-05-11T08:05:13.662 DEBUG:teuthology.orchestra.run.smithi204:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-05-11T08:05:13.739 INFO:teuthology.orchestra.run.smithi204.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-05-11T08:05:13.739 INFO:teuthology.orchestra.run.smithi204.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-11T08:05:13.739 INFO:teuthology.orchestra.run.smithi204.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-11T08:05:13.739 INFO:teuthology.orchestra.run.smithi204.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-11T08:05:13.739 INFO:teuthology.orchestra.run.smithi204.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-11T08:05:13.739 INFO:teuthology.orchestra.run.smithi204.stdout: = sunit=0 swidth=0 blks 2024-05-11T08:05:13.740 INFO:teuthology.orchestra.run.smithi204.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-11T08:05:13.740 INFO:teuthology.orchestra.run.smithi204.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-11T08:05:13.740 INFO:teuthology.orchestra.run.smithi204.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-11T08:05:13.740 INFO:teuthology.orchestra.run.smithi204.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-11T08:05:14.245 INFO:teuthology.orchestra.run.smithi204.stdout:Discarding blocks...Done. 2024-05-11T08:05:14.247 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi204.front.sepia.ceph.com -o noatime 2024-05-11T08:05:14.248 DEBUG:teuthology.orchestra.run.smithi204:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-05-11T08:05:14.397 DEBUG:teuthology.orchestra.run.smithi204:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-05-11T08:05:14.431 DEBUG:teuthology.orchestra.run.smithi204:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-05-11T08:05:14.497 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-11T08:05:14.497 INFO:tasks.ceph:role: osd.5 2024-05-11T08:05:14.497 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi204.front.sepia.ceph.com 2024-05-11T08:05:14.498 DEBUG:teuthology.orchestra.run.smithi204:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-05-11T08:05:14.571 INFO:teuthology.orchestra.run.smithi204.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-05-11T08:05:14.571 INFO:teuthology.orchestra.run.smithi204.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-11T08:05:14.572 INFO:teuthology.orchestra.run.smithi204.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-11T08:05:14.572 INFO:teuthology.orchestra.run.smithi204.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-11T08:05:14.572 INFO:teuthology.orchestra.run.smithi204.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-11T08:05:14.572 INFO:teuthology.orchestra.run.smithi204.stdout: = sunit=0 swidth=0 blks 2024-05-11T08:05:14.572 INFO:teuthology.orchestra.run.smithi204.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-11T08:05:14.572 INFO:teuthology.orchestra.run.smithi204.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-11T08:05:14.572 INFO:teuthology.orchestra.run.smithi204.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-11T08:05:14.572 INFO:teuthology.orchestra.run.smithi204.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-11T08:05:15.023 INFO:teuthology.orchestra.run.smithi204.stdout:Discarding blocks...Done. 2024-05-11T08:05:15.026 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi204.front.sepia.ceph.com -o noatime 2024-05-11T08:05:15.026 DEBUG:teuthology.orchestra.run.smithi204:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-05-11T08:05:15.119 DEBUG:teuthology.orchestra.run.smithi204:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-05-11T08:05:15.151 DEBUG:teuthology.orchestra.run.smithi204:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-05-11T08:05:15.216 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-11T08:05:15.216 INFO:tasks.ceph:role: osd.6 2024-05-11T08:05:15.217 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi204.front.sepia.ceph.com 2024-05-11T08:05:15.217 DEBUG:teuthology.orchestra.run.smithi204:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-05-11T08:05:15.289 INFO:teuthology.orchestra.run.smithi204.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-05-11T08:05:15.289 INFO:teuthology.orchestra.run.smithi204.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-11T08:05:15.290 INFO:teuthology.orchestra.run.smithi204.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-11T08:05:15.290 INFO:teuthology.orchestra.run.smithi204.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-11T08:05:15.290 INFO:teuthology.orchestra.run.smithi204.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-11T08:05:15.290 INFO:teuthology.orchestra.run.smithi204.stdout: = sunit=0 swidth=0 blks 2024-05-11T08:05:15.290 INFO:teuthology.orchestra.run.smithi204.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-11T08:05:15.290 INFO:teuthology.orchestra.run.smithi204.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-11T08:05:15.290 INFO:teuthology.orchestra.run.smithi204.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-11T08:05:15.290 INFO:teuthology.orchestra.run.smithi204.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-11T08:05:15.766 INFO:teuthology.orchestra.run.smithi204.stdout:Discarding blocks...Done. 2024-05-11T08:05:15.768 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi204.front.sepia.ceph.com -o noatime 2024-05-11T08:05:15.768 DEBUG:teuthology.orchestra.run.smithi204:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-05-11T08:05:15.906 DEBUG:teuthology.orchestra.run.smithi204:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-05-11T08:05:15.935 DEBUG:teuthology.orchestra.run.smithi204:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-05-11T08:05:16.000 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-11T08:05:16.000 INFO:tasks.ceph:role: osd.7 2024-05-11T08:05:16.000 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi204.front.sepia.ceph.com 2024-05-11T08:05:16.001 DEBUG:teuthology.orchestra.run.smithi204:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-05-11T08:05:16.073 INFO:teuthology.orchestra.run.smithi204.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-05-11T08:05:16.073 INFO:teuthology.orchestra.run.smithi204.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-11T08:05:16.073 INFO:teuthology.orchestra.run.smithi204.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-11T08:05:16.073 INFO:teuthology.orchestra.run.smithi204.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-11T08:05:16.073 INFO:teuthology.orchestra.run.smithi204.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-11T08:05:16.073 INFO:teuthology.orchestra.run.smithi204.stdout: = sunit=0 swidth=0 blks 2024-05-11T08:05:16.073 INFO:teuthology.orchestra.run.smithi204.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-11T08:05:16.073 INFO:teuthology.orchestra.run.smithi204.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-11T08:05:16.073 INFO:teuthology.orchestra.run.smithi204.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-11T08:05:16.073 INFO:teuthology.orchestra.run.smithi204.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-11T08:05:16.605 INFO:teuthology.orchestra.run.smithi204.stdout:Discarding blocks...Done. 2024-05-11T08:05:16.607 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi204.front.sepia.ceph.com -o noatime 2024-05-11T08:05:16.607 DEBUG:teuthology.orchestra.run.smithi204:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-05-11T08:05:16.744 DEBUG:teuthology.orchestra.run.smithi204:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-05-11T08:05:16.777 DEBUG:teuthology.orchestra.run.smithi204:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-11T08:05:16.868 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:16.868+0000 7f468d0df640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-05-11T08:05:16.868 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:16.869+0000 7f468d0df640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-05-11T08:05:16.869 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:16.869+0000 7f468d0df640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-05-11T08:05:16.869 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:16.869+0000 7f468d0df640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-05-11T08:05:16.869 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:16.869+0000 7f468d0df640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-05-11T08:05:16.869 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:16.869+0000 7f468d0df640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-05-11T08:05:19.722 DEBUG:teuthology.orchestra.run.smithi204:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-05-11T08:05:19.751 DEBUG:teuthology.orchestra.run.smithi204:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-11T08:05:19.842 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:19.842+0000 7f9bfea11640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-05-11T08:05:19.842 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:19.842+0000 7f9bfea11640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-05-11T08:05:19.842 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:19.842+0000 7f9bfea11640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-05-11T08:05:19.842 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:19.842+0000 7f9bfea11640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-05-11T08:05:19.842 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:19.842+0000 7f9bfea11640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-05-11T08:05:19.843 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:19.843+0000 7f9bfea11640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-05-11T08:05:22.699 DEBUG:teuthology.orchestra.run.smithi204:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-05-11T08:05:22.729 DEBUG:teuthology.orchestra.run.smithi204:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-11T08:05:22.821 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:22.821+0000 7f85a9fc4640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-05-11T08:05:22.821 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:22.821+0000 7f85a9fc4640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-05-11T08:05:22.821 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:22.821+0000 7f85a9fc4640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-05-11T08:05:22.822 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:22.821+0000 7f85a9fc4640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-05-11T08:05:22.822 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:22.821+0000 7f85a9fc4640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-05-11T08:05:22.822 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:22.822+0000 7f85a9fc4640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-05-11T08:05:25.434 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-11T08:05:25.437 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-11T08:05:25.673 DEBUG:teuthology.orchestra.run.smithi204:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-05-11T08:05:25.703 DEBUG:teuthology.orchestra.run.smithi204:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-11T08:05:25.793 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:25.793+0000 7f3d8e5d6640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-05-11T08:05:25.793 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:25.793+0000 7f3d8e5d6640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-05-11T08:05:25.794 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:25.793+0000 7f3d8e5d6640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-05-11T08:05:25.794 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:25.793+0000 7f3d8e5d6640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-05-11T08:05:25.794 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:25.793+0000 7f3d8e5d6640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-05-11T08:05:25.794 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:25.794+0000 7f3d8e5d6640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-05-11T08:05:28.644 DEBUG:teuthology.orchestra.run.smithi204:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-05-11T08:05:28.673 INFO:tasks.ceph:Reading keys from all nodes... 2024-05-11T08:05:28.673 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-05-11T08:05:28.673 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-05-11T08:05:28.704 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-05-11T08:05:28.704 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-05-11T08:05:28.775 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-05-11T08:05:28.775 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-05-11T08:05:28.844 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-05-11T08:05:28.844 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-05-11T08:05:28.912 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-05-11T08:05:28.913 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-05-11T08:05:28.979 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-05-11T08:05:28.980 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-05-11T08:05:29.045 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-05-11T08:05:29.045 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-05-11T08:05:29.114 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-11T08:05:29.114 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-05-11T08:05:29.145 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-11T08:05:29.145 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-05-11T08:05:29.214 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-11T08:05:29.214 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-05-11T08:05:29.283 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-11T08:05:29.283 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-05-11T08:05:29.349 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-11T08:05:29.350 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-05-11T08:05:29.417 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-11T08:05:29.417 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-05-11T08:05:29.482 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-11T08:05:29.482 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-05-11T08:05:29.548 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-05-11T08:05:29.548 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-05-11T08:05:29.565 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-05-11T08:05:29.565 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-05-11T08:05:29.622 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-11T08:05:29.622 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-05-11T08:05:29.638 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-11T08:05:29.638 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-05-11T08:05:29.695 INFO:tasks.ceph:Adding keys to all mons... 2024-05-11T08:05:29.695 DEBUG:teuthology.orchestra.run.smithi167:> sudo tee -a /etc/ceph/ceph.keyring 2024-05-11T08:05:29.697 DEBUG:teuthology.orchestra.run.smithi204:> sudo tee -a /etc/ceph/ceph.keyring 2024-05-11T08:05:29.737 INFO:teuthology.orchestra.run.smithi167.stdout:[mgr.y] 2024-05-11T08:05:29.738 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQCpJj9mHQi/MRAAa1ks1sdtI2VqeUQElx7gyw== 2024-05-11T08:05:29.738 INFO:teuthology.orchestra.run.smithi167.stdout:[mds.a] 2024-05-11T08:05:29.738 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQCpJj9mdAurOBAAHcD6a4z7xCorLPtMvKt7+g== 2024-05-11T08:05:29.738 INFO:teuthology.orchestra.run.smithi167.stdout:[mds.c] 2024-05-11T08:05:29.738 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQCqJj9mWHUoBBAATca00N8k4vAPQSLss3QaKQ== 2024-05-11T08:05:29.738 INFO:teuthology.orchestra.run.smithi167.stdout:[osd.0] 2024-05-11T08:05:29.738 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQCtJj9mPzPPMhAADIgOJxI1YIZwLPuF04h4Iw== 2024-05-11T08:05:29.738 INFO:teuthology.orchestra.run.smithi167.stdout:[osd.1] 2024-05-11T08:05:29.738 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQCwJj9msz+BMBAAgvNo97yu9vWqVMi6TLFWZA== 2024-05-11T08:05:29.738 INFO:teuthology.orchestra.run.smithi167.stdout:[osd.2] 2024-05-11T08:05:29.738 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQCzJj9mDa5gLhAAdzaIlR9nyF6mYHi/xxXxqA== 2024-05-11T08:05:29.739 INFO:teuthology.orchestra.run.smithi167.stdout:[osd.3] 2024-05-11T08:05:29.739 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQC2Jj9mHlmiLBAAu28f59Qm6L7ZozDdjcHpEA== 2024-05-11T08:05:29.739 INFO:teuthology.orchestra.run.smithi167.stdout:[mgr.x] 2024-05-11T08:05:29.739 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQCpJj9mqi54NRAAd33DWaNHSGFFWs57F2Ft7Q== 2024-05-11T08:05:29.739 INFO:teuthology.orchestra.run.smithi167.stdout:[mds.b] 2024-05-11T08:05:29.739 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQCqJj9mCXXOCBAAWLxzC/EaKyn/AP/ScQ6arA== 2024-05-11T08:05:29.739 INFO:teuthology.orchestra.run.smithi167.stdout:[mds.d] 2024-05-11T08:05:29.739 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQCqJj9m7N3qDxAAGgpj+5XiYNtiwB0to3JsHQ== 2024-05-11T08:05:29.739 INFO:teuthology.orchestra.run.smithi167.stdout:[osd.4] 2024-05-11T08:05:29.739 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQC8Jj9mOdjVMxAAoEhXYMeNeoqPqwVB8nHTpQ== 2024-05-11T08:05:29.739 INFO:teuthology.orchestra.run.smithi167.stdout:[osd.5] 2024-05-11T08:05:29.739 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQC/Jj9mEzhAMhAAjSzkBk4ucVR1bGhT1BAE4w== 2024-05-11T08:05:29.739 INFO:teuthology.orchestra.run.smithi167.stdout:[osd.6] 2024-05-11T08:05:29.739 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQDCJj9mWSICMRAACdekwLQOZxUVv9YUTvCDwg== 2024-05-11T08:05:29.740 INFO:teuthology.orchestra.run.smithi167.stdout:[osd.7] 2024-05-11T08:05:29.740 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQDFJj9mNNBZLxAAddJmZ9qfi77p4Uh/x7NA8g== 2024-05-11T08:05:29.740 INFO:teuthology.orchestra.run.smithi167.stdout:[client.0] 2024-05-11T08:05:29.740 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQCqJj9m/0ccFBAAu878v2GdClfu/qvQ/KOnvQ== 2024-05-11T08:05:29.740 INFO:teuthology.orchestra.run.smithi167.stdout:[client.2] 2024-05-11T08:05:29.740 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQCqJj9mnwmyGRAA7lQ7Y8GJ5xnNwt+SF4D2LA== 2024-05-11T08:05:29.740 INFO:teuthology.orchestra.run.smithi167.stdout:[client.1] 2024-05-11T08:05:29.740 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQCqJj9mWa3dHBAA4lrVGY9iSXZFb9WoZLVOVA== 2024-05-11T08:05:29.740 INFO:teuthology.orchestra.run.smithi167.stdout:[client.3] 2024-05-11T08:05:29.740 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQCqJj9mof5uIhAAsL1gMVkOcN9CMyrQc6brWg== 2024-05-11T08:05:29.758 INFO:teuthology.orchestra.run.smithi204.stdout:[mgr.y] 2024-05-11T08:05:29.758 INFO:teuthology.orchestra.run.smithi204.stdout: key = AQCpJj9mHQi/MRAAa1ks1sdtI2VqeUQElx7gyw== 2024-05-11T08:05:29.758 INFO:teuthology.orchestra.run.smithi204.stdout:[mds.a] 2024-05-11T08:05:29.758 INFO:teuthology.orchestra.run.smithi204.stdout: key = AQCpJj9mdAurOBAAHcD6a4z7xCorLPtMvKt7+g== 2024-05-11T08:05:29.758 INFO:teuthology.orchestra.run.smithi204.stdout:[mds.c] 2024-05-11T08:05:29.758 INFO:teuthology.orchestra.run.smithi204.stdout: key = AQCqJj9mWHUoBBAATca00N8k4vAPQSLss3QaKQ== 2024-05-11T08:05:29.758 INFO:teuthology.orchestra.run.smithi204.stdout:[osd.0] 2024-05-11T08:05:29.758 INFO:teuthology.orchestra.run.smithi204.stdout: key = AQCtJj9mPzPPMhAADIgOJxI1YIZwLPuF04h4Iw== 2024-05-11T08:05:29.758 INFO:teuthology.orchestra.run.smithi204.stdout:[osd.1] 2024-05-11T08:05:29.758 INFO:teuthology.orchestra.run.smithi204.stdout: key = AQCwJj9msz+BMBAAgvNo97yu9vWqVMi6TLFWZA== 2024-05-11T08:05:29.758 INFO:teuthology.orchestra.run.smithi204.stdout:[osd.2] 2024-05-11T08:05:29.759 INFO:teuthology.orchestra.run.smithi204.stdout: key = AQCzJj9mDa5gLhAAdzaIlR9nyF6mYHi/xxXxqA== 2024-05-11T08:05:29.759 INFO:teuthology.orchestra.run.smithi204.stdout:[osd.3] 2024-05-11T08:05:29.759 INFO:teuthology.orchestra.run.smithi204.stdout: key = AQC2Jj9mHlmiLBAAu28f59Qm6L7ZozDdjcHpEA== 2024-05-11T08:05:29.759 INFO:teuthology.orchestra.run.smithi204.stdout:[mgr.x] 2024-05-11T08:05:29.759 INFO:teuthology.orchestra.run.smithi204.stdout: key = AQCpJj9mqi54NRAAd33DWaNHSGFFWs57F2Ft7Q== 2024-05-11T08:05:29.759 INFO:teuthology.orchestra.run.smithi204.stdout:[mds.b] 2024-05-11T08:05:29.759 INFO:teuthology.orchestra.run.smithi204.stdout: key = AQCqJj9mCXXOCBAAWLxzC/EaKyn/AP/ScQ6arA== 2024-05-11T08:05:29.759 INFO:teuthology.orchestra.run.smithi204.stdout:[mds.d] 2024-05-11T08:05:29.759 INFO:teuthology.orchestra.run.smithi204.stdout: key = AQCqJj9m7N3qDxAAGgpj+5XiYNtiwB0to3JsHQ== 2024-05-11T08:05:29.759 INFO:teuthology.orchestra.run.smithi204.stdout:[osd.4] 2024-05-11T08:05:29.759 INFO:teuthology.orchestra.run.smithi204.stdout: key = AQC8Jj9mOdjVMxAAoEhXYMeNeoqPqwVB8nHTpQ== 2024-05-11T08:05:29.759 INFO:teuthology.orchestra.run.smithi204.stdout:[osd.5] 2024-05-11T08:05:29.759 INFO:teuthology.orchestra.run.smithi204.stdout: key = AQC/Jj9mEzhAMhAAjSzkBk4ucVR1bGhT1BAE4w== 2024-05-11T08:05:29.759 INFO:teuthology.orchestra.run.smithi204.stdout:[osd.6] 2024-05-11T08:05:29.759 INFO:teuthology.orchestra.run.smithi204.stdout: key = AQDCJj9mWSICMRAACdekwLQOZxUVv9YUTvCDwg== 2024-05-11T08:05:29.760 INFO:teuthology.orchestra.run.smithi204.stdout:[osd.7] 2024-05-11T08:05:29.760 INFO:teuthology.orchestra.run.smithi204.stdout: key = AQDFJj9mNNBZLxAAddJmZ9qfi77p4Uh/x7NA8g== 2024-05-11T08:05:29.760 INFO:teuthology.orchestra.run.smithi204.stdout:[client.0] 2024-05-11T08:05:29.760 INFO:teuthology.orchestra.run.smithi204.stdout: key = AQCqJj9m/0ccFBAAu878v2GdClfu/qvQ/KOnvQ== 2024-05-11T08:05:29.760 INFO:teuthology.orchestra.run.smithi204.stdout:[client.2] 2024-05-11T08:05:29.760 INFO:teuthology.orchestra.run.smithi204.stdout: key = AQCqJj9mnwmyGRAA7lQ7Y8GJ5xnNwt+SF4D2LA== 2024-05-11T08:05:29.760 INFO:teuthology.orchestra.run.smithi204.stdout:[client.1] 2024-05-11T08:05:29.760 INFO:teuthology.orchestra.run.smithi204.stdout: key = AQCqJj9mWa3dHBAA4lrVGY9iSXZFb9WoZLVOVA== 2024-05-11T08:05:29.760 INFO:teuthology.orchestra.run.smithi204.stdout:[client.3] 2024-05-11T08:05:29.760 INFO:teuthology.orchestra.run.smithi204.stdout: key = AQCqJj9mof5uIhAAsL1gMVkOcN9CMyrQc6brWg== 2024-05-11T08:05:29.761 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-11T08:05:29.784 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-11T08:05:29.853 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-11T08:05:29.856 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-11T08:05:29.909 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-11T08:05:29.911 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-11T08:05:29.965 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-11T08:05:29.968 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-11T08:05:30.019 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-11T08:05:30.021 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-11T08:05:30.072 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-11T08:05:30.075 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-11T08:05:30.125 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-11T08:05:30.127 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-11T08:05:30.178 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-11T08:05:30.181 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-11T08:05:30.231 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-11T08:05:30.233 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-11T08:05:30.287 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-11T08:05:30.290 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-11T08:05:30.343 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-11T08:05:30.345 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-11T08:05:30.395 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-11T08:05:30.398 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-11T08:05:30.448 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-11T08:05:30.451 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-11T08:05:30.502 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-11T08:05:30.505 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-11T08:05:30.556 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-11T08:05:30.558 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-11T08:05:30.612 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-11T08:05:30.614 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-11T08:05:30.668 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-11T08:05:30.670 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-11T08:05:30.723 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-11T08:05:30.726 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-11T08:05:30.779 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-05-11T08:05:30.779 DEBUG:teuthology.orchestra.run.smithi167:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-05-11T08:05:30.804 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-05-11T08:05:30.900 DEBUG:teuthology.orchestra.run.smithi167:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-05-11T08:05:30.925 DEBUG:teuthology.orchestra.run.smithi204:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-05-11T08:05:30.950 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-05-11T08:05:31.046 DEBUG:teuthology.orchestra.run.smithi204:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-05-11T08:05:31.071 DEBUG:teuthology.orchestra.run.smithi204:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-05-11T08:05:31.135 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-05-11T08:05:31.234 DEBUG:teuthology.orchestra.run.smithi204:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-05-11T08:05:31.258 DEBUG:teuthology.orchestra.run.smithi167:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-05-11T08:05:31.261 DEBUG:teuthology.orchestra.run.smithi204:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-05-11T08:05:31.314 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-05-11T08:05:31.314 INFO:tasks.ceph.mon.a:Restarting daemon 2024-05-11T08:05:31.314 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-05-11T08:05:31.316 INFO:tasks.ceph.mon.a:Started 2024-05-11T08:05:31.317 INFO:tasks.ceph.mon.b:Restarting daemon 2024-05-11T08:05:31.317 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-05-11T08:05:31.356 INFO:tasks.ceph.mon.b:Started 2024-05-11T08:05:31.356 INFO:tasks.ceph.mon.c:Restarting daemon 2024-05-11T08:05:31.356 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-05-11T08:05:31.358 INFO:tasks.ceph.mon.c:Started 2024-05-11T08:05:31.358 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-05-11T08:05:31.358 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-05-11T08:05:31.358 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-05-11T08:05:31.362 INFO:tasks.ceph.mgr.y:Started 2024-05-11T08:05:31.362 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-05-11T08:05:31.362 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-05-11T08:05:31.364 INFO:tasks.ceph.mgr.x:Started 2024-05-11T08:05:31.365 INFO:tasks.ceph:Setting crush tunables to default 2024-05-11T08:05:31.365 DEBUG:teuthology.orchestra.run.smithi167:> sudo ceph --cluster ceph osd crush tunables default 2024-05-11T08:05:31.464 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:31.464+0000 7f366edb6640 1 Processor -- start 2024-05-11T08:05:31.465 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:31.464+0000 7f366edb6640 1 -- start start 2024-05-11T08:05:31.466 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:31.465+0000 7f366edb6640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f36680cf8a0 0x7f36680d07e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:05:31.466 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:31.465+0000 7f366edb6640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f36680cfd30 0x7f36680d0d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:05:31.466 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:31.465+0000 7f366edb6640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36680b6410 con 0x7f36681458f0 2024-05-11T08:05:31.466 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:31.465+0000 7f366edb6640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f36680b7020 con 0x7f36680cfd30 2024-05-11T08:05:31.466 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:31.465+0000 7f366edb6640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f36680b66f0 con 0x7f36680cf8a0 2024-05-11T08:05:31.466 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:31.465+0000 7f366d353640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f36680cfd30 0x7f36680d0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:05:31.467 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:31.465+0000 7f366cb52640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f36681458f0 0x7f36680b74b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:45578/0 (socket says 172.21.15.167:45578) 2024-05-11T08:05:31.467 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:31.465+0000 7f366cb52640 1 -- 172.21.15.167:0/2947546137 learned_addr learned my addr 172.21.15.167:0/2947546137 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T08:05:31.467 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:31.466+0000 7f365ffff640 1 --2- 172.21.15.167:0/2947546137 >> v2:172.21.15.204:3300/0 conn(0x7f36680cf8a0 0x7f36680d07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:05:31.467 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:31.466+0000 7f366d353640 1 -- 172.21.15.167:0/2947546137 >> v2:172.21.15.167:3300/0 conn(0x7f36680cfd30 msgr2=0x7f36680d0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-05-11T08:05:31.467 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:31.466+0000 7f366d353640 1 -- 172.21.15.167:0/2947546137 >> v2:172.21.15.167:3300/0 conn(0x7f36680cfd30 msgr2=0x7f36680d0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-11T08:05:31.467 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:31.466+0000 7f366d353640 1 --2- 172.21.15.167:0/2947546137 >> v2:172.21.15.167:3300/0 conn(0x7f36680cfd30 0x7f36680d0d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-11T08:05:31.467 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:31.466+0000 7f366d353640 1 --2- 172.21.15.167:0/2947546137 >> v2:172.21.15.167:3300/0 conn(0x7f36680cfd30 0x7f36680d0d20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-11T08:05:31.467 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:31.466+0000 7f365ffff640 1 -- 172.21.15.167:0/2947546137 >> v2:172.21.15.204:3300/0 conn(0x7f36680cf8a0 msgr2=0x7f36680d07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-11T08:05:31.467 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:31.466+0000 7f365ffff640 1 -- 172.21.15.167:0/2947546137 >> v2:172.21.15.204:3300/0 conn(0x7f36680cf8a0 msgr2=0x7f36680d07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-11T08:05:31.467 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:31.466+0000 7f365ffff640 1 --2- 172.21.15.167:0/2947546137 >> v2:172.21.15.204:3300/0 conn(0x7f36680cf8a0 0x7f36680d07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-11T08:05:31.467 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:31.466+0000 7f365ffff640 1 --2- 172.21.15.167:0/2947546137 >> v2:172.21.15.204:3300/0 conn(0x7f36680cf8a0 0x7f36680d07e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-11T08:05:31.634 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:31.631+0000 7f365f7fe640 1 -- 172.21.15.167:0/2947546137 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 69184507 0 0) 0x7f36680b6410 con 0x7f36681458f0 2024-05-11T08:05:31.634 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:31.631+0000 7f365f7fe640 1 -- 172.21.15.167:0/2947546137 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3650003520 con 0x7f36681458f0 2024-05-11T08:05:31.634 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:31.631+0000 7f365f7fe640 1 -- 172.21.15.167:0/2947546137 <== mon.0 v1:172.21.15.167:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3834317563 0 0) 0x7f3650003520 con 0x7f36681458f0 2024-05-11T08:05:31.635 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:31.631+0000 7f365f7fe640 1 -- 172.21.15.167:0/2947546137 >> v2:172.21.15.204:3300/0 conn(0x7f36680cf8a0 msgr2=0x7f36680d07e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-11T08:05:31.635 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:31.631+0000 7f365f7fe640 1 --2- 172.21.15.167:0/2947546137 >> v2:172.21.15.204:3300/0 conn(0x7f36680cf8a0 0x7f36680d07e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:31.635 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:31.631+0000 7f365f7fe640 1 -- 172.21.15.167:0/2947546137 >> v2:172.21.15.167:3300/0 conn(0x7f36680cfd30 msgr2=0x7f36680d0d20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-11T08:05:31.635 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:31.631+0000 7f365f7fe640 1 --2- 172.21.15.167:0/2947546137 >> v2:172.21.15.167:3300/0 conn(0x7f36680cfd30 0x7f36680d0d20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:31.635 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:31.631+0000 7f365f7fe640 1 -- 172.21.15.167:0/2947546137 --> v1:172.21.15.167:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36680b69d0 con 0x7f36681458f0 2024-05-11T08:05:31.635 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:31.631+0000 7f365f7fe640 1 -- 172.21.15.167:0/2947546137 <== mon.0 v1:172.21.15.167:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3602277846 0 0) 0x7f36640016f0 con 0x7f36681458f0 2024-05-11T08:05:31.635 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:31.632+0000 7f365f7fe640 1 -- 172.21.15.167:0/2947546137 >> v1:172.21.15.167:6789/0 conn(0x7f36681458f0 legacy=0x7f36680b74b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:05:31.635 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:31.632+0000 7f365f7fe640 1 --2- 172.21.15.167:0/2947546137 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3650003aa0 0x7f3650003ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:05:31.635 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:31.632+0000 7f365f7fe640 1 --2- 172.21.15.167:0/2947546137 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3650004800 0x7f3650004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:05:31.635 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:31.632+0000 7f365f7fe640 1 --2- 172.21.15.167:0/2947546137 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f36500052e0 0x7f365000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:05:31.635 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:31.632+0000 7f365f7fe640 1 -- 172.21.15.167:0/2947546137 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f36680b7020 con 0x7f3650004800 2024-05-11T08:05:31.635 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:31.632+0000 7f365f7fe640 1 -- 172.21.15.167:0/2947546137 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f36680b66f0 con 0x7f36500052e0 2024-05-11T08:05:31.635 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:31.632+0000 7f365f7fe640 1 -- 172.21.15.167:0/2947546137 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f365000c230 con 0x7f3650003aa0 2024-05-11T08:05:31.635 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:31.632+0000 7f365ffff640 1 --2- 172.21.15.167:0/2947546137 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3650004800 0x7f3650004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:05:31.636 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:31.632+0000 7f366cb52640 1 --2- 172.21.15.167:0/2947546137 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3650003aa0 0x7f3650003ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:05:31.636 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:31.632+0000 7f366d353640 1 --2- 172.21.15.167:0/2947546137 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f36500052e0 0x7f365000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:05:31.636 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:31.632+0000 7f365ffff640 1 -- 172.21.15.167:0/2947546137 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3650003aa0 msgr2=0x7f3650003ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:05:31.636 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:31.632+0000 7f365ffff640 1 --2- 172.21.15.167:0/2947546137 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3650003aa0 0x7f3650003ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:31.636 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:31.632+0000 7f365ffff640 1 -- 172.21.15.167:0/2947546137 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f36500052e0 msgr2=0x7f365000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:05:31.636 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:31.632+0000 7f365ffff640 1 --2- 172.21.15.167:0/2947546137 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f36500052e0 0x7f365000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:31.636 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:31.632+0000 7f366d353640 1 -- 172.21.15.167:0/2947546137 reap_dead start 2024-05-11T08:05:31.636 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:31.632+0000 7f365ffff640 1 -- 172.21.15.167:0/2947546137 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3650003520 con 0x7f3650004800 2024-05-11T08:05:31.636 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:31.633+0000 7f365ffff640 1 --2- 172.21.15.167:0/2947546137 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3650004800 0x7f3650004c80 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f365800b9e0 tx=0x7f3650005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd3b6430d7737c95 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:05:31.636 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:31.633+0000 7f365f7fe640 1 -- 172.21.15.167:0/2947546137 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3658030070 con 0x7f3650004800 2024-05-11T08:05:31.636 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:31.633+0000 7f365f7fe640 1 -- 172.21.15.167:0/2947546137 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f365802ada0 con 0x7f3650004800 2024-05-11T08:05:31.636 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:31.633+0000 7f365f7fe640 1 -- 172.21.15.167:0/2947546137 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f365800b5e0 con 0x7f3650004800 2024-05-11T08:05:31.636 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:31.633+0000 7f366edb6640 1 -- 172.21.15.167:0/2947546137 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3650004800 msgr2=0x7f3650004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:05:31.636 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:31.633+0000 7f366edb6640 1 --2- 172.21.15.167:0/2947546137 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3650004800 0x7f3650004c80 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f365800b9e0 tx=0x7f3650005830 comp rx=0 tx=0).stop 2024-05-11T08:05:31.636 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:31.634+0000 7f366edb6640 1 -- 172.21.15.167:0/2947546137 shutdown_connections 2024-05-11T08:05:31.637 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:31.634+0000 7f366edb6640 1 --2- 172.21.15.167:0/2947546137 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3650004800 0x7f3650004c80 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:31.637 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:31.634+0000 7f366edb6640 1 -- 172.21.15.167:0/2947546137 >> 172.21.15.167:0/2947546137 conn(0x7f36680732c0 msgr2=0x7f36680736d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:05:31.637 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:31.634+0000 7f366edb6640 1 -- 172.21.15.167:0/2947546137 shutdown_connections 2024-05-11T08:05:31.637 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:31.634+0000 7f366edb6640 1 -- 172.21.15.167:0/2947546137 wait complete. 2024-05-11T08:05:31.639 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:31.635+0000 7f366edb6640 1 Processor -- start 2024-05-11T08:05:31.639 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:31.635+0000 7f366edb6640 1 -- start start 2024-05-11T08:05:31.639 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:31.636+0000 7f366edb6640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3650004800 0x7f366815a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:05:31.639 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:31.636+0000 7f366edb6640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f36681f72b0 0x7f3668133590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:05:31.639 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:31.636+0000 7f366edb6640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f366815ea90 0x7f36681f66d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:05:31.639 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:31.636+0000 7f366edb6640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f36680b5fc0 con 0x7f36681f72b0 2024-05-11T08:05:31.640 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:31.636+0000 7f366edb6640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f3668145cd0 con 0x7f3650004800 2024-05-11T08:05:31.640 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:31.636+0000 7f366edb6640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f36680b6140 con 0x7f366815ea90 2024-05-11T08:05:31.640 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:31.636+0000 7f365ffff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f36681f72b0 0x7f3668133590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:05:31.640 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:31.636+0000 7f366cb52640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3650004800 0x7f366815a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:05:31.640 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:31.636+0000 7f366cb52640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3650004800 0x7f366815a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.167:59536/0 (socket says 172.21.15.167:59536) 2024-05-11T08:05:31.640 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:31.636+0000 7f366d353640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f366815ea90 0x7f36681f66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:05:31.640 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:31.636+0000 7f366cb52640 1 -- 172.21.15.167:0/999324393 learned_addr learned my addr 172.21.15.167:0/999324393 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:05:31.640 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:31.636+0000 7f365ffff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f36681f72b0 0x7f3668133590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:44118/0 (socket says 172.21.15.167:44118) 2024-05-11T08:05:31.640 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:31.636+0000 7f365ffff640 1 -- 172.21.15.167:0/999324393 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f366815ea90 msgr2=0x7f36681f66d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:05:31.640 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:31.636+0000 7f365ffff640 1 --2- 172.21.15.167:0/999324393 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f366815ea90 0x7f36681f66d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:31.640 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:31.636+0000 7f365ffff640 1 -- 172.21.15.167:0/999324393 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3650004800 msgr2=0x7f366815a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:05:31.640 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:31.636+0000 7f365ffff640 1 --2- 172.21.15.167:0/999324393 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3650004800 0x7f366815a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:31.640 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:31.636+0000 7f365ffff640 1 -- 172.21.15.167:0/999324393 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36681327d0 con 0x7f36681f72b0 2024-05-11T08:05:31.640 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:31.636+0000 7f365ffff640 1 --2- 172.21.15.167:0/999324393 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f36681f72b0 0x7f3668133590 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f365000c180 tx=0x7f3658004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:05:31.640 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:31.637+0000 7f365d7fa640 1 -- 172.21.15.167:0/999324393 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3658043020 con 0x7f36681f72b0 2024-05-11T08:05:31.641 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:31.637+0000 7f365d7fa640 1 -- 172.21.15.167:0/999324393 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f365803f680 con 0x7f36681f72b0 2024-05-11T08:05:31.641 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:31.637+0000 7f366edb6640 1 -- 172.21.15.167:0/999324393 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3668132ab0 con 0x7f36681f72b0 2024-05-11T08:05:31.641 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:31.637+0000 7f365d7fa640 1 -- 172.21.15.167:0/999324393 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f365800b5e0 con 0x7f36681f72b0 2024-05-11T08:05:31.641 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:31.637+0000 7f366edb6640 1 -- 172.21.15.167:0/999324393 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3668133070 con 0x7f36681f72b0 2024-05-11T08:05:31.641 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:31.637+0000 7f365d7fa640 1 -- 172.21.15.167:0/999324393 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f3658046020 con 0x7f36681f72b0 2024-05-11T08:05:31.641 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:31.637+0000 7f365d7fa640 1 -- 172.21.15.167:0/999324393 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f365803dce0 con 0x7f36681f72b0 2024-05-11T08:05:31.641 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:31.638+0000 7f366edb6640 1 -- 172.21.15.167:0/999324393 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3668069bc0 con 0x7f36681f72b0 2024-05-11T08:05:31.641 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:31.640+0000 7f365d7fa640 1 -- 172.21.15.167:0/999324393 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f365804d030 con 0x7f36681f72b0 2024-05-11T08:05:31.706 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:31.706+0000 7f366edb6640 1 -- 172.21.15.167:0/999324393 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f36680741b0 con 0x7f36681f72b0 2024-05-11T08:05:31.708 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:31.708+0000 7f365d7fa640 1 -- 172.21.15.167:0/999324393 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7f365803d440 con 0x7f36681f72b0 2024-05-11T08:05:31.708 INFO:teuthology.orchestra.run.smithi167.stderr:adjusted tunables profile to default 2024-05-11T08:05:31.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:31.709+0000 7f366edb6640 1 -- 172.21.15.167:0/999324393 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f36681f72b0 msgr2=0x7f3668133590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:05:31.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:31.709+0000 7f366edb6640 1 --2- 172.21.15.167:0/999324393 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f36681f72b0 0x7f3668133590 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f365000c180 tx=0x7f3658004730 comp rx=0 tx=0).stop 2024-05-11T08:05:31.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:31.710+0000 7f366edb6640 1 -- 172.21.15.167:0/999324393 shutdown_connections 2024-05-11T08:05:31.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:31.710+0000 7f366edb6640 1 --2- 172.21.15.167:0/999324393 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f366815ea90 0x7f36681f66d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:31.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:31.710+0000 7f366edb6640 1 --2- 172.21.15.167:0/999324393 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f36681f72b0 0x7f3668133590 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:31.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:31.710+0000 7f366edb6640 1 --2- 172.21.15.167:0/999324393 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3650004800 0x7f366815a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:31.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:31.710+0000 7f366edb6640 1 -- 172.21.15.167:0/999324393 >> 172.21.15.167:0/999324393 conn(0x7f36680732c0 msgr2=0x7f366813ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:05:31.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:31.710+0000 7f366edb6640 1 -- 172.21.15.167:0/999324393 shutdown_connections 2024-05-11T08:05:31.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:31.710+0000 7f366edb6640 1 -- 172.21.15.167:0/999324393 wait complete. 2024-05-11T08:05:31.724 INFO:tasks.ceph:check_enable_crimson: False 2024-05-11T08:05:31.724 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-05-11T08:05:31.724 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-05-11T08:05:31.724 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-05-11T08:05:31.735 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-05-11T08:05:31.735+0000 7fa318e13040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-11T08:05:31.749 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-05-11T08:05:31.749 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-05-11T08:05:31.755 INFO:tasks.ceph.mgr.y.smithi167.stderr:2024-05-11T08:05:31.756+0000 7f84d39b8040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-11T08:05:31.780 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-05-11T08:05:31.780 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-05-11T08:05:31.845 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-05-11T08:05:31.845 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-05-11T08:05:31.887 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-05-11T08:05:31.887+0000 7fa318e13040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-11T08:05:31.909 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-11T08:05:31.909 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-05-11T08:05:31.915 INFO:tasks.ceph.mgr.y.smithi167.stderr:2024-05-11T08:05:31.915+0000 7f84d39b8040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-11T08:05:31.935 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-11T08:05:31.935 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-05-11T08:05:31.941 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-05-11T08:05:31.941+0000 7fa318e13040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-11T08:05:31.965 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-11T08:05:31.966 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-05-11T08:05:31.969 INFO:tasks.ceph.mgr.y.smithi167.stderr:2024-05-11T08:05:31.970+0000 7f84d39b8040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-11T08:05:32.031 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-11T08:05:32.031 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-05-11T08:05:32.096 DEBUG:teuthology.orchestra.run.smithi204:> sudo ceph --cluster ceph osd new 587eaced-5ded-406d-884b-30c1caa8686b 0 2024-05-11T08:05:32.212 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-05-11T08:05:32.212+0000 7fa318e13040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-11T08:05:32.216 INFO:tasks.ceph.mgr.y.smithi167.stderr:2024-05-11T08:05:32.217+0000 7f84d39b8040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-11T08:05:32.238 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.238+0000 7fb9c22a7640 1 Processor -- start 2024-05-11T08:05:32.238 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.238+0000 7fb9c22a7640 1 -- start start 2024-05-11T08:05:32.240 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.239+0000 7fb9c22a7640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fb9bc15d0d0 0x7fb9bc06e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:05:32.240 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.239+0000 7fb9c22a7640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fb9bc0d0b10 0x7fb9bc0d0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:05:32.240 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.239+0000 7fb9c22a7640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9bc0711c0 con 0x7fb9bc06cef0 2024-05-11T08:05:32.240 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.239+0000 7fb9c22a7640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7fb9bc06d350 con 0x7fb9bc15d0d0 2024-05-11T08:05:32.240 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.239+0000 7fb9c22a7640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fb9bc06fd30 con 0x7fb9bc0d0b10 2024-05-11T08:05:32.240 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.239+0000 7fb9c0844640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fb9bc0d0b10 0x7fb9bc0d0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:05:32.240 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.239+0000 7fb9c0844640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fb9bc0d0b10 0x7fb9bc0d0fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:33460/0 (socket says 172.21.15.204:33460) 2024-05-11T08:05:32.240 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.239+0000 7fb9c0844640 1 -- 172.21.15.204:0/3694730745 learned_addr learned my addr 172.21.15.204:0/3694730745 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-05-11T08:05:32.240 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.239+0000 7fb9bbfff640 1 --2- 172.21.15.204:0/3694730745 >> v2:172.21.15.167:3300/0 conn(0x7fb9bc15d0d0 0x7fb9bc06e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:05:32.240 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.239+0000 7fb9c0844640 1 -- 172.21.15.204:0/3694730745 >> v2:172.21.15.167:3300/0 conn(0x7fb9bc15d0d0 msgr2=0x7fb9bc06e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:05:32.240 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.239+0000 7fb9c0844640 1 --2- 172.21.15.204:0/3694730745 >> v2:172.21.15.167:3300/0 conn(0x7fb9bc15d0d0 0x7fb9bc06e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:32.240 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.239+0000 7fb9c0844640 1 -- 172.21.15.204:0/3694730745 >> v1:172.21.15.167:6789/0 conn(0x7fb9bc06cef0 legacy=0x7fb9bc070ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:05:32.241 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.239+0000 7fb9c0844640 1 -- 172.21.15.204:0/3694730745 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9bc0b71e0 con 0x7fb9bc0d0b10 2024-05-11T08:05:32.241 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.240+0000 7fb9bbfff640 1 --2- 172.21.15.204:0/3694730745 >> v2:172.21.15.167:3300/0 conn(0x7fb9bc15d0d0 0x7fb9bc06e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T08:05:32.242 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.240+0000 7fb9c0844640 1 --2- 172.21.15.204:0/3694730745 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb9bc0d0b10 0x7fb9bc0d0fa0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb9bc0b4f40 tx=0x7fb9b402a730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3c2fa8dac92fa157 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:05:32.242 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.240+0000 7fb9baffd640 1 -- 172.21.15.204:0/3694730745 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9b4036070 con 0x7fb9bc0d0b10 2024-05-11T08:05:32.242 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.240+0000 7fb9baffd640 1 -- 172.21.15.204:0/3694730745 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9b4031600 con 0x7fb9bc0d0b10 2024-05-11T08:05:32.242 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.240+0000 7fb9baffd640 1 -- 172.21.15.204:0/3694730745 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9b40319d0 con 0x7fb9bc0d0b10 2024-05-11T08:05:32.242 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.241+0000 7fb9c22a7640 1 -- 172.21.15.204:0/3694730745 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb9bc0d0b10 msgr2=0x7fb9bc0d0fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:05:32.242 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.241+0000 7fb9c22a7640 1 --2- 172.21.15.204:0/3694730745 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb9bc0d0b10 0x7fb9bc0d0fa0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb9bc0b4f40 tx=0x7fb9b402a730 comp rx=0 tx=0).stop 2024-05-11T08:05:32.242 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.241+0000 7fb9c22a7640 1 -- 172.21.15.204:0/3694730745 shutdown_connections 2024-05-11T08:05:32.243 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.241+0000 7fb9c22a7640 1 --2- 172.21.15.204:0/3694730745 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb9bc0d0b10 0x7fb9bc0d0fa0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:32.243 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.241+0000 7fb9c22a7640 1 --2- 172.21.15.204:0/3694730745 >> v2:172.21.15.167:3300/0 conn(0x7fb9bc15d0d0 0x7fb9bc06e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:32.243 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.241+0000 7fb9c22a7640 1 -- 172.21.15.204:0/3694730745 >> 172.21.15.204:0/3694730745 conn(0x7fb9bc0732c0 msgr2=0x7fb9bc0736d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:05:32.243 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.241+0000 7fb9c22a7640 1 -- 172.21.15.204:0/3694730745 shutdown_connections 2024-05-11T08:05:32.243 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.241+0000 7fb9c22a7640 1 -- 172.21.15.204:0/3694730745 wait complete. 2024-05-11T08:05:32.243 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.242+0000 7fb9c22a7640 1 Processor -- start 2024-05-11T08:05:32.243 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.242+0000 7fb9c22a7640 1 -- start start 2024-05-11T08:05:32.245 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.243+0000 7fb9c22a7640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb9bc06cef0 0x7fb9bc152d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:05:32.245 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.243+0000 7fb9c22a7640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb9bc0d0b10 0x7fb9bc151050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:05:32.245 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.243+0000 7fb9c22a7640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb9bc15d0d0 0x7fb9bc154c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:05:32.246 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.243+0000 7fb9c22a7640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb9bc0702b0 con 0x7fb9bc06cef0 2024-05-11T08:05:32.246 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.243+0000 7fb9c22a7640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb9bc06ffd0 con 0x7fb9bc0d0b10 2024-05-11T08:05:32.246 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.243+0000 7fb9c22a7640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fb9bc070590 con 0x7fb9bc15d0d0 2024-05-11T08:05:32.246 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.243+0000 7fb9bb7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb9bc0d0b10 0x7fb9bc151050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:05:32.246 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.243+0000 7fb9c0844640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb9bc15d0d0 0x7fb9bc154c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:05:32.246 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.243+0000 7fb9c0844640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb9bc15d0d0 0x7fb9bc154c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:58282/0 (socket says 172.21.15.204:58282) 2024-05-11T08:05:32.246 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.243+0000 7fb9bb7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb9bc0d0b10 0x7fb9bc151050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:33466/0 (socket says 172.21.15.204:33466) 2024-05-11T08:05:32.246 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.243+0000 7fb9c0844640 1 -- 172.21.15.204:0/1048851433 learned_addr learned my addr 172.21.15.204:0/1048851433 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-05-11T08:05:32.246 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.243+0000 7fb9bbfff640 1 --2- 172.21.15.204:0/1048851433 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb9bc06cef0 0x7fb9bc152d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:05:32.246 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.244+0000 7fb9c0844640 1 -- 172.21.15.204:0/1048851433 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb9bc0d0b10 msgr2=0x7fb9bc151050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:05:32.246 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.244+0000 7fb9c0844640 1 --2- 172.21.15.204:0/1048851433 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb9bc0d0b10 0x7fb9bc151050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:32.246 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.244+0000 7fb9c0844640 1 -- 172.21.15.204:0/1048851433 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb9bc06cef0 msgr2=0x7fb9bc152d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:05:32.246 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.244+0000 7fb9c0844640 1 --2- 172.21.15.204:0/1048851433 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb9bc06cef0 0x7fb9bc152d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:32.247 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.244+0000 7fb9c0844640 1 -- 172.21.15.204:0/1048851433 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9bc1520f0 con 0x7fb9bc15d0d0 2024-05-11T08:05:32.247 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.244+0000 7fb9bb7fe640 1 --2- 172.21.15.204:0/1048851433 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb9bc0d0b10 0x7fb9bc151050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T08:05:32.247 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.244+0000 7fb9bbfff640 1 --2- 172.21.15.204:0/1048851433 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb9bc06cef0 0x7fb9bc152d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T08:05:32.247 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.244+0000 7fb9c0844640 1 --2- 172.21.15.204:0/1048851433 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb9bc15d0d0 0x7fb9bc154c60 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb9b402ad20 tx=0x7fb9b4004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:05:32.247 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.244+0000 7fb9b8ff9640 1 -- 172.21.15.204:0/1048851433 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9b4041020 con 0x7fb9bc15d0d0 2024-05-11T08:05:32.247 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.244+0000 7fb9b8ff9640 1 -- 172.21.15.204:0/1048851433 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9b40077a0 con 0x7fb9bc15d0d0 2024-05-11T08:05:32.247 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.244+0000 7fb9c22a7640 1 -- 172.21.15.204:0/1048851433 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9bc153420 con 0x7fb9bc15d0d0 2024-05-11T08:05:32.247 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.244+0000 7fb9b8ff9640 1 -- 172.21.15.204:0/1048851433 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9b4036040 con 0x7fb9bc15d0d0 2024-05-11T08:05:32.247 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.244+0000 7fb9c22a7640 1 -- 172.21.15.204:0/1048851433 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9bc1539e0 con 0x7fb9bc15d0d0 2024-05-11T08:05:32.247 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.244+0000 7fb9b8ff9640 1 -- 172.21.15.204:0/1048851433 <== mon.2 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fb9b4044020 con 0x7fb9bc15d0d0 2024-05-11T08:05:32.247 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.244+0000 7fb9b8ff9640 1 -- 172.21.15.204:0/1048851433 <== mon.2 v2:172.21.15.204:3301/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7fb9b403c650 con 0x7fb9bc15d0d0 2024-05-11T08:05:32.247 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.245+0000 7fb9c22a7640 1 -- 172.21.15.204:0/1048851433 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9bc069bc0 con 0x7fb9bc15d0d0 2024-05-11T08:05:32.248 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.247+0000 7fb9b8ff9640 1 -- 172.21.15.204:0/1048851433 <== mon.2 v2:172.21.15.204:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fb9b404b030 con 0x7fb9bc15d0d0 2024-05-11T08:05:32.260 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-05-11T08:05:32.261+0000 7fa318e13040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-11T08:05:32.262 INFO:tasks.ceph.mgr.y.smithi167.stderr:2024-05-11T08:05:32.263+0000 7f84d39b8040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-11T08:05:32.320 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.320+0000 7fb9c22a7640 1 -- 172.21.15.204:0/1048851433 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd new", "uuid": "587eaced-5ded-406d-884b-30c1caa8686b", "id": 0} v 0) -- 0x7fb9bc0b23c0 con 0x7fb9bc15d0d0 2024-05-11T08:05:32.323 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.323+0000 7fb9b8ff9640 1 -- 172.21.15.204:0/1048851433 <== mon.2 v2:172.21.15.204:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "587eaced-5ded-406d-884b-30c1caa8686b", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7fb9b4007b90 con 0x7fb9bc15d0d0 2024-05-11T08:05:32.323 INFO:teuthology.orchestra.run.smithi204.stdout:0 2024-05-11T08:05:32.324 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.324+0000 7fb9c22a7640 1 -- 172.21.15.204:0/1048851433 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb9bc15d0d0 msgr2=0x7fb9bc154c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:05:32.324 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.324+0000 7fb9c22a7640 1 --2- 172.21.15.204:0/1048851433 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb9bc15d0d0 0x7fb9bc154c60 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb9b402ad20 tx=0x7fb9b4004a40 comp rx=0 tx=0).stop 2024-05-11T08:05:32.324 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.324+0000 7fb9c22a7640 1 -- 172.21.15.204:0/1048851433 shutdown_connections 2024-05-11T08:05:32.324 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.324+0000 7fb9c22a7640 1 --2- 172.21.15.204:0/1048851433 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb9bc15d0d0 0x7fb9bc154c60 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:32.324 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.324+0000 7fb9c22a7640 1 --2- 172.21.15.204:0/1048851433 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb9bc0d0b10 0x7fb9bc151050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:32.324 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.324+0000 7fb9c22a7640 1 --2- 172.21.15.204:0/1048851433 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb9bc06cef0 0x7fb9bc152d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:32.324 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.324+0000 7fb9c22a7640 1 -- 172.21.15.204:0/1048851433 >> 172.21.15.204:0/1048851433 conn(0x7fb9bc0732c0 msgr2=0x7fb9bc15c340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:05:32.324 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.324+0000 7fb9c22a7640 1 -- 172.21.15.204:0/1048851433 shutdown_connections 2024-05-11T08:05:32.324 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.324+0000 7fb9c22a7640 1 -- 172.21.15.204:0/1048851433 wait complete. 2024-05-11T08:05:32.335 DEBUG:teuthology.orchestra.run.smithi204:> sudo ceph --cluster ceph osd new 37c59104-2dd0-4cf9-85ea-56636f6b02a1 1 2024-05-11T08:05:32.434 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.434+0000 7fa4faac8640 1 Processor -- start 2024-05-11T08:05:32.434 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.434+0000 7fa4faac8640 1 -- start start 2024-05-11T08:05:32.436 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.435+0000 7fa4faac8640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa4f415ac90 0x7fa4f40b3e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:05:32.436 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.435+0000 7fa4faac8640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fa4f40b4340 0x7fa4f40b4a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:05:32.436 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.435+0000 7fa4faac8640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4f4160d80 con 0x7fa4f40b4f60 2024-05-11T08:05:32.436 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.435+0000 7fa4faac8640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fa4f415e700 con 0x7fa4f40b4340 2024-05-11T08:05:32.436 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.435+0000 7fa4faac8640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fa4f4161060 con 0x7fa4f415ac90 2024-05-11T08:05:32.437 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.435+0000 7fa4f8864640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa4f415ac90 0x7fa4f40b3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:05:32.437 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.435+0000 7fa4f3fff640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fa4f40b4340 0x7fa4f40b4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:05:32.437 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.435+0000 7fa4f3fff640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fa4f40b4340 0x7fa4f40b4a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:33478/0 (socket says 172.21.15.204:33478) 2024-05-11T08:05:32.437 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.435+0000 7fa4f9065640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7fa4f40b4f60 0x7fa4f40d0b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.204:57944/0 (socket says 172.21.15.204:57944) 2024-05-11T08:05:32.437 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.435+0000 7fa4f3fff640 1 -- 172.21.15.204:0/2056110970 learned_addr learned my addr 172.21.15.204:0/2056110970 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-05-11T08:05:32.437 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.435+0000 7fa4f8864640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa4f415ac90 0x7fa4f40b3e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:58292/0 (socket says 172.21.15.204:58292) 2024-05-11T08:05:32.437 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.435+0000 7fa4f3fff640 1 -- 172.21.15.204:0/2056110970 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa4f415ac90 msgr2=0x7fa4f40b3e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:05:32.437 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.435+0000 7fa4f3fff640 1 --2- 172.21.15.204:0/2056110970 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa4f415ac90 0x7fa4f40b3e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:32.437 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.435+0000 7fa4f3fff640 1 -- 172.21.15.204:0/2056110970 >> v1:172.21.15.204:6789/0 conn(0x7fa4f40b4f60 legacy=0x7fa4f40d0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:05:32.437 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.435+0000 7fa4f3fff640 1 -- 172.21.15.204:0/2056110970 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4f4161340 con 0x7fa4f40b4340 2024-05-11T08:05:32.437 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.435+0000 7fa4f37fe640 1 -- 172.21.15.204:0/2056110970 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 50609484 0 0) 0x7fa4f4160d80 con 0x7fa4f40b4f60 2024-05-11T08:05:32.437 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.436+0000 7fa4f3fff640 1 --2- 172.21.15.204:0/2056110970 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa4f40b4340 0x7fa4f40b4a20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa4e0009870 tx=0x7fa4e002a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b335cbb054ccc32 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:05:32.437 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.436+0000 7fa4f37fe640 1 -- 172.21.15.204:0/2056110970 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4e0037070 con 0x7fa4f40b4340 2024-05-11T08:05:32.437 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.436+0000 7fa4f37fe640 1 -- 172.21.15.204:0/2056110970 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4e0032440 con 0x7fa4f40b4340 2024-05-11T08:05:32.438 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.436+0000 7fa4f37fe640 1 -- 172.21.15.204:0/2056110970 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4e0032810 con 0x7fa4f40b4340 2024-05-11T08:05:32.440 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.437+0000 7fa4faac8640 1 -- 172.21.15.204:0/2056110970 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa4f40b4340 msgr2=0x7fa4f40b4a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:05:32.440 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.437+0000 7fa4faac8640 1 --2- 172.21.15.204:0/2056110970 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa4f40b4340 0x7fa4f40b4a20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa4e0009870 tx=0x7fa4e002a270 comp rx=0 tx=0).stop 2024-05-11T08:05:32.440 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.437+0000 7fa4faac8640 1 -- 172.21.15.204:0/2056110970 shutdown_connections 2024-05-11T08:05:32.441 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.437+0000 7fa4faac8640 1 --2- 172.21.15.204:0/2056110970 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa4f40b4340 0x7fa4f40b4a20 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:32.441 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.437+0000 7fa4faac8640 1 --2- 172.21.15.204:0/2056110970 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa4f415ac90 0x7fa4f40b3e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:32.441 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.437+0000 7fa4faac8640 1 -- 172.21.15.204:0/2056110970 >> 172.21.15.204:0/2056110970 conn(0x7fa4f406ca80 msgr2=0x7fa4f40d2fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:05:32.441 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.437+0000 7fa4faac8640 1 -- 172.21.15.204:0/2056110970 shutdown_connections 2024-05-11T08:05:32.441 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.437+0000 7fa4faac8640 1 -- 172.21.15.204:0/2056110970 wait complete. 2024-05-11T08:05:32.441 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.438+0000 7fa4faac8640 1 Processor -- start 2024-05-11T08:05:32.441 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.438+0000 7fa4faac8640 1 -- start start 2024-05-11T08:05:32.441 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.439+0000 7fa4faac8640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa4f40b4340 0x7fa4f4151790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:05:32.441 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.439+0000 7fa4faac8640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa4f40b4f60 0x7fa4f4152fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:05:32.441 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.439+0000 7fa4faac8640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa4f415ac90 0x7fa4f4153520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:05:32.441 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.439+0000 7fa4faac8640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa4f415e8b0 con 0x7fa4f40b4340 2024-05-11T08:05:32.441 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.439+0000 7fa4faac8640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa4f415b070 con 0x7fa4f40b4f60 2024-05-11T08:05:32.441 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.439+0000 7fa4faac8640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fa4f41f3900 con 0x7fa4f415ac90 2024-05-11T08:05:32.442 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.439+0000 7fa4f3fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa4f40b4f60 0x7fa4f4152fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:05:32.442 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.439+0000 7fa4f3fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa4f40b4f60 0x7fa4f4152fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:33494/0 (socket says 172.21.15.204:33494) 2024-05-11T08:05:32.442 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.439+0000 7fa4f3fff640 1 -- 172.21.15.204:0/3578470207 learned_addr learned my addr 172.21.15.204:0/3578470207 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-05-11T08:05:32.442 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.439+0000 7fa4f9065640 1 --2- 172.21.15.204:0/3578470207 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa4f415ac90 0x7fa4f4153520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:05:32.442 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.439+0000 7fa4f8864640 1 --2- 172.21.15.204:0/3578470207 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa4f40b4340 0x7fa4f4151790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:05:32.442 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.439+0000 7fa4f3fff640 1 -- 172.21.15.204:0/3578470207 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa4f415ac90 msgr2=0x7fa4f4153520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:05:32.442 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.439+0000 7fa4f3fff640 1 --2- 172.21.15.204:0/3578470207 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa4f415ac90 0x7fa4f4153520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:32.442 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.439+0000 7fa4f3fff640 1 -- 172.21.15.204:0/3578470207 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa4f40b4340 msgr2=0x7fa4f4151790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:05:32.442 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.439+0000 7fa4f3fff640 1 --2- 172.21.15.204:0/3578470207 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa4f40b4340 0x7fa4f4151790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:32.442 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.439+0000 7fa4f3fff640 1 -- 172.21.15.204:0/3578470207 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4f41f3be0 con 0x7fa4f40b4f60 2024-05-11T08:05:32.442 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.440+0000 7fa4f3fff640 1 --2- 172.21.15.204:0/3578470207 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa4f40b4f60 0x7fa4f4152fe0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa4e0000c00 tx=0x7fa4e00046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:05:32.442 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.440+0000 7fa4f17fa640 1 -- 172.21.15.204:0/3578470207 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4e0037040 con 0x7fa4f40b4f60 2024-05-11T08:05:32.442 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.440+0000 7fa4f17fa640 1 -- 172.21.15.204:0/3578470207 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4e0042070 con 0x7fa4f40b4f60 2024-05-11T08:05:32.442 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.440+0000 7fa4f17fa640 1 -- 172.21.15.204:0/3578470207 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4e0030880 con 0x7fa4f40b4f60 2024-05-11T08:05:32.443 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.440+0000 7fa4faac8640 1 -- 172.21.15.204:0/3578470207 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4f41f3ec0 con 0x7fa4f40b4f60 2024-05-11T08:05:32.443 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.440+0000 7fa4faac8640 1 -- 172.21.15.204:0/3578470207 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4f41f35f0 con 0x7fa4f40b4f60 2024-05-11T08:05:32.443 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.440+0000 7fa4f17fa640 1 -- 172.21.15.204:0/3578470207 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fa4e0030a20 con 0x7fa4f40b4f60 2024-05-11T08:05:32.443 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.440+0000 7fa4f17fa640 1 -- 172.21.15.204:0/3578470207 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7fa4e0047c10 con 0x7fa4f40b4f60 2024-05-11T08:05:32.443 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.441+0000 7fa4faac8640 1 -- 172.21.15.204:0/3578470207 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4f4069bc0 con 0x7fa4f40b4f60 2024-05-11T08:05:32.443 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.443+0000 7fa4f17fa640 1 -- 172.21.15.204:0/3578470207 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fa4e0047460 con 0x7fa4f40b4f60 2024-05-11T08:05:32.514 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.514+0000 7fa4faac8640 1 -- 172.21.15.204:0/3578470207 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd new", "uuid": "37c59104-2dd0-4cf9-85ea-56636f6b02a1", "id": 1} v 0) -- 0x7fa4f40b47d0 con 0x7fa4f40b4f60 2024-05-11T08:05:32.519 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.519+0000 7fa4f17fa640 1 -- 172.21.15.204:0/3578470207 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "37c59104-2dd0-4cf9-85ea-56636f6b02a1", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7fa4e0047e10 con 0x7fa4f40b4f60 2024-05-11T08:05:32.519 INFO:teuthology.orchestra.run.smithi204.stdout:1 2024-05-11T08:05:32.520 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.520+0000 7fa4faac8640 1 -- 172.21.15.204:0/3578470207 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa4f40b4f60 msgr2=0x7fa4f4152fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:05:32.520 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.520+0000 7fa4faac8640 1 --2- 172.21.15.204:0/3578470207 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa4f40b4f60 0x7fa4f4152fe0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa4e0000c00 tx=0x7fa4e00046c0 comp rx=0 tx=0).stop 2024-05-11T08:05:32.520 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.520+0000 7fa4faac8640 1 -- 172.21.15.204:0/3578470207 shutdown_connections 2024-05-11T08:05:32.520 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.520+0000 7fa4faac8640 1 --2- 172.21.15.204:0/3578470207 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa4f415ac90 0x7fa4f4153520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:32.520 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.520+0000 7fa4faac8640 1 --2- 172.21.15.204:0/3578470207 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa4f40b4f60 0x7fa4f4152fe0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:32.520 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.520+0000 7fa4faac8640 1 --2- 172.21.15.204:0/3578470207 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa4f40b4340 0x7fa4f4151790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:32.520 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.520+0000 7fa4faac8640 1 -- 172.21.15.204:0/3578470207 >> 172.21.15.204:0/3578470207 conn(0x7fa4f406ca80 msgr2=0x7fa4f40681c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:05:32.520 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.520+0000 7fa4faac8640 1 -- 172.21.15.204:0/3578470207 shutdown_connections 2024-05-11T08:05:32.521 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.520+0000 7fa4faac8640 1 -- 172.21.15.204:0/3578470207 wait complete. 2024-05-11T08:05:32.530 DEBUG:teuthology.orchestra.run.smithi204:> sudo ceph --cluster ceph osd new 2bb4f7f5-c698-451b-b4bb-78e716921291 2 2024-05-11T08:05:32.632 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.631+0000 7f2e363be640 1 Processor -- start 2024-05-11T08:05:32.632 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.632+0000 7f2e363be640 1 -- start start 2024-05-11T08:05:32.634 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.632+0000 7f2e363be640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f2e3015cbf0 0x7f2e300cd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:05:32.634 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.632+0000 7f2e363be640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2e3015cfd0 0x7f2e3006f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:05:32.634 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.632+0000 7f2e363be640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f2e3006fb30 0x7f2e300b49f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:05:32.634 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.632+0000 7f2e363be640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f2e30160850 con 0x7f2e3006fb30 2024-05-11T08:05:32.634 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.632+0000 7f2e363be640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f2e300cdb30 con 0x7f2e3015cbf0 2024-05-11T08:05:32.634 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.632+0000 7f2e363be640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f2e3015fcd0 con 0x7f2e3015cfd0 2024-05-11T08:05:32.634 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.632+0000 7f2e2f7fe640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2e3015cfd0 0x7f2e3006f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:05:32.634 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.633+0000 7f2e2f7fe640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2e3015cfd0 0x7f2e3006f5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:58312/0 (socket says 172.21.15.204:58312) 2024-05-11T08:05:32.634 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.633+0000 7f2e2f7fe640 1 -- 172.21.15.204:0/1995050232 learned_addr learned my addr 172.21.15.204:0/1995050232 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-05-11T08:05:32.634 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.633+0000 7f2e2ffff640 1 --2- 172.21.15.204:0/1995050232 >> v2:172.21.15.204:3300/0 conn(0x7f2e3015cbf0 0x7f2e300cd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:05:32.634 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.633+0000 7f2e3495b640 1 --2- 172.21.15.204:0/1995050232 >> v2:172.21.15.167:3300/0 conn(0x7f2e3006fb30 0x7f2e300b49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:05:32.635 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.633+0000 7f2e2f7fe640 1 -- 172.21.15.204:0/1995050232 >> v2:172.21.15.204:3300/0 conn(0x7f2e3015cbf0 msgr2=0x7f2e300cd5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:05:32.635 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.633+0000 7f2e2f7fe640 1 --2- 172.21.15.204:0/1995050232 >> v2:172.21.15.204:3300/0 conn(0x7f2e3015cbf0 0x7f2e300cd5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:32.635 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.633+0000 7f2e2f7fe640 1 -- 172.21.15.204:0/1995050232 >> v2:172.21.15.167:3300/0 conn(0x7f2e3006fb30 msgr2=0x7f2e300b49f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:05:32.635 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.633+0000 7f2e2f7fe640 1 --2- 172.21.15.204:0/1995050232 >> v2:172.21.15.167:3300/0 conn(0x7f2e3006fb30 0x7f2e300b49f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:32.635 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.633+0000 7f2e2f7fe640 1 -- 172.21.15.204:0/1995050232 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e3015ffb0 con 0x7f2e3015cfd0 2024-05-11T08:05:32.635 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.634+0000 7f2e2f7fe640 1 --2- 172.21.15.204:0/1995050232 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2e3015cfd0 0x7f2e3006f5f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2e3006d100 tx=0x7f2e2402a140 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f6367e450f43 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:05:32.635 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.634+0000 7f2e2effd640 1 -- 172.21.15.204:0/1995050232 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e24037070 con 0x7f2e3015cfd0 2024-05-11T08:05:32.635 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.634+0000 7f2e2effd640 1 -- 172.21.15.204:0/1995050232 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e24032440 con 0x7f2e3015cfd0 2024-05-11T08:05:32.635 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.634+0000 7f2e2effd640 1 -- 172.21.15.204:0/1995050232 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e24032810 con 0x7f2e3015cfd0 2024-05-11T08:05:32.636 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.634+0000 7f2e363be640 1 -- 172.21.15.204:0/1995050232 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2e3015cfd0 msgr2=0x7f2e3006f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:05:32.636 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.634+0000 7f2e363be640 1 --2- 172.21.15.204:0/1995050232 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2e3015cfd0 0x7f2e3006f5f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2e3006d100 tx=0x7f2e2402a140 comp rx=0 tx=0).stop 2024-05-11T08:05:32.636 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.634+0000 7f2e363be640 1 -- 172.21.15.204:0/1995050232 shutdown_connections 2024-05-11T08:05:32.636 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.634+0000 7f2e363be640 1 --2- 172.21.15.204:0/1995050232 >> v2:172.21.15.167:3300/0 conn(0x7f2e3006fb30 0x7f2e300b49f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:32.636 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.634+0000 7f2e363be640 1 --2- 172.21.15.204:0/1995050232 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2e3015cfd0 0x7f2e3006f5f0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:32.636 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.634+0000 7f2e363be640 1 --2- 172.21.15.204:0/1995050232 >> v2:172.21.15.204:3300/0 conn(0x7f2e3015cbf0 0x7f2e300cd5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:32.637 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.635+0000 7f2e363be640 1 -- 172.21.15.204:0/1995050232 >> 172.21.15.204:0/1995050232 conn(0x7f2e3006ca80 msgr2=0x7f2e300d2fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:05:32.637 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.635+0000 7f2e363be640 1 -- 172.21.15.204:0/1995050232 shutdown_connections 2024-05-11T08:05:32.637 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.635+0000 7f2e363be640 1 -- 172.21.15.204:0/1995050232 wait complete. 2024-05-11T08:05:32.637 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.636+0000 7f2e363be640 1 Processor -- start 2024-05-11T08:05:32.637 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.636+0000 7f2e363be640 1 -- start start 2024-05-11T08:05:32.639 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.637+0000 7f2e363be640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2e3006fb30 0x7f2e3015dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:05:32.639 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.637+0000 7f2e363be640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2e3015cbf0 0x7f2e3015e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:05:32.639 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.637+0000 7f2e363be640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2e3015cfd0 0x7f2e3014f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:05:32.639 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.637+0000 7f2e363be640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2e3015f910 con 0x7f2e3015cbf0 2024-05-11T08:05:32.639 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.637+0000 7f2e363be640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f2e30160570 con 0x7f2e3006fb30 2024-05-11T08:05:32.640 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.637+0000 7f2e363be640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f2e3015fa90 con 0x7f2e3015cfd0 2024-05-11T08:05:32.640 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.637+0000 7f2e3495b640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2e3015cfd0 0x7f2e3014f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:05:32.640 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.637+0000 7f2e2ffff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2e3006fb30 0x7f2e3015dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:05:32.640 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.637+0000 7f2e3495b640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2e3015cfd0 0x7f2e3014f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:58322/0 (socket says 172.21.15.204:58322) 2024-05-11T08:05:32.640 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.637+0000 7f2e3495b640 1 -- 172.21.15.204:0/2172299877 learned_addr learned my addr 172.21.15.204:0/2172299877 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-05-11T08:05:32.640 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.637+0000 7f2e2ffff640 1 --2- 172.21.15.204:0/2172299877 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2e3006fb30 0x7f2e3015dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:33512/0 (socket says 172.21.15.204:33512) 2024-05-11T08:05:32.640 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.637+0000 7f2e2f7fe640 1 --2- 172.21.15.204:0/2172299877 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2e3015cbf0 0x7f2e3015e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:05:32.640 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.637+0000 7f2e3495b640 1 -- 172.21.15.204:0/2172299877 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2e3006fb30 msgr2=0x7f2e3015dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:05:32.640 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.637+0000 7f2e3495b640 1 --2- 172.21.15.204:0/2172299877 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2e3006fb30 0x7f2e3015dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:32.640 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.637+0000 7f2e3495b640 1 -- 172.21.15.204:0/2172299877 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2e3015cbf0 msgr2=0x7f2e3015e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:05:32.640 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.637+0000 7f2e3495b640 1 --2- 172.21.15.204:0/2172299877 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2e3015cbf0 0x7f2e3015e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:32.640 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.637+0000 7f2e3495b640 1 -- 172.21.15.204:0/2172299877 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e30158a70 con 0x7f2e3015cfd0 2024-05-11T08:05:32.640 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.637+0000 7f2e2ffff640 1 --2- 172.21.15.204:0/2172299877 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2e3006fb30 0x7f2e3015dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T08:05:32.641 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.637+0000 7f2e3495b640 1 --2- 172.21.15.204:0/2172299877 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2e3015cfd0 0x7f2e3014f470 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2e2800eed0 tx=0x7f2e2800b480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:05:32.641 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.638+0000 7f2e2cff9640 1 -- 172.21.15.204:0/2172299877 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e28018020 con 0x7f2e3015cfd0 2024-05-11T08:05:32.641 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.638+0000 7f2e2cff9640 1 -- 172.21.15.204:0/2172299877 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e2800bd30 con 0x7f2e3015cfd0 2024-05-11T08:05:32.641 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.638+0000 7f2e363be640 1 -- 172.21.15.204:0/2172299877 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e30157cb0 con 0x7f2e3015cfd0 2024-05-11T08:05:32.641 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.638+0000 7f2e2cff9640 1 -- 172.21.15.204:0/2172299877 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e28014640 con 0x7f2e3015cfd0 2024-05-11T08:05:32.641 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.638+0000 7f2e363be640 1 -- 172.21.15.204:0/2172299877 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e30158270 con 0x7f2e3015cfd0 2024-05-11T08:05:32.641 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.638+0000 7f2e2cff9640 1 -- 172.21.15.204:0/2172299877 <== mon.2 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f2e28005d10 con 0x7f2e3015cfd0 2024-05-11T08:05:32.641 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.638+0000 7f2e2cff9640 1 -- 172.21.15.204:0/2172299877 <== mon.2 v2:172.21.15.204:3301/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7f2e2801dc30 con 0x7f2e3015cfd0 2024-05-11T08:05:32.641 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.639+0000 7f2e363be640 1 -- 172.21.15.204:0/2172299877 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e30069bc0 con 0x7f2e3015cfd0 2024-05-11T08:05:32.641 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.641+0000 7f2e2cff9640 1 -- 172.21.15.204:0/2172299877 <== mon.2 v2:172.21.15.204:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f2e28022030 con 0x7f2e3015cfd0 2024-05-11T08:05:32.714 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.713+0000 7f2e363be640 1 -- 172.21.15.204:0/2172299877 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd new", "uuid": "2bb4f7f5-c698-451b-b4bb-78e716921291", "id": 2} v 0) -- 0x7f2e300b8f20 con 0x7f2e3015cfd0 2024-05-11T08:05:32.720 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.720+0000 7f2e2cff9640 1 -- 172.21.15.204:0/2172299877 <== mon.2 v2:172.21.15.204:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "2bb4f7f5-c698-451b-b4bb-78e716921291", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7f2e2801d810 con 0x7f2e3015cfd0 2024-05-11T08:05:32.720 INFO:teuthology.orchestra.run.smithi204.stdout:2 2024-05-11T08:05:32.721 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.721+0000 7f2e363be640 1 -- 172.21.15.204:0/2172299877 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2e3015cfd0 msgr2=0x7f2e3014f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:05:32.721 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.721+0000 7f2e363be640 1 --2- 172.21.15.204:0/2172299877 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2e3015cfd0 0x7f2e3014f470 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2e2800eed0 tx=0x7f2e2800b480 comp rx=0 tx=0).stop 2024-05-11T08:05:32.721 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.721+0000 7f2e363be640 1 -- 172.21.15.204:0/2172299877 shutdown_connections 2024-05-11T08:05:32.721 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.721+0000 7f2e363be640 1 --2- 172.21.15.204:0/2172299877 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2e3015cfd0 0x7f2e3014f470 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:32.721 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.721+0000 7f2e363be640 1 --2- 172.21.15.204:0/2172299877 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2e3015cbf0 0x7f2e3015e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:32.722 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.721+0000 7f2e363be640 1 --2- 172.21.15.204:0/2172299877 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2e3006fb30 0x7f2e3015dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:32.722 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.721+0000 7f2e363be640 1 -- 172.21.15.204:0/2172299877 >> 172.21.15.204:0/2172299877 conn(0x7f2e3006ca80 msgr2=0x7f2e300d2fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:05:32.722 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.721+0000 7f2e363be640 1 -- 172.21.15.204:0/2172299877 shutdown_connections 2024-05-11T08:05:32.722 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.721+0000 7f2e363be640 1 -- 172.21.15.204:0/2172299877 wait complete. 2024-05-11T08:05:32.732 DEBUG:teuthology.orchestra.run.smithi204:> sudo ceph --cluster ceph osd new 9dd26759-a338-49d2-8103-45d068b9c000 3 2024-05-11T08:05:32.833 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.832+0000 7f7685c59640 1 Processor -- start 2024-05-11T08:05:32.833 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.832+0000 7f7685c59640 1 -- start start 2024-05-11T08:05:32.834 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.833+0000 7f7685c59640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f7680159970 0x7f76800d1270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:05:32.835 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.833+0000 7f7685c59640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f76800d17b0 0x7f768006d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:05:32.835 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.833+0000 7f7685c59640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7680159d50 con 0x7f768006d740 2024-05-11T08:05:32.835 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.833+0000 7f7685c59640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f76800d04b0 con 0x7f7680159970 2024-05-11T08:05:32.835 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.833+0000 7f7685c59640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f76800d0790 con 0x7f76800d17b0 2024-05-11T08:05:32.835 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.833+0000 7f767ffff640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f768006d740 0x7f76800d2db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.204:57960/0 (socket says 172.21.15.204:57960) 2024-05-11T08:05:32.835 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.833+0000 7f767ffff640 1 -- 172.21.15.204:0/2100116407 learned_addr learned my addr 172.21.15.204:0/2100116407 (peer_addr_for_me v1:172.21.15.204:0/0) 2024-05-11T08:05:32.835 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.833+0000 7f767effd640 1 --2- 172.21.15.204:0/2100116407 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f76800d17b0 0x7f768006d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:05:32.835 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.834+0000 7f767f7fe640 1 --2- 172.21.15.204:0/2100116407 >> v2:172.21.15.167:3300/0 conn(0x7f7680159970 0x7f76800d1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:05:32.835 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.834+0000 7f767e7fc640 1 -- 172.21.15.204:0/2100116407 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2129433941 0 0) 0x7f7680159d50 con 0x7f768006d740 2024-05-11T08:05:32.835 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.834+0000 7f767e7fc640 1 -- 172.21.15.204:0/2100116407 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7668003520 con 0x7f768006d740 2024-05-11T08:05:32.835 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.834+0000 7f767effd640 1 -- 172.21.15.204:0/2100116407 >> v2:172.21.15.167:3300/0 conn(0x7f7680159970 msgr2=0x7f76800d1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:05:32.835 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.834+0000 7f767effd640 1 --2- 172.21.15.204:0/2100116407 >> v2:172.21.15.167:3300/0 conn(0x7f7680159970 0x7f76800d1270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:32.836 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.834+0000 7f767effd640 1 -- 172.21.15.204:0/2100116407 >> v1:172.21.15.204:6789/0 conn(0x7f768006d740 legacy=0x7f76800d2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:05:32.836 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.834+0000 7f767effd640 1 -- 172.21.15.204:0/2100116407 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76800d0a70 con 0x7f76800d17b0 2024-05-11T08:05:32.836 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.834+0000 7f767effd640 1 --2- 172.21.15.204:0/2100116407 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f76800d17b0 0x7f768006d200 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f76700097d0 tx=0x7f767002a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=95416a1b1e36e172 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:05:32.836 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.835+0000 7f767e7fc640 1 -- 172.21.15.204:0/2100116407 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7670037070 con 0x7f76800d17b0 2024-05-11T08:05:32.836 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.835+0000 7f767e7fc640 1 -- 172.21.15.204:0/2100116407 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7670032440 con 0x7f76800d17b0 2024-05-11T08:05:32.836 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.835+0000 7f767e7fc640 1 -- 172.21.15.204:0/2100116407 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7670032810 con 0x7f76800d17b0 2024-05-11T08:05:32.837 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.835+0000 7f7685c59640 1 -- 172.21.15.204:0/2100116407 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f76800d17b0 msgr2=0x7f768006d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:05:32.837 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.835+0000 7f7685c59640 1 --2- 172.21.15.204:0/2100116407 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f76800d17b0 0x7f768006d200 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f76700097d0 tx=0x7f767002a1c0 comp rx=0 tx=0).stop 2024-05-11T08:05:32.837 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.835+0000 7f7685c59640 1 -- 172.21.15.204:0/2100116407 shutdown_connections 2024-05-11T08:05:32.837 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.835+0000 7f7685c59640 1 --2- 172.21.15.204:0/2100116407 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f76800d17b0 0x7f768006d200 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:32.837 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.835+0000 7f7685c59640 1 --2- 172.21.15.204:0/2100116407 >> v2:172.21.15.167:3300/0 conn(0x7f7680159970 0x7f76800d1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:32.837 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.835+0000 7f7685c59640 1 -- 172.21.15.204:0/2100116407 >> 172.21.15.204:0/2100116407 conn(0x7f76800d51e0 msgr2=0x7f76800d55f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:05:32.837 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.835+0000 7f7685c59640 1 -- 172.21.15.204:0/2100116407 shutdown_connections 2024-05-11T08:05:32.837 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.835+0000 7f7685c59640 1 -- 172.21.15.204:0/2100116407 wait complete. 2024-05-11T08:05:32.837 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.837+0000 7f7685c59640 1 Processor -- start 2024-05-11T08:05:32.838 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.837+0000 7f7685c59640 1 -- start start 2024-05-11T08:05:32.840 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.837+0000 7f7685c59640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f768006d740 0x7f768015c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:05:32.840 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.837+0000 7f7685c59640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f76800d17b0 0x7f7680145b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:05:32.840 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.837+0000 7f7685c59640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7680159970 0x7f76801ea330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:05:32.840 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.838+0000 7f7685c59640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f76800d34c0 con 0x7f76800d17b0 2024-05-11T08:05:32.840 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.838+0000 7f7685c59640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7680163d00 con 0x7f768006d740 2024-05-11T08:05:32.840 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.838+0000 7f7685c59640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f76800d3640 con 0x7f7680159970 2024-05-11T08:05:32.840 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.838+0000 7f767f7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f768006d740 0x7f768015c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:05:32.840 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.838+0000 7f767ffff640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7680159970 0x7f76801ea330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:05:32.841 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.838+0000 7f767ffff640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7680159970 0x7f76801ea330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:58340/0 (socket says 172.21.15.204:58340) 2024-05-11T08:05:32.841 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.838+0000 7f767ffff640 1 -- 172.21.15.204:0/196483090 learned_addr learned my addr 172.21.15.204:0/196483090 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-05-11T08:05:32.841 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.838+0000 7f767effd640 1 --2- 172.21.15.204:0/196483090 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f76800d17b0 0x7f7680145b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:05:32.841 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.838+0000 7f767f7fe640 1 -- 172.21.15.204:0/196483090 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7680159970 msgr2=0x7f76801ea330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:05:32.841 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.838+0000 7f767f7fe640 1 --2- 172.21.15.204:0/196483090 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7680159970 0x7f76801ea330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:32.841 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.838+0000 7f767f7fe640 1 -- 172.21.15.204:0/196483090 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f76800d17b0 msgr2=0x7f7680145b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:05:32.841 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.838+0000 7f767f7fe640 1 --2- 172.21.15.204:0/196483090 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f76800d17b0 0x7f7680145b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:32.841 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.838+0000 7f767f7fe640 1 -- 172.21.15.204:0/196483090 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76801e9290 con 0x7f768006d740 2024-05-11T08:05:32.841 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.838+0000 7f767ffff640 1 --2- 172.21.15.204:0/196483090 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7680159970 0x7f76801ea330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T08:05:32.841 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.838+0000 7f767f7fe640 1 --2- 172.21.15.204:0/196483090 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f768006d740 0x7f768015c930 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f767400e850 tx=0x7f767400ed10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:05:32.841 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.838+0000 7f7667fff640 1 -- 172.21.15.204:0/196483090 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7674017070 con 0x7f768006d740 2024-05-11T08:05:32.841 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.838+0000 7f7667fff640 1 -- 172.21.15.204:0/196483090 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f767400c940 con 0x7f768006d740 2024-05-11T08:05:32.841 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.838+0000 7f7685c59640 1 -- 172.21.15.204:0/196483090 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76801e9570 con 0x7f768006d740 2024-05-11T08:05:32.841 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.838+0000 7f7667fff640 1 -- 172.21.15.204:0/196483090 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f767400cd10 con 0x7f768006d740 2024-05-11T08:05:32.841 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.838+0000 7f7685c59640 1 -- 172.21.15.204:0/196483090 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76801e9b30 con 0x7f768006d740 2024-05-11T08:05:32.842 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.839+0000 7f7667fff640 1 -- 172.21.15.204:0/196483090 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f767401a070 con 0x7f768006d740 2024-05-11T08:05:32.842 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.839+0000 7f7667fff640 1 -- 172.21.15.204:0/196483090 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7f767401c8e0 con 0x7f768006d740 2024-05-11T08:05:32.842 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.839+0000 7f7685c59640 1 -- 172.21.15.204:0/196483090 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7680069bc0 con 0x7f768006d740 2024-05-11T08:05:32.842 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.841+0000 7f7667fff640 1 -- 172.21.15.204:0/196483090 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f767401c510 con 0x7f768006d740 2024-05-11T08:05:32.914 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.914+0000 7f7685c59640 1 -- 172.21.15.204:0/196483090 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd new", "uuid": "9dd26759-a338-49d2-8103-45d068b9c000", "id": 3} v 0) -- 0x7f7680159190 con 0x7f768006d740 2024-05-11T08:05:32.919 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.919+0000 7f7667fff640 1 -- 172.21.15.204:0/196483090 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "9dd26759-a338-49d2-8103-45d068b9c000", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7f767401cbb0 con 0x7f768006d740 2024-05-11T08:05:32.919 INFO:teuthology.orchestra.run.smithi204.stdout:3 2024-05-11T08:05:32.920 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.920+0000 7f7685c59640 1 -- 172.21.15.204:0/196483090 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f768006d740 msgr2=0x7f768015c930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:05:32.920 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.920+0000 7f7685c59640 1 --2- 172.21.15.204:0/196483090 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f768006d740 0x7f768015c930 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f767400e850 tx=0x7f767400ed10 comp rx=0 tx=0).stop 2024-05-11T08:05:32.921 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.920+0000 7f7685c59640 1 -- 172.21.15.204:0/196483090 shutdown_connections 2024-05-11T08:05:32.921 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.920+0000 7f7685c59640 1 --2- 172.21.15.204:0/196483090 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7680159970 0x7f76801ea330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:32.921 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.920+0000 7f7685c59640 1 --2- 172.21.15.204:0/196483090 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f76800d17b0 0x7f7680145b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:32.921 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.920+0000 7f7685c59640 1 --2- 172.21.15.204:0/196483090 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f768006d740 0x7f768015c930 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:32.921 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.920+0000 7f7685c59640 1 -- 172.21.15.204:0/196483090 >> 172.21.15.204:0/196483090 conn(0x7f76800d51e0 msgr2=0x7f76800ba860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:05:32.921 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.920+0000 7f7685c59640 1 -- 172.21.15.204:0/196483090 shutdown_connections 2024-05-11T08:05:32.921 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:32.920+0000 7f7685c59640 1 -- 172.21.15.204:0/196483090 wait complete. 2024-05-11T08:05:32.931 DEBUG:teuthology.orchestra.run.smithi204:> sudo ceph --cluster ceph osd new cb27250d-2a80-4c7b-8cef-3a7af05c8a16 4 2024-05-11T08:05:32.990 INFO:tasks.ceph.mgr.y.smithi167.stderr:2024-05-11T08:05:32.991+0000 7f84d39b8040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-11T08:05:33.031 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.031+0000 7f09676da640 1 Processor -- start 2024-05-11T08:05:33.031 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.031+0000 7f09676da640 1 -- start start 2024-05-11T08:05:33.033 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.032+0000 7f09676da640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f096015ac90 0x7f09600b3e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:05:33.033 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.032+0000 7f09676da640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0960160d80 con 0x7f09600b4340 2024-05-11T08:05:33.033 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.032+0000 7f09676da640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0960161060 con 0x7f09600b5130 2024-05-11T08:05:33.033 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.032+0000 7f09676da640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f0960161340 con 0x7f096015ac90 2024-05-11T08:05:33.034 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.032+0000 7f0965c77640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f09600b5130 0x7f096015d6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.204:57962/0 (socket says 172.21.15.204:57962) 2024-05-11T08:05:33.034 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.032+0000 7f0965c77640 1 -- 172.21.15.204:0/2138244315 learned_addr learned my addr 172.21.15.204:0/2138244315 (peer_addr_for_me v1:172.21.15.204:0/0) 2024-05-11T08:05:33.034 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.032+0000 7f0965476640 1 --2- 172.21.15.204:0/2138244315 >> v2:172.21.15.167:3300/0 conn(0x7f096015ac90 0x7f09600b3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:05:33.034 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.032+0000 7f0957fff640 1 -- 172.21.15.204:0/2138244315 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3375201393 0 0) 0x7f0960161060 con 0x7f09600b5130 2024-05-11T08:05:33.034 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.032+0000 7f0957fff640 1 -- 172.21.15.204:0/2138244315 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f094c003520 con 0x7f09600b5130 2024-05-11T08:05:33.034 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.032+0000 7f0957fff640 1 -- 172.21.15.204:0/2138244315 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3345018139 0 0) 0x7f0960160d80 con 0x7f09600b4340 2024-05-11T08:05:33.034 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.032+0000 7f0957fff640 1 -- 172.21.15.204:0/2138244315 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0960161060 con 0x7f09600b4340 2024-05-11T08:05:33.034 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.033+0000 7f0965476640 1 -- 172.21.15.204:0/2138244315 >> v1:172.21.15.204:6789/0 conn(0x7f09600b5130 legacy=0x7f096015d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:05:33.034 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.033+0000 7f0965476640 1 -- 172.21.15.204:0/2138244315 >> v1:172.21.15.167:6789/0 conn(0x7f09600b4340 legacy=0x7f09600b4a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:05:33.034 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.033+0000 7f0965476640 1 -- 172.21.15.204:0/2138244315 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0960161620 con 0x7f096015ac90 2024-05-11T08:05:33.034 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.033+0000 7f0965476640 1 --2- 172.21.15.204:0/2138244315 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f096015ac90 0x7f09600b3e00 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f09500097a0 tx=0x7f095002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ef9d0ba4a0626d6 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:05:33.034 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.033+0000 7f0957fff640 1 -- 172.21.15.204:0/2138244315 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0950037070 con 0x7f096015ac90 2024-05-11T08:05:33.034 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.034+0000 7f0957fff640 1 -- 172.21.15.204:0/2138244315 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0950032440 con 0x7f096015ac90 2024-05-11T08:05:33.035 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.034+0000 7f0957fff640 1 -- 172.21.15.204:0/2138244315 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0950032810 con 0x7f096015ac90 2024-05-11T08:05:33.036 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.034+0000 7f09676da640 1 -- 172.21.15.204:0/2138244315 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f096015ac90 msgr2=0x7f09600b3e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:05:33.036 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.034+0000 7f09676da640 1 --2- 172.21.15.204:0/2138244315 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f096015ac90 0x7f09600b3e00 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f09500097a0 tx=0x7f095002a230 comp rx=0 tx=0).stop 2024-05-11T08:05:33.036 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.034+0000 7f09676da640 1 -- 172.21.15.204:0/2138244315 shutdown_connections 2024-05-11T08:05:33.036 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.034+0000 7f09676da640 1 --2- 172.21.15.204:0/2138244315 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f096015ac90 0x7f09600b3e00 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:33.036 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.034+0000 7f09676da640 1 -- 172.21.15.204:0/2138244315 >> 172.21.15.204:0/2138244315 conn(0x7f096006ca80 msgr2=0x7f09600d2fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:05:33.036 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.034+0000 7f09676da640 1 -- 172.21.15.204:0/2138244315 shutdown_connections 2024-05-11T08:05:33.036 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.034+0000 7f09676da640 1 -- 172.21.15.204:0/2138244315 wait complete. 2024-05-11T08:05:33.036 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.035+0000 7f09676da640 1 Processor -- start 2024-05-11T08:05:33.036 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.036+0000 7f09676da640 1 -- start start 2024-05-11T08:05:33.036 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-05-11T08:05:33.036+0000 7fa318e13040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-11T08:05:33.038 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.036+0000 7f09676da640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f09600b4340 0x7f09600711c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:05:33.039 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.036+0000 7f09676da640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f09600b5130 0x7f096015d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:05:33.039 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.036+0000 7f09676da640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f096015ac90 0x7f096015b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:05:33.039 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.036+0000 7f09676da640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f096015e8b0 con 0x7f09600b5130 2024-05-11T08:05:33.039 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.036+0000 7f09676da640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f09600cf440 con 0x7f09600b4340 2024-05-11T08:05:33.039 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.036+0000 7f09676da640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f09600cc080 con 0x7f096015ac90 2024-05-11T08:05:33.039 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.036+0000 7f0965476640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f09600b4340 0x7f09600711c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:05:33.039 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.036+0000 7f0965476640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f09600b4340 0x7f09600711c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:33518/0 (socket says 172.21.15.204:33518) 2024-05-11T08:05:33.039 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.036+0000 7f0965476640 1 -- 172.21.15.204:0/2267876910 learned_addr learned my addr 172.21.15.204:0/2267876910 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-05-11T08:05:33.039 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.036+0000 7f0964c75640 1 --2- 172.21.15.204:0/2267876910 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f09600b5130 0x7f096015d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:05:33.039 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.036+0000 7f0965c77640 1 --2- 172.21.15.204:0/2267876910 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f096015ac90 0x7f096015b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:05:33.039 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.037+0000 7f0965476640 1 -- 172.21.15.204:0/2267876910 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f096015ac90 msgr2=0x7f096015b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:05:33.039 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.037+0000 7f0965476640 1 --2- 172.21.15.204:0/2267876910 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f096015ac90 0x7f096015b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:33.039 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.037+0000 7f0965476640 1 -- 172.21.15.204:0/2267876910 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f09600b5130 msgr2=0x7f096015d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:05:33.039 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.037+0000 7f0965476640 1 --2- 172.21.15.204:0/2267876910 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f09600b5130 0x7f096015d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:33.040 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.037+0000 7f0965476640 1 -- 172.21.15.204:0/2267876910 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0960152340 con 0x7f09600b4340 2024-05-11T08:05:33.040 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.037+0000 7f0965476640 1 --2- 172.21.15.204:0/2267876910 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f09600b4340 0x7f09600711c0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f0950035040 tx=0x7f09500046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:05:33.040 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.037+0000 7f0955ffb640 1 -- 172.21.15.204:0/2267876910 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0950041030 con 0x7f09600b4340 2024-05-11T08:05:33.040 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.037+0000 7f0955ffb640 1 -- 172.21.15.204:0/2267876910 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f095003d4e0 con 0x7f09600b4340 2024-05-11T08:05:33.040 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.037+0000 7f09676da640 1 -- 172.21.15.204:0/2267876910 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0960152620 con 0x7f09600b4340 2024-05-11T08:05:33.040 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.037+0000 7f0955ffb640 1 -- 172.21.15.204:0/2267876910 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0950037040 con 0x7f09600b4340 2024-05-11T08:05:33.040 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.037+0000 7f09676da640 1 -- 172.21.15.204:0/2267876910 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f096015bd10 con 0x7f09600b4340 2024-05-11T08:05:33.040 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.037+0000 7f0955ffb640 1 -- 172.21.15.204:0/2267876910 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f0950044070 con 0x7f09600b4340 2024-05-11T08:05:33.040 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.037+0000 7f0955ffb640 1 -- 172.21.15.204:0/2267876910 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7f0950046be0 con 0x7f09600b4340 2024-05-11T08:05:33.040 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.038+0000 7f09676da640 1 -- 172.21.15.204:0/2267876910 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f096006e800 con 0x7f09600b4340 2024-05-11T08:05:33.040 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.040+0000 7f0955ffb640 1 -- 172.21.15.204:0/2267876910 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f096006e800 con 0x7f09600b4340 2024-05-11T08:05:33.079 INFO:tasks.ceph.mgr.y.smithi167.stderr:2024-05-11T08:05:33.079+0000 7f84d39b8040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-11T08:05:33.111 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.111+0000 7f09676da640 1 -- 172.21.15.204:0/2267876910 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd new", "uuid": "cb27250d-2a80-4c7b-8cef-3a7af05c8a16", "id": 4} v 0) -- 0x7f09600c9d30 con 0x7f09600b4340 2024-05-11T08:05:33.117 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.117+0000 7f0955ffb640 1 -- 172.21.15.204:0/2267876910 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "cb27250d-2a80-4c7b-8cef-3a7af05c8a16", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7f09600c9d30 con 0x7f09600b4340 2024-05-11T08:05:33.117 INFO:teuthology.orchestra.run.smithi204.stdout:4 2024-05-11T08:05:33.118 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.118+0000 7f09676da640 1 -- 172.21.15.204:0/2267876910 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f09600b4340 msgr2=0x7f09600711c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:05:33.118 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.118+0000 7f09676da640 1 --2- 172.21.15.204:0/2267876910 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f09600b4340 0x7f09600711c0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f0950035040 tx=0x7f09500046c0 comp rx=0 tx=0).stop 2024-05-11T08:05:33.118 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.118+0000 7f09676da640 1 -- 172.21.15.204:0/2267876910 shutdown_connections 2024-05-11T08:05:33.118 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.118+0000 7f09676da640 1 --2- 172.21.15.204:0/2267876910 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f096015ac90 0x7f096015b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:33.118 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.118+0000 7f09676da640 1 --2- 172.21.15.204:0/2267876910 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f09600b5130 0x7f096015d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:33.118 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.118+0000 7f09676da640 1 --2- 172.21.15.204:0/2267876910 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f09600b4340 0x7f09600711c0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:33.118 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.118+0000 7f09676da640 1 -- 172.21.15.204:0/2267876910 >> 172.21.15.204:0/2267876910 conn(0x7f096006ca80 msgr2=0x7f09600d2b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:05:33.119 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.118+0000 7f09676da640 1 -- 172.21.15.204:0/2267876910 shutdown_connections 2024-05-11T08:05:33.119 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.118+0000 7f09676da640 1 -- 172.21.15.204:0/2267876910 wait complete. 2024-05-11T08:05:33.121 INFO:tasks.ceph.mgr.y.smithi167.stderr:2024-05-11T08:05:33.122+0000 7f84d39b8040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-11T08:05:33.126 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-05-11T08:05:33.127+0000 7fa318e13040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-11T08:05:33.129 DEBUG:teuthology.orchestra.run.smithi204:> sudo ceph --cluster ceph osd new 92998541-ffec-430b-ab76-8c3cb7c2752d 5 2024-05-11T08:05:33.172 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-05-11T08:05:33.172+0000 7fa318e13040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-11T08:05:33.229 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.229+0000 7f83da045640 1 Processor -- start 2024-05-11T08:05:33.229 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.229+0000 7f83da045640 1 -- start start 2024-05-11T08:05:33.233 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.229+0000 7f83da045640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f83d415c4f0 0x7f83d40d1860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:05:33.233 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.230+0000 7f83da045640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f83d415cfe0 0x7f83d4165aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:05:33.233 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.230+0000 7f83da045640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83d40d0940 con 0x7f83d40cfe80 2024-05-11T08:05:33.233 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.230+0000 7f83da045640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f83d415c330 con 0x7f83d415cfe0 2024-05-11T08:05:33.233 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.230+0000 7f83da045640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f83d40d0660 con 0x7f83d415c4f0 2024-05-11T08:05:33.234 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.230+0000 7f83d37fe640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f83d415c4f0 0x7f83d40d1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:05:33.234 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.230+0000 7f83d37fe640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f83d415c4f0 0x7f83d40d1860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:33526/0 (socket says 172.21.15.204:33526) 2024-05-11T08:05:33.234 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.230+0000 7f83d37fe640 1 -- 172.21.15.204:0/1283714483 learned_addr learned my addr 172.21.15.204:0/1283714483 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-05-11T08:05:33.234 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.230+0000 7f83d3fff640 1 --2- 172.21.15.204:0/1283714483 >> v2:172.21.15.167:3300/0 conn(0x7f83d415cfe0 0x7f83d4165aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:05:33.234 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.230+0000 7f83d37fe640 1 -- 172.21.15.204:0/1283714483 >> v2:172.21.15.167:3300/0 conn(0x7f83d415cfe0 msgr2=0x7f83d4165aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:05:33.234 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.230+0000 7f83d37fe640 1 --2- 172.21.15.204:0/1283714483 >> v2:172.21.15.167:3300/0 conn(0x7f83d415cfe0 0x7f83d4165aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:33.234 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.230+0000 7f83d37fe640 1 -- 172.21.15.204:0/1283714483 >> v1:172.21.15.167:6789/0 conn(0x7f83d40cfe80 legacy=0x7f83d415c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:05:33.234 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.230+0000 7f83d27fc640 1 -- 172.21.15.204:0/1283714483 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1208458673 0 0) 0x7f83d40d0940 con 0x7f83d40cfe80 2024-05-11T08:05:33.234 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.230+0000 7f83d37fe640 1 -- 172.21.15.204:0/1283714483 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83d40b4f50 con 0x7f83d415c4f0 2024-05-11T08:05:33.234 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.231+0000 7f83d37fe640 1 --2- 172.21.15.204:0/1283714483 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f83d415c4f0 0x7f83d40d1860 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f83d40ceb30 tx=0x7f83c402a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c8c0161ad2d95191 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:05:33.234 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.231+0000 7f83d27fc640 1 -- 172.21.15.204:0/1283714483 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c4037070 con 0x7f83d415c4f0 2024-05-11T08:05:33.234 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.231+0000 7f83d27fc640 1 -- 172.21.15.204:0/1283714483 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83c4032440 con 0x7f83d415c4f0 2024-05-11T08:05:33.234 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.231+0000 7f83d27fc640 1 -- 172.21.15.204:0/1283714483 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c4032810 con 0x7f83d415c4f0 2024-05-11T08:05:33.237 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.231+0000 7f83da045640 1 -- 172.21.15.204:0/1283714483 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f83d415c4f0 msgr2=0x7f83d40d1860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:05:33.237 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.232+0000 7f83da045640 1 --2- 172.21.15.204:0/1283714483 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f83d415c4f0 0x7f83d40d1860 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f83d40ceb30 tx=0x7f83c402a270 comp rx=0 tx=0).stop 2024-05-11T08:05:33.237 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.232+0000 7f83da045640 1 -- 172.21.15.204:0/1283714483 shutdown_connections 2024-05-11T08:05:33.238 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.232+0000 7f83da045640 1 --2- 172.21.15.204:0/1283714483 >> v2:172.21.15.167:3300/0 conn(0x7f83d415cfe0 0x7f83d4165aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:33.238 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.232+0000 7f83da045640 1 --2- 172.21.15.204:0/1283714483 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f83d415c4f0 0x7f83d40d1860 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:33.238 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.232+0000 7f83da045640 1 -- 172.21.15.204:0/1283714483 >> 172.21.15.204:0/1283714483 conn(0x7f83d40d2000 msgr2=0x7f83d40d2410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:05:33.238 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.232+0000 7f83da045640 1 -- 172.21.15.204:0/1283714483 shutdown_connections 2024-05-11T08:05:33.238 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.232+0000 7f83da045640 1 -- 172.21.15.204:0/1283714483 wait complete. 2024-05-11T08:05:33.238 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.233+0000 7f83da045640 1 Processor -- start 2024-05-11T08:05:33.238 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.233+0000 7f83da045640 1 -- start start 2024-05-11T08:05:33.238 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.234+0000 7f83da045640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f83d40cfe80 0x7f83d41407a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:05:33.238 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.234+0000 7f83da045640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f83d415c4f0 0x7f83d4158d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:05:33.238 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.234+0000 7f83da045640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f83d415cfe0 0x7f83d40b1b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:05:33.238 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.234+0000 7f83da045640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f83d40d0310 con 0x7f83d40cfe80 2024-05-11T08:05:33.238 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.234+0000 7f83da045640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f83d40b5510 con 0x7f83d415cfe0 2024-05-11T08:05:33.238 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.234+0000 7f83da045640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f83d415d640 con 0x7f83d415c4f0 2024-05-11T08:05:33.238 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.234+0000 7f83d3fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f83d415cfe0 0x7f83d40b1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:05:33.239 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.234+0000 7f83d2ffd640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f83d415c4f0 0x7f83d4158d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:05:33.239 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.234+0000 7f83d37fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f83d40cfe80 0x7f83d41407a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:05:33.239 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.234+0000 7f83d3fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f83d415cfe0 0x7f83d40b1b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:33536/0 (socket says 172.21.15.204:33536) 2024-05-11T08:05:33.239 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.234+0000 7f83d3fff640 1 -- 172.21.15.204:0/524862719 learned_addr learned my addr 172.21.15.204:0/524862719 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-05-11T08:05:33.239 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.234+0000 7f83d3fff640 1 -- 172.21.15.204:0/524862719 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f83d415c4f0 msgr2=0x7f83d4158d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:05:33.239 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.234+0000 7f83d3fff640 1 --2- 172.21.15.204:0/524862719 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f83d415c4f0 0x7f83d4158d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:33.239 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.234+0000 7f83d3fff640 1 -- 172.21.15.204:0/524862719 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f83d40cfe80 msgr2=0x7f83d41407a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:05:33.239 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.234+0000 7f83d3fff640 1 --2- 172.21.15.204:0/524862719 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f83d40cfe80 0x7f83d41407a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:33.239 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.234+0000 7f83d3fff640 1 -- 172.21.15.204:0/524862719 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83d415aba0 con 0x7f83d415cfe0 2024-05-11T08:05:33.239 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.234+0000 7f83d3fff640 1 --2- 172.21.15.204:0/524862719 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f83d415cfe0 0x7f83d40b1b20 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f83cc00efc0 tx=0x7f83cc00c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:05:33.239 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.234+0000 7f83bbfff640 1 -- 172.21.15.204:0/524862719 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83cc018020 con 0x7f83d415cfe0 2024-05-11T08:05:33.239 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.234+0000 7f83bbfff640 1 -- 172.21.15.204:0/524862719 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83cc014480 con 0x7f83d415cfe0 2024-05-11T08:05:33.239 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.234+0000 7f83da045640 1 -- 172.21.15.204:0/524862719 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83d415ae80 con 0x7f83d415cfe0 2024-05-11T08:05:33.239 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.234+0000 7f83bbfff640 1 -- 172.21.15.204:0/524862719 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83cc014870 con 0x7f83d415cfe0 2024-05-11T08:05:33.239 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.235+0000 7f83da045640 1 -- 172.21.15.204:0/524862719 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83d415b440 con 0x7f83d415cfe0 2024-05-11T08:05:33.240 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.235+0000 7f83bbfff640 1 -- 172.21.15.204:0/524862719 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f83cc005d10 con 0x7f83d415cfe0 2024-05-11T08:05:33.240 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.235+0000 7f83bbfff640 1 -- 172.21.15.204:0/524862719 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7f83cc013960 con 0x7f83d415cfe0 2024-05-11T08:05:33.240 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.235+0000 7f83da045640 1 -- 172.21.15.204:0/524862719 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83d4069bc0 con 0x7f83d415cfe0 2024-05-11T08:05:33.240 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.239+0000 7f83bbfff640 1 -- 172.21.15.204:0/524862719 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f83cc022070 con 0x7f83d415cfe0 2024-05-11T08:05:33.285 INFO:tasks.ceph.mgr.y.smithi167.stderr:2024-05-11T08:05:33.285+0000 7f84d39b8040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-11T08:05:33.312 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.312+0000 7f83da045640 1 -- 172.21.15.204:0/524862719 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd new", "uuid": "92998541-ffec-430b-ab76-8c3cb7c2752d", "id": 5} v 0) -- 0x7f83d40b6290 con 0x7f83d415cfe0 2024-05-11T08:05:33.317 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.317+0000 7f83bbfff640 1 -- 172.21.15.204:0/524862719 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "92998541-ffec-430b-ab76-8c3cb7c2752d", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7f83cc010040 con 0x7f83d415cfe0 2024-05-11T08:05:33.317 INFO:teuthology.orchestra.run.smithi204.stdout:5 2024-05-11T08:05:33.318 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.318+0000 7f83da045640 1 -- 172.21.15.204:0/524862719 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f83d415cfe0 msgr2=0x7f83d40b1b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:05:33.318 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.318+0000 7f83da045640 1 --2- 172.21.15.204:0/524862719 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f83d415cfe0 0x7f83d40b1b20 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f83cc00efc0 tx=0x7f83cc00c490 comp rx=0 tx=0).stop 2024-05-11T08:05:33.318 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.318+0000 7f83da045640 1 -- 172.21.15.204:0/524862719 shutdown_connections 2024-05-11T08:05:33.319 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.318+0000 7f83da045640 1 --2- 172.21.15.204:0/524862719 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f83d415cfe0 0x7f83d40b1b20 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:33.319 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.318+0000 7f83da045640 1 --2- 172.21.15.204:0/524862719 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f83d415c4f0 0x7f83d4158d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:33.319 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.318+0000 7f83da045640 1 --2- 172.21.15.204:0/524862719 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f83d40cfe80 0x7f83d41407a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:33.319 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.318+0000 7f83da045640 1 -- 172.21.15.204:0/524862719 >> 172.21.15.204:0/524862719 conn(0x7f83d40d2000 msgr2=0x7f83d41e26b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:05:33.319 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.318+0000 7f83da045640 1 -- 172.21.15.204:0/524862719 shutdown_connections 2024-05-11T08:05:33.319 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.318+0000 7f83da045640 1 -- 172.21.15.204:0/524862719 wait complete. 2024-05-11T08:05:33.329 DEBUG:teuthology.orchestra.run.smithi204:> sudo ceph --cluster ceph osd new 9543479c-2ec5-42a8-8933-707bfed2e048 6 2024-05-11T08:05:33.332 INFO:tasks.ceph.mgr.y.smithi167.stderr:2024-05-11T08:05:33.333+0000 7f84d39b8040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-11T08:05:33.353 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-05-11T08:05:33.353+0000 7fa318e13040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-11T08:05:33.384 INFO:tasks.ceph.mgr.y.smithi167.stderr:2024-05-11T08:05:33.384+0000 7f84d39b8040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-11T08:05:33.415 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-05-11T08:05:33.415+0000 7fa318e13040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-11T08:05:33.434 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.434+0000 7f3156a9c640 1 Processor -- start 2024-05-11T08:05:33.434 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.434+0000 7f3156a9c640 1 -- start start 2024-05-11T08:05:33.435 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.435+0000 7f3156a9c640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f31500cf8a0 0x7f31500d07e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:05:33.436 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.435+0000 7f3156a9c640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31500b6410 con 0x7f31501458f0 2024-05-11T08:05:33.436 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.435+0000 7f3156a9c640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31500b66f0 con 0x7f31500cfd30 2024-05-11T08:05:33.436 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.435+0000 7f3156a9c640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f31500b7020 con 0x7f31500cf8a0 2024-05-11T08:05:33.436 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.435+0000 7f3155039640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f31500cfd30 0x7f315006ec10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.204:57974/0 (socket says 172.21.15.204:57974) 2024-05-11T08:05:33.436 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.435+0000 7f3155039640 1 -- 172.21.15.204:0/388235546 learned_addr learned my addr 172.21.15.204:0/388235546 (peer_addr_for_me v1:172.21.15.204:0/0) 2024-05-11T08:05:33.436 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.435+0000 7f314ffff640 1 --2- 172.21.15.204:0/388235546 >> v2:172.21.15.167:3300/0 conn(0x7f31500cf8a0 0x7f31500d07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:05:33.436 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.435+0000 7f314f7fe640 1 -- 172.21.15.204:0/388235546 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1611694123 0 0) 0x7f31500b66f0 con 0x7f31500cfd30 2024-05-11T08:05:33.436 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.435+0000 7f314f7fe640 1 -- 172.21.15.204:0/388235546 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3140003520 con 0x7f31500cfd30 2024-05-11T08:05:33.436 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.435+0000 7f314f7fe640 1 -- 172.21.15.204:0/388235546 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2055635892 0 0) 0x7f31500b6410 con 0x7f31501458f0 2024-05-11T08:05:33.436 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.435+0000 7f314f7fe640 1 -- 172.21.15.204:0/388235546 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f31500b66f0 con 0x7f31501458f0 2024-05-11T08:05:33.436 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.435+0000 7f314ffff640 1 -- 172.21.15.204:0/388235546 >> v1:172.21.15.204:6789/0 conn(0x7f31500cfd30 legacy=0x7f315006ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:05:33.436 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.435+0000 7f314ffff640 1 -- 172.21.15.204:0/388235546 >> v1:172.21.15.167:6789/0 conn(0x7f31501458f0 legacy=0x7f31500b74b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:05:33.436 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.435+0000 7f314ffff640 1 -- 172.21.15.204:0/388235546 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31500b69d0 con 0x7f31500cf8a0 2024-05-11T08:05:33.438 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.436+0000 7f314ffff640 1 --2- 172.21.15.204:0/388235546 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f31500cf8a0 0x7f31500d07e0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f31500b71a0 tx=0x7f313c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdfc463f02e15b47 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:05:33.438 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.436+0000 7f314f7fe640 1 -- 172.21.15.204:0/388235546 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f313c037070 con 0x7f31500cf8a0 2024-05-11T08:05:33.438 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.436+0000 7f314f7fe640 1 -- 172.21.15.204:0/388235546 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f313c032440 con 0x7f31500cf8a0 2024-05-11T08:05:33.438 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.436+0000 7f314f7fe640 1 -- 172.21.15.204:0/388235546 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f313c032810 con 0x7f31500cf8a0 2024-05-11T08:05:33.438 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.437+0000 7f3156a9c640 1 -- 172.21.15.204:0/388235546 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f31500cf8a0 msgr2=0x7f31500d07e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:05:33.438 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.437+0000 7f3156a9c640 1 --2- 172.21.15.204:0/388235546 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f31500cf8a0 0x7f31500d07e0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f31500b71a0 tx=0x7f313c02a180 comp rx=0 tx=0).stop 2024-05-11T08:05:33.438 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.437+0000 7f3156a9c640 1 -- 172.21.15.204:0/388235546 shutdown_connections 2024-05-11T08:05:33.438 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.437+0000 7f3156a9c640 1 --2- 172.21.15.204:0/388235546 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f31500cf8a0 0x7f31500d07e0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:33.438 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.437+0000 7f3156a9c640 1 -- 172.21.15.204:0/388235546 >> 172.21.15.204:0/388235546 conn(0x7f31500732c0 msgr2=0x7f31500736d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:05:33.438 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.437+0000 7f3156a9c640 1 -- 172.21.15.204:0/388235546 shutdown_connections 2024-05-11T08:05:33.438 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.437+0000 7f3156a9c640 1 -- 172.21.15.204:0/388235546 wait complete. 2024-05-11T08:05:33.441 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.438+0000 7f3156a9c640 1 Processor -- start 2024-05-11T08:05:33.441 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.438+0000 7f3156a9c640 1 -- start start 2024-05-11T08:05:33.441 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.439+0000 7f3156a9c640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f31500cf8a0 0x7f31501f3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:05:33.441 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.439+0000 7f3156a9c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f31500cfd30 0x7f315006cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:05:33.441 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.439+0000 7f3156a9c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f31501458f0 0x7f315013ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:05:33.441 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.439+0000 7f3156a9c640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f31500b5fc0 con 0x7f31501458f0 2024-05-11T08:05:33.441 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.439+0000 7f3156a9c640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f3150145cd0 con 0x7f31500cfd30 2024-05-11T08:05:33.441 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.439+0000 7f3156a9c640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f31500b6140 con 0x7f31500cf8a0 2024-05-11T08:05:33.441 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.439+0000 7f314ffff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f31500cfd30 0x7f315006cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:05:33.441 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.439+0000 7f314ffff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f31500cfd30 0x7f315006cf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:33550/0 (socket says 172.21.15.204:33550) 2024-05-11T08:05:33.442 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.439+0000 7f314ffff640 1 -- 172.21.15.204:0/54666 learned_addr learned my addr 172.21.15.204:0/54666 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-05-11T08:05:33.442 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.439+0000 7f3154838640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f31500cf8a0 0x7f31501f3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:05:33.442 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.439+0000 7f3155039640 1 --2- 172.21.15.204:0/54666 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f31501458f0 0x7f315013ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:05:33.442 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.439+0000 7f314ffff640 1 -- 172.21.15.204:0/54666 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f31500cf8a0 msgr2=0x7f31501f3910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:05:33.442 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.439+0000 7f314ffff640 1 --2- 172.21.15.204:0/54666 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f31500cf8a0 0x7f31501f3910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:33.442 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.439+0000 7f314ffff640 1 -- 172.21.15.204:0/54666 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f31501458f0 msgr2=0x7f315013ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:05:33.442 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.440+0000 7f314ffff640 1 --2- 172.21.15.204:0/54666 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f31501458f0 0x7f315013ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:33.442 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.440+0000 7f314ffff640 1 -- 172.21.15.204:0/54666 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31501f7330 con 0x7f31500cfd30 2024-05-11T08:05:33.442 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.440+0000 7f314ffff640 1 --2- 172.21.15.204:0/54666 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f31500cfd30 0x7f315006cf50 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f313c035040 tx=0x7f313c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:05:33.442 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.440+0000 7f314d7fa640 1 -- 172.21.15.204:0/54666 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f313c041030 con 0x7f31500cfd30 2024-05-11T08:05:33.442 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.440+0000 7f314d7fa640 1 -- 172.21.15.204:0/54666 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f313c03d4e0 con 0x7f31500cfd30 2024-05-11T08:05:33.442 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.440+0000 7f3156a9c640 1 -- 172.21.15.204:0/54666 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f315013f340 con 0x7f31500cfd30 2024-05-11T08:05:33.442 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.440+0000 7f314d7fa640 1 -- 172.21.15.204:0/54666 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f313c037040 con 0x7f31500cfd30 2024-05-11T08:05:33.443 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.440+0000 7f3156a9c640 1 -- 172.21.15.204:0/54666 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31501f6850 con 0x7f31500cfd30 2024-05-11T08:05:33.443 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.440+0000 7f314d7fa640 1 -- 172.21.15.204:0/54666 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f313c044070 con 0x7f31500cfd30 2024-05-11T08:05:33.443 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.440+0000 7f314d7fa640 1 -- 172.21.15.204:0/54666 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7f313c046c70 con 0x7f31500cfd30 2024-05-11T08:05:33.443 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.441+0000 7f3156a9c640 1 -- 172.21.15.204:0/54666 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3150069bc0 con 0x7f31500cfd30 2024-05-11T08:05:33.443 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.443+0000 7f314d7fa640 1 -- 172.21.15.204:0/54666 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f313c046680 con 0x7f31500cfd30 2024-05-11T08:05:33.471 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-05-11T08:05:33.471+0000 7fa318e13040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-11T08:05:33.496 INFO:tasks.ceph.mgr.y.smithi167.stderr:2024-05-11T08:05:33.497+0000 7f84d39b8040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-11T08:05:33.515 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.516+0000 7f3156a9c640 1 -- 172.21.15.204:0/54666 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd new", "uuid": "9543479c-2ec5-42a8-8933-707bfed2e048", "id": 6} v 0) -- 0x7f31500741b0 con 0x7f31500cfd30 2024-05-11T08:05:33.521 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.521+0000 7f314d7fa640 1 -- 172.21.15.204:0/54666 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "9543479c-2ec5-42a8-8933-707bfed2e048", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7f313c046860 con 0x7f31500cfd30 2024-05-11T08:05:33.521 INFO:teuthology.orchestra.run.smithi204.stdout:6 2024-05-11T08:05:33.522 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.522+0000 7f3156a9c640 1 -- 172.21.15.204:0/54666 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f31500cfd30 msgr2=0x7f315006cf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:05:33.522 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.522+0000 7f3156a9c640 1 --2- 172.21.15.204:0/54666 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f31500cfd30 0x7f315006cf50 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f313c035040 tx=0x7f313c0046c0 comp rx=0 tx=0).stop 2024-05-11T08:05:33.522 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.522+0000 7f3156a9c640 1 -- 172.21.15.204:0/54666 shutdown_connections 2024-05-11T08:05:33.523 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.522+0000 7f3156a9c640 1 --2- 172.21.15.204:0/54666 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f31501458f0 0x7f315013ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:33.523 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.522+0000 7f3156a9c640 1 --2- 172.21.15.204:0/54666 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f31500cfd30 0x7f315006cf50 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:33.523 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.522+0000 7f3156a9c640 1 --2- 172.21.15.204:0/54666 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f31500cf8a0 0x7f31501f3910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:33.523 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.522+0000 7f3156a9c640 1 -- 172.21.15.204:0/54666 >> 172.21.15.204:0/54666 conn(0x7f31500732c0 msgr2=0x7f31500b2910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:05:33.523 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.522+0000 7f3156a9c640 1 -- 172.21.15.204:0/54666 shutdown_connections 2024-05-11T08:05:33.523 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.522+0000 7f3156a9c640 1 -- 172.21.15.204:0/54666 wait complete. 2024-05-11T08:05:33.533 DEBUG:teuthology.orchestra.run.smithi204:> sudo ceph --cluster ceph osd new c45a5647-8358-409e-af91-c05dda65f0b7 7 2024-05-11T08:05:33.588 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-05-11T08:05:33.588+0000 7fa318e13040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-11T08:05:33.675 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.675+0000 7fbb43862640 1 Processor -- start 2024-05-11T08:05:33.675 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.675+0000 7fbb43862640 1 -- start start 2024-05-11T08:05:33.677 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.676+0000 7fbb43862640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbb3c073270 0x7fbb3c15d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:05:33.677 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.676+0000 7fbb43862640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fbb3c073650 0x7fbb3c06e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:05:33.677 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.676+0000 7fbb43862640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fbb3c0ce8f0 0x7fbb3c0ced80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:05:33.677 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.676+0000 7fbb43862640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7fbb3c160b00 con 0x7fbb3c0ce8f0 2024-05-11T08:05:33.678 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.676+0000 7fbb43862640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fbb3c15cce0 con 0x7fbb3c073650 2024-05-11T08:05:33.678 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.676+0000 7fbb43862640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fbb3c15c6d0 con 0x7fbb3c073270 2024-05-11T08:05:33.678 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.676+0000 7fbb415fe640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbb3c073270 0x7fbb3c15d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:05:33.678 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.676+0000 7fbb40dfd640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fbb3c073650 0x7fbb3c06e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:05:33.678 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.676+0000 7fbb415fe640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbb3c073270 0x7fbb3c15d170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:58378/0 (socket says 172.21.15.204:58378) 2024-05-11T08:05:33.678 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.676+0000 7fbb415fe640 1 -- 172.21.15.204:0/2307129228 learned_addr learned my addr 172.21.15.204:0/2307129228 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-05-11T08:05:33.678 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.676+0000 7fbb41dff640 1 --2- 172.21.15.204:0/2307129228 >> v2:172.21.15.167:3300/0 conn(0x7fbb3c0ce8f0 0x7fbb3c0ced80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:05:33.678 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.677+0000 7fbb415fe640 1 -- 172.21.15.204:0/2307129228 >> v2:172.21.15.204:3300/0 conn(0x7fbb3c073650 msgr2=0x7fbb3c06e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:05:33.678 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.677+0000 7fbb415fe640 1 --2- 172.21.15.204:0/2307129228 >> v2:172.21.15.204:3300/0 conn(0x7fbb3c073650 0x7fbb3c06e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:33.678 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.677+0000 7fbb415fe640 1 -- 172.21.15.204:0/2307129228 >> v2:172.21.15.167:3300/0 conn(0x7fbb3c0ce8f0 msgr2=0x7fbb3c0ced80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:05:33.678 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.677+0000 7fbb415fe640 1 --2- 172.21.15.204:0/2307129228 >> v2:172.21.15.167:3300/0 conn(0x7fbb3c0ce8f0 0x7fbb3c0ced80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:33.678 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.677+0000 7fbb415fe640 1 -- 172.21.15.204:0/2307129228 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb3c0700b0 con 0x7fbb3c073270 2024-05-11T08:05:33.678 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.677+0000 7fbb40dfd640 1 --2- 172.21.15.204:0/2307129228 >> v2:172.21.15.204:3300/0 conn(0x7fbb3c073650 0x7fbb3c06e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T08:05:33.678 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.677+0000 7fbb41dff640 1 --2- 172.21.15.204:0/2307129228 >> v2:172.21.15.167:3300/0 conn(0x7fbb3c0ce8f0 0x7fbb3c0ced80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T08:05:33.679 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.677+0000 7fbb415fe640 1 --2- 172.21.15.204:0/2307129228 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbb3c073270 0x7fbb3c15d170 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c0d0c80 tx=0x7fbb3802a1e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a8cc78c7ecebfc26 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:05:33.679 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.677+0000 7fbb33fff640 1 -- 172.21.15.204:0/2307129228 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb38037070 con 0x7fbb3c073270 2024-05-11T08:05:33.679 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.677+0000 7fbb33fff640 1 -- 172.21.15.204:0/2307129228 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb38032440 con 0x7fbb3c073270 2024-05-11T08:05:33.679 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.677+0000 7fbb33fff640 1 -- 172.21.15.204:0/2307129228 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb38032810 con 0x7fbb3c073270 2024-05-11T08:05:33.681 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.678+0000 7fbb43862640 1 -- 172.21.15.204:0/2307129228 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbb3c073270 msgr2=0x7fbb3c15d170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:05:33.682 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.678+0000 7fbb43862640 1 --2- 172.21.15.204:0/2307129228 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbb3c073270 0x7fbb3c15d170 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c0d0c80 tx=0x7fbb3802a1e0 comp rx=0 tx=0).stop 2024-05-11T08:05:33.682 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.678+0000 7fbb43862640 1 -- 172.21.15.204:0/2307129228 shutdown_connections 2024-05-11T08:05:33.682 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.678+0000 7fbb43862640 1 --2- 172.21.15.204:0/2307129228 >> v2:172.21.15.167:3300/0 conn(0x7fbb3c0ce8f0 0x7fbb3c0ced80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:33.682 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.678+0000 7fbb43862640 1 --2- 172.21.15.204:0/2307129228 >> v2:172.21.15.204:3300/0 conn(0x7fbb3c073650 0x7fbb3c06e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:33.682 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.678+0000 7fbb43862640 1 --2- 172.21.15.204:0/2307129228 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbb3c073270 0x7fbb3c15d170 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:33.682 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.678+0000 7fbb43862640 1 -- 172.21.15.204:0/2307129228 >> 172.21.15.204:0/2307129228 conn(0x7fbb3c06c450 msgr2=0x7fbb3c06c860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:05:33.682 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.678+0000 7fbb43862640 1 -- 172.21.15.204:0/2307129228 shutdown_connections 2024-05-11T08:05:33.682 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.678+0000 7fbb43862640 1 -- 172.21.15.204:0/2307129228 wait complete. 2024-05-11T08:05:33.682 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.679+0000 7fbb43862640 1 Processor -- start 2024-05-11T08:05:33.682 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.679+0000 7fbb43862640 1 -- start start 2024-05-11T08:05:33.682 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.680+0000 7fbb43862640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbb3c073270 0x7fbb3c1f5c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:05:33.682 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.680+0000 7fbb43862640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbb3c073650 0x7fbb3c1f3130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:05:33.683 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.680+0000 7fbb43862640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbb3c0ce8f0 0x7fbb3c0ae820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:05:33.683 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.680+0000 7fbb43862640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fbb3c06fd60 con 0x7fbb3c073270 2024-05-11T08:05:33.683 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.680+0000 7fbb43862640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fbb3c070390 con 0x7fbb3c0ce8f0 2024-05-11T08:05:33.683 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.680+0000 7fbb43862640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fbb3c073ad0 con 0x7fbb3c073650 2024-05-11T08:05:33.683 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.680+0000 7fbb41dff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbb3c0ce8f0 0x7fbb3c0ae820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:05:33.683 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.680+0000 7fbb40dfd640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbb3c073650 0x7fbb3c1f3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:05:33.683 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.680+0000 7fbb40dfd640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbb3c073650 0x7fbb3c1f3130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:58386/0 (socket says 172.21.15.204:58386) 2024-05-11T08:05:33.683 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.680+0000 7fbb41dff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbb3c0ce8f0 0x7fbb3c0ae820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:33566/0 (socket says 172.21.15.204:33566) 2024-05-11T08:05:33.683 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.680+0000 7fbb40dfd640 1 -- 172.21.15.204:0/1194277899 learned_addr learned my addr 172.21.15.204:0/1194277899 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-05-11T08:05:33.683 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.680+0000 7fbb415fe640 1 --2- 172.21.15.204:0/1194277899 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbb3c073270 0x7fbb3c1f5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:05:33.683 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.681+0000 7fbb40dfd640 1 -- 172.21.15.204:0/1194277899 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbb3c0ce8f0 msgr2=0x7fbb3c0ae820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:05:33.683 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.681+0000 7fbb40dfd640 1 --2- 172.21.15.204:0/1194277899 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbb3c0ce8f0 0x7fbb3c0ae820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:33.683 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.681+0000 7fbb40dfd640 1 -- 172.21.15.204:0/1194277899 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbb3c073270 msgr2=0x7fbb3c1f5c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:05:33.683 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.681+0000 7fbb40dfd640 1 --2- 172.21.15.204:0/1194277899 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbb3c073270 0x7fbb3c1f5c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:33.683 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.681+0000 7fbb40dfd640 1 -- 172.21.15.204:0/1194277899 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb3c1f3670 con 0x7fbb3c073650 2024-05-11T08:05:33.684 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.681+0000 7fbb40dfd640 1 --2- 172.21.15.204:0/1194277899 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbb3c073650 0x7fbb3c1f3130 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c00e980 tx=0x7fbb2c00ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:05:33.684 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.681+0000 7fbb31ffb640 1 -- 172.21.15.204:0/1194277899 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb2c017070 con 0x7fbb3c073650 2024-05-11T08:05:33.684 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.681+0000 7fbb31ffb640 1 -- 172.21.15.204:0/1194277899 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb2c00ca00 con 0x7fbb3c073650 2024-05-11T08:05:33.684 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.681+0000 7fbb43862640 1 -- 172.21.15.204:0/1194277899 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb3c0af070 con 0x7fbb3c073650 2024-05-11T08:05:33.684 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.681+0000 7fbb31ffb640 1 -- 172.21.15.204:0/1194277899 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb2c00cdd0 con 0x7fbb3c073650 2024-05-11T08:05:33.684 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.681+0000 7fbb43862640 1 -- 172.21.15.204:0/1194277899 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb3c0af630 con 0x7fbb3c073650 2024-05-11T08:05:33.684 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.681+0000 7fbb31ffb640 1 -- 172.21.15.204:0/1194277899 <== mon.2 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fbb2c01a070 con 0x7fbb3c073650 2024-05-11T08:05:33.684 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.681+0000 7fbb31ffb640 1 -- 172.21.15.204:0/1194277899 <== mon.2 v2:172.21.15.204:3301/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7fbb2c013ad0 con 0x7fbb3c073650 2024-05-11T08:05:33.685 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.682+0000 7fbb43862640 1 -- 172.21.15.204:0/1194277899 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb3c069520 con 0x7fbb3c073650 2024-05-11T08:05:33.685 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.684+0000 7fbb31ffb640 1 -- 172.21.15.204:0/1194277899 <== mon.2 v2:172.21.15.204:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fbb2c013df0 con 0x7fbb3c073650 2024-05-11T08:05:33.755 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.755+0000 7fbb43862640 1 -- 172.21.15.204:0/1194277899 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd new", "uuid": "c45a5647-8358-409e-af91-c05dda65f0b7", "id": 7} v 0) -- 0x7fbb3c0d0820 con 0x7fbb3c073650 2024-05-11T08:05:33.763 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.763+0000 7fbb31ffb640 1 -- 172.21.15.204:0/1194277899 <== mon.2 v2:172.21.15.204:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "c45a5647-8358-409e-af91-c05dda65f0b7", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7fbb2c0064d0 con 0x7fbb3c073650 2024-05-11T08:05:33.763 INFO:teuthology.orchestra.run.smithi204.stdout:7 2024-05-11T08:05:33.764 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.764+0000 7fbb43862640 1 -- 172.21.15.204:0/1194277899 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbb3c073650 msgr2=0x7fbb3c1f3130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:05:33.764 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.764+0000 7fbb43862640 1 --2- 172.21.15.204:0/1194277899 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbb3c073650 0x7fbb3c1f3130 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c00e980 tx=0x7fbb2c00ee40 comp rx=0 tx=0).stop 2024-05-11T08:05:33.764 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.764+0000 7fbb43862640 1 -- 172.21.15.204:0/1194277899 shutdown_connections 2024-05-11T08:05:33.764 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.764+0000 7fbb43862640 1 --2- 172.21.15.204:0/1194277899 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbb3c0ce8f0 0x7fbb3c0ae820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:33.764 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.764+0000 7fbb43862640 1 --2- 172.21.15.204:0/1194277899 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbb3c073650 0x7fbb3c1f3130 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:33.765 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.764+0000 7fbb43862640 1 --2- 172.21.15.204:0/1194277899 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbb3c073270 0x7fbb3c1f5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:33.765 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.764+0000 7fbb43862640 1 -- 172.21.15.204:0/1194277899 >> 172.21.15.204:0/1194277899 conn(0x7fbb3c06c450 msgr2=0x7fbb3c144d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:05:33.765 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.764+0000 7fbb43862640 1 -- 172.21.15.204:0/1194277899 shutdown_connections 2024-05-11T08:05:33.765 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:05:33.764+0000 7fbb43862640 1 -- 172.21.15.204:0/1194277899 wait complete. 2024-05-11T08:05:33.775 INFO:tasks.ceph.osd.0:Restarting daemon 2024-05-11T08:05:33.775 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-05-11T08:05:33.777 INFO:tasks.ceph.osd.0:Started 2024-05-11T08:05:33.777 INFO:tasks.ceph.osd.1:Restarting daemon 2024-05-11T08:05:33.777 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-05-11T08:05:33.779 INFO:tasks.ceph.osd.1:Started 2024-05-11T08:05:33.779 INFO:tasks.ceph.osd.2:Restarting daemon 2024-05-11T08:05:33.780 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-05-11T08:05:33.781 INFO:tasks.ceph.osd.2:Started 2024-05-11T08:05:33.782 INFO:tasks.ceph.osd.3:Restarting daemon 2024-05-11T08:05:33.782 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-05-11T08:05:33.784 INFO:tasks.ceph.osd.3:Started 2024-05-11T08:05:33.784 INFO:tasks.ceph.osd.4:Restarting daemon 2024-05-11T08:05:33.784 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-05-11T08:05:33.817 INFO:tasks.ceph.osd.4:Started 2024-05-11T08:05:33.818 INFO:tasks.ceph.osd.5:Restarting daemon 2024-05-11T08:05:33.818 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-05-11T08:05:33.820 INFO:tasks.ceph.osd.5:Started 2024-05-11T08:05:33.820 INFO:tasks.ceph.osd.6:Restarting daemon 2024-05-11T08:05:33.820 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-05-11T08:05:33.822 INFO:tasks.ceph.osd.6:Started 2024-05-11T08:05:33.822 INFO:tasks.ceph.osd.7:Restarting daemon 2024-05-11T08:05:33.823 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-05-11T08:05:33.825 INFO:tasks.ceph.osd.7:Started 2024-05-11T08:05:33.825 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-11T08:05:33.915 INFO:tasks.ceph.mgr.y.smithi167.stderr:2024-05-11T08:05:33.915+0000 7f84d39b8040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-11T08:05:33.937 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:33.937+0000 7f6af9247640 1 Processor -- start 2024-05-11T08:05:33.938 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:33.937+0000 7f6af9247640 1 -- start start 2024-05-11T08:05:33.940 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:33.939+0000 7f6af9247640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6af40d2cf0 0x7f6af40cafb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:05:33.940 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:33.939+0000 7f6af9247640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f6af40cb4f0 0x7f6af406dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:05:33.940 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:33.939+0000 7f6af9247640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6af40d1190 con 0x7f6af41598d0 2024-05-11T08:05:33.940 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:33.939+0000 7f6af9247640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f6af40d3150 con 0x7f6af40cb4f0 2024-05-11T08:05:33.940 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:33.939+0000 7f6af9247640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f6af40cb940 con 0x7f6af40d2cf0 2024-05-11T08:05:33.940 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:33.939+0000 7f6af2d9d640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f6af41598d0 0x7f6af40d0640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:45618/0 (socket says 172.21.15.167:45618) 2024-05-11T08:05:33.940 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:33.939+0000 7f6af2d9d640 1 -- 172.21.15.167:0/1003363441 learned_addr learned my addr 172.21.15.167:0/1003363441 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T08:05:33.940 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:33.939+0000 7f6af259c640 1 --2- 172.21.15.167:0/1003363441 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6af40d2cf0 0x7f6af40cafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:05:33.940 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:33.939+0000 7f6af359e640 1 --2- 172.21.15.167:0/1003363441 >> v2:172.21.15.204:3300/0 conn(0x7f6af40cb4f0 0x7f6af406dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:05:33.940 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:33.939+0000 7f6af1d9b640 1 -- 172.21.15.167:0/1003363441 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 717929208 0 0) 0x7f6af40d1190 con 0x7f6af41598d0 2024-05-11T08:05:33.940 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:33.939+0000 7f6af1d9b640 1 -- 172.21.15.167:0/1003363441 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6adc003520 con 0x7f6af41598d0 2024-05-11T08:05:33.940 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:33.939+0000 7f6af259c640 1 -- 172.21.15.167:0/1003363441 >> v2:172.21.15.204:3300/0 conn(0x7f6af40cb4f0 msgr2=0x7f6af406dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:05:33.940 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:33.939+0000 7f6af259c640 1 --2- 172.21.15.167:0/1003363441 >> v2:172.21.15.204:3300/0 conn(0x7f6af40cb4f0 0x7f6af406dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:33.941 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:33.939+0000 7f6af259c640 1 -- 172.21.15.167:0/1003363441 >> v1:172.21.15.167:6789/0 conn(0x7f6af41598d0 legacy=0x7f6af40d0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:05:33.941 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:33.939+0000 7f6af259c640 1 -- 172.21.15.167:0/1003363441 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6af406d500 con 0x7f6af40d2cf0 2024-05-11T08:05:33.942 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:33.940+0000 7f6af259c640 1 --2- 172.21.15.167:0/1003363441 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6af40d2cf0 0x7f6af40cafb0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f6ae80097a0 tx=0x7f6ae802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=92a2024a4a032afd server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:05:33.942 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:33.940+0000 7f6af1d9b640 1 -- 172.21.15.167:0/1003363441 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae8037070 con 0x7f6af40d2cf0 2024-05-11T08:05:33.943 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:33.940+0000 7f6af1d9b640 1 -- 172.21.15.167:0/1003363441 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ae8032440 con 0x7f6af40d2cf0 2024-05-11T08:05:33.943 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:33.940+0000 7f6af1d9b640 1 -- 172.21.15.167:0/1003363441 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae8032810 con 0x7f6af40d2cf0 2024-05-11T08:05:33.943 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:33.941+0000 7f6af9247640 1 -- 172.21.15.167:0/1003363441 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6af40d2cf0 msgr2=0x7f6af40cafb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:05:33.943 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:33.941+0000 7f6af9247640 1 --2- 172.21.15.167:0/1003363441 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6af40d2cf0 0x7f6af40cafb0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f6ae80097a0 tx=0x7f6ae802a180 comp rx=0 tx=0).stop 2024-05-11T08:05:33.943 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:33.941+0000 7f6af9247640 1 -- 172.21.15.167:0/1003363441 shutdown_connections 2024-05-11T08:05:33.943 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:33.941+0000 7f6af9247640 1 --2- 172.21.15.167:0/1003363441 >> v2:172.21.15.204:3300/0 conn(0x7f6af40cb4f0 0x7f6af406dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:33.943 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:33.941+0000 7f6af9247640 1 --2- 172.21.15.167:0/1003363441 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6af40d2cf0 0x7f6af40cafb0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:33.943 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:33.941+0000 7f6af9247640 1 -- 172.21.15.167:0/1003363441 >> 172.21.15.167:0/1003363441 conn(0x7f6af40d1ba0 msgr2=0x7f6af40d1fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:05:33.943 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:33.941+0000 7f6af9247640 1 -- 172.21.15.167:0/1003363441 shutdown_connections 2024-05-11T08:05:33.943 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:33.941+0000 7f6af9247640 1 -- 172.21.15.167:0/1003363441 wait complete. 2024-05-11T08:05:33.944 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:33.943+0000 7f6af9247640 1 Processor -- start 2024-05-11T08:05:33.944 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:33.944+0000 7f6af9247640 1 -- start start 2024-05-11T08:05:33.946 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:33.945+0000 7f6af9247640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6af40cb4f0 0x7f6af40ced50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:05:33.946 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:33.945+0000 7f6af9247640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6af40d2cf0 0x7f6af40b87c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:05:33.946 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:33.945+0000 7f6af9247640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6af41598d0 0x7f6af414d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:05:33.946 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:33.945+0000 7f6af9247640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6af41607c0 con 0x7f6af40cb4f0 2024-05-11T08:05:33.946 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:33.945+0000 7f6af9247640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6af406dac0 con 0x7f6af41598d0 2024-05-11T08:05:33.946 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:33.945+0000 7f6af9247640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f6af4159cb0 con 0x7f6af40d2cf0 2024-05-11T08:05:33.946 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:33.945+0000 7f6af2d9d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6af40cb4f0 0x7f6af40ced50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:05:33.946 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:33.945+0000 7f6af2d9d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6af40cb4f0 0x7f6af40ced50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:44162/0 (socket says 172.21.15.167:44162) 2024-05-11T08:05:33.947 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:33.945+0000 7f6af2d9d640 1 -- 172.21.15.167:0/3397083520 learned_addr learned my addr 172.21.15.167:0/3397083520 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:05:33.947 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:33.945+0000 7f6af259c640 1 --2- 172.21.15.167:0/3397083520 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6af40d2cf0 0x7f6af40b87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:05:33.947 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:33.945+0000 7f6af359e640 1 --2- 172.21.15.167:0/3397083520 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6af41598d0 0x7f6af414d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:05:33.947 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:33.945+0000 7f6af2d9d640 1 -- 172.21.15.167:0/3397083520 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6af40d2cf0 msgr2=0x7f6af40b87c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:05:33.947 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:33.945+0000 7f6af2d9d640 1 --2- 172.21.15.167:0/3397083520 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6af40d2cf0 0x7f6af40b87c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:33.947 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:33.945+0000 7f6af2d9d640 1 -- 172.21.15.167:0/3397083520 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6af41598d0 msgr2=0x7f6af414d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:05:33.947 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:33.945+0000 7f6af2d9d640 1 --2- 172.21.15.167:0/3397083520 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6af41598d0 0x7f6af414d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:33.947 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:33.945+0000 7f6af2d9d640 1 -- 172.21.15.167:0/3397083520 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6af40b8ea0 con 0x7f6af40cb4f0 2024-05-11T08:05:33.947 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:33.945+0000 7f6af2d9d640 1 --2- 172.21.15.167:0/3397083520 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6af40cb4f0 0x7f6af40ced50 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f6ae400c410 tx=0x7f6ae400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:05:33.947 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:33.946+0000 7f6adb7fe640 1 -- 172.21.15.167:0/3397083520 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae4017070 con 0x7f6af40cb4f0 2024-05-11T08:05:33.947 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:33.946+0000 7f6adb7fe640 1 -- 172.21.15.167:0/3397083520 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ae4005ce0 con 0x7f6af40cb4f0 2024-05-11T08:05:33.947 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:33.946+0000 7f6adb7fe640 1 -- 172.21.15.167:0/3397083520 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae40072d0 con 0x7f6af40cb4f0 2024-05-11T08:05:33.948 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:33.947+0000 7f6af9247640 1 -- 172.21.15.167:0/3397083520 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6af40b9a90 con 0x7f6af40cb4f0 2024-05-11T08:05:33.948 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:33.947+0000 7f6af9247640 1 -- 172.21.15.167:0/3397083520 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6af40ba050 con 0x7f6af40cb4f0 2024-05-11T08:05:33.948 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:33.947+0000 7f6adb7fe640 1 -- 172.21.15.167:0/3397083520 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f6ae401a070 con 0x7f6af40cb4f0 2024-05-11T08:05:33.948 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:33.947+0000 7f6adb7fe640 1 -- 172.21.15.167:0/3397083520 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f6ae401cbc0 con 0x7f6af40cb4f0 2024-05-11T08:05:33.948 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:33.948+0000 7f6af9247640 1 -- 172.21.15.167:0/3397083520 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6af4069590 con 0x7f6af40cb4f0 2024-05-11T08:05:33.950 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:33.950+0000 7f6adb7fe640 1 -- 172.21.15.167:0/3397083520 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f6ae4007470 con 0x7f6af40cb4f0 2024-05-11T08:05:34.010 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-05-11T08:05:34.010+0000 7fa318e13040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-11T08:05:34.011 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:34.011+0000 7f6af9247640 1 -- 172.21.15.167:0/3397083520 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6af4157350 con 0x7f6af40cb4f0 2024-05-11T08:05:34.011 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:34.012+0000 7f6adb7fe640 1 -- 172.21.15.167:0/3397083520 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f6ae4007650 con 0x7f6af40cb4f0 2024-05-11T08:05:34.012 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T08:05:34.012 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":10,"fsid":"2034778d-8ff1-4823-aa44-ded32b518241","created":"2024-05-11T08:05:31.612421+0000","modified":"2024-05-11T08:05:33.757311+0000","last_up_change":"0.000000","last_in_change":"2024-05-11T08:05:33.757311+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"587eaced-5ded-406d-884b-30c1caa8686b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"37c59104-2dd0-4cf9-85ea-56636f6b02a1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"2bb4f7f5-c698-451b-b4bb-78e716921291","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"9dd26759-a338-49d2-8103-45d068b9c000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"cb27250d-2a80-4c7b-8cef-3a7af05c8a16","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"92998541-ffec-430b-ab76-8c3cb7c2752d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"9543479c-2ec5-42a8-8933-707bfed2e048","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"c45a5647-8358-409e-af91-c05dda65f0b7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-11T08:05:34.013 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:34.012+0000 7f6af9247640 1 -- 172.21.15.167:0/3397083520 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6af40cb4f0 msgr2=0x7f6af40ced50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:05:34.013 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:34.012+0000 7f6af9247640 1 --2- 172.21.15.167:0/3397083520 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6af40cb4f0 0x7f6af40ced50 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f6ae400c410 tx=0x7f6ae400c8d0 comp rx=0 tx=0).stop 2024-05-11T08:05:34.013 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:34.012+0000 7f6af9247640 1 -- 172.21.15.167:0/3397083520 shutdown_connections 2024-05-11T08:05:34.013 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:34.012+0000 7f6af9247640 1 --2- 172.21.15.167:0/3397083520 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6af41598d0 0x7f6af414d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:34.013 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:34.012+0000 7f6af9247640 1 --2- 172.21.15.167:0/3397083520 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6af40d2cf0 0x7f6af40b87c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:34.013 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:34.012+0000 7f6af9247640 1 --2- 172.21.15.167:0/3397083520 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6af40cb4f0 0x7f6af40ced50 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:34.013 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:34.012+0000 7f6af9247640 1 -- 172.21.15.167:0/3397083520 >> 172.21.15.167:0/3397083520 conn(0x7f6af40d1ba0 msgr2=0x7f6af40d3e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:05:34.013 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:34.013+0000 7f6af9247640 1 -- 172.21.15.167:0/3397083520 shutdown_connections 2024-05-11T08:05:34.013 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:34.013+0000 7f6af9247640 1 -- 172.21.15.167:0/3397083520 wait complete. 2024-05-11T08:05:34.024 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-05-11T08:05:34.024 INFO:tasks.ceph:Waiting for OSDs to come up 2024-05-11T08:05:34.063 INFO:tasks.ceph.mgr.y.smithi167.stderr:2024-05-11T08:05:34.063+0000 7f84d39b8040 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-11T08:05:34.148 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-05-11T08:05:34.148+0000 7fa318e13040 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-11T08:05:34.322 INFO:tasks.ceph.mgr.y.smithi167.stderr:2024-05-11T08:05:34.322+0000 7f84d39b8040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-11T08:05:34.374 INFO:tasks.ceph.mgr.y.smithi167.stderr:2024-05-11T08:05:34.374+0000 7f84d39b8040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-11T08:05:34.396 INFO:tasks.ceph.osd.2.smithi167.stderr:2024-05-11T08:05:34.396+0000 7f81d7b68640 -1 Falling back to public interface 2024-05-11T08:05:34.396 INFO:tasks.ceph.osd.0.smithi167.stderr:2024-05-11T08:05:34.396+0000 7f1d57a15640 -1 Falling back to public interface 2024-05-11T08:05:34.397 INFO:tasks.ceph.osd.2.smithi167.stderr:2024-05-11T08:05:34.397+0000 7f81d6b17640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6804/0: (98) Address already in use 2024-05-11T08:05:34.401 INFO:tasks.ceph.osd.1.smithi167.stderr:2024-05-11T08:05:34.401+0000 7fd83fda7640 -1 Falling back to public interface 2024-05-11T08:05:34.405 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-05-11T08:05:34.405+0000 7fa318e13040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-11T08:05:34.406 INFO:tasks.ceph.osd.3.smithi167.stderr:2024-05-11T08:05:34.406+0000 7f8f8e544640 -1 Falling back to public interface 2024-05-11T08:05:34.427 INFO:tasks.ceph.osd.4.smithi204.stderr:2024-05-11T08:05:34.427+0000 7f1577418640 -1 Falling back to public interface 2024-05-11T08:05:34.431 INFO:tasks.ceph.osd.7.smithi204.stderr:2024-05-11T08:05:34.431+0000 7fb957018640 -1 Falling back to public interface 2024-05-11T08:05:34.441 INFO:tasks.ceph.osd.6.smithi204.stderr:2024-05-11T08:05:34.441+0000 7f995e133640 -1 Falling back to public interface 2024-05-11T08:05:34.452 INFO:tasks.ceph.osd.5.smithi204.stderr:2024-05-11T08:05:34.453+0000 7eff1112a640 -1 Falling back to public interface 2024-05-11T08:05:34.462 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-05-11T08:05:34.463+0000 7fa318e13040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-11T08:05:34.496 INFO:tasks.ceph.mgr.y.smithi167.stderr:2024-05-11T08:05:34.496+0000 7f84d39b8040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-11T08:05:34.577 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-05-11T08:05:34.578+0000 7fa318e13040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-11T08:05:34.793 INFO:tasks.ceph.mgr.y.smithi167.stderr:2024-05-11T08:05:34.793+0000 7f84d39b8040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-11T08:05:34.829 DEBUG:teuthology.orchestra.run.smithi167:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-05-11T08:05:34.842 INFO:tasks.ceph.mgr.y.smithi167.stderr:2024-05-11T08:05:34.843+0000 7f84d39b8040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-11T08:05:34.877 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-05-11T08:05:34.877+0000 7fa318e13040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-11T08:05:34.922 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:34.922+0000 7f68bdc54640 1 Processor -- start 2024-05-11T08:05:34.922 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:34.922+0000 7f68bdc54640 1 -- start start 2024-05-11T08:05:34.924 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:34.923+0000 7f68bdc54640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f68b815f450 0x7f68b80d01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:05:34.924 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:34.923+0000 7f68bdc54640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f68b80d3ca0 0x7f68b80d40f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:05:34.924 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:34.923+0000 7f68bdc54640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f68b80cf710 con 0x7f68b80b4620 2024-05-11T08:05:34.924 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:34.923+0000 7f68bdc54640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f68b80d4630 con 0x7f68b80d3ca0 2024-05-11T08:05:34.924 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:34.923+0000 7f68bdc54640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f68b80d06f0 con 0x7f68b815f450 2024-05-11T08:05:34.924 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:34.923+0000 7f68b7fff640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f68b80d3ca0 0x7f68b80d40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:05:34.924 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:34.923+0000 7f68b7fff640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f68b80d3ca0 0x7f68b80d40f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:52144/0 (socket says 172.21.15.167:52144) 2024-05-11T08:05:34.924 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:34.923+0000 7f68b7fff640 1 -- 172.21.15.167:0/1410574079 learned_addr learned my addr 172.21.15.167:0/1410574079 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:05:34.924 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:34.923+0000 7f68b77fe640 1 --2- 172.21.15.167:0/1410574079 >> v2:172.21.15.204:3300/0 conn(0x7f68b815f450 0x7f68b80d01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:05:34.924 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:34.923+0000 7f68b7fff640 1 -- 172.21.15.167:0/1410574079 >> v2:172.21.15.204:3300/0 conn(0x7f68b815f450 msgr2=0x7f68b80d01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:05:34.924 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:34.923+0000 7f68b7fff640 1 --2- 172.21.15.167:0/1410574079 >> v2:172.21.15.204:3300/0 conn(0x7f68b815f450 0x7f68b80d01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:34.925 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:34.923+0000 7f68b7fff640 1 -- 172.21.15.167:0/1410574079 >> v1:172.21.15.204:6789/0 conn(0x7f68b80b4620 legacy=0x7f68b806d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:05:34.925 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:34.923+0000 7f68b7fff640 1 -- 172.21.15.167:0/1410574079 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68b806dbe0 con 0x7f68b80d3ca0 2024-05-11T08:05:34.925 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:34.924+0000 7f68b7fff640 1 --2- 172.21.15.167:0/1410574079 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f68b80d3ca0 0x7f68b80d40f0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f68b80744c0 tx=0x7f68b002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2ed08bd6400f889 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:05:34.925 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:34.924+0000 7f68b67fc640 1 -- 172.21.15.167:0/1410574079 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68b0036070 con 0x7f68b80d3ca0 2024-05-11T08:05:34.925 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:34.924+0000 7f68b67fc640 1 -- 172.21.15.167:0/1410574079 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68b0031600 con 0x7f68b80d3ca0 2024-05-11T08:05:34.925 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:34.924+0000 7f68b67fc640 1 -- 172.21.15.167:0/1410574079 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68b00319d0 con 0x7f68b80d3ca0 2024-05-11T08:05:34.925 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-05-11T08:05:34.924+0000 7fa318e13040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-11T08:05:34.928 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:34.924+0000 7f68bdc54640 1 -- 172.21.15.167:0/1410574079 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f68b80d3ca0 msgr2=0x7f68b80d40f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:05:34.928 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:34.924+0000 7f68bdc54640 1 --2- 172.21.15.167:0/1410574079 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f68b80d3ca0 0x7f68b80d40f0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f68b80744c0 tx=0x7f68b002a6a0 comp rx=0 tx=0).stop 2024-05-11T08:05:34.928 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:34.925+0000 7f68bdc54640 1 -- 172.21.15.167:0/1410574079 shutdown_connections 2024-05-11T08:05:34.929 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:34.925+0000 7f68bdc54640 1 --2- 172.21.15.167:0/1410574079 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f68b80d3ca0 0x7f68b80d40f0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:34.929 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:34.925+0000 7f68bdc54640 1 --2- 172.21.15.167:0/1410574079 >> v2:172.21.15.204:3300/0 conn(0x7f68b815f450 0x7f68b80d01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:34.929 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:34.925+0000 7f68bdc54640 1 -- 172.21.15.167:0/1410574079 >> 172.21.15.167:0/1410574079 conn(0x7f68b80b6bb0 msgr2=0x7f68b80b6fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:05:34.929 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:34.925+0000 7f68bdc54640 1 -- 172.21.15.167:0/1410574079 shutdown_connections 2024-05-11T08:05:34.929 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:34.925+0000 7f68bdc54640 1 -- 172.21.15.167:0/1410574079 wait complete. 2024-05-11T08:05:34.929 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:34.926+0000 7f68bdc54640 1 Processor -- start 2024-05-11T08:05:34.929 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:34.926+0000 7f68bdc54640 1 -- start start 2024-05-11T08:05:34.929 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:34.927+0000 7f68bdc54640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f68b80b4620 0x7f68b81e8260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:05:34.929 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:34.927+0000 7f68bdc54640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f68b80d3ca0 0x7f68b814cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:05:34.929 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:34.927+0000 7f68bdc54640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f68b815f450 0x7f68b80cb030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:05:34.929 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:34.927+0000 7f68bdc54640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f68b80cfc90 con 0x7f68b80b4620 2024-05-11T08:05:34.929 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:34.927+0000 7f68bdc54640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f68b80cf9b0 con 0x7f68b80d3ca0 2024-05-11T08:05:34.929 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:34.927+0000 7f68bdc54640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f68b80b80c0 con 0x7f68b815f450 2024-05-11T08:05:34.929 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:34.927+0000 7f68b77fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f68b80b4620 0x7f68b81e8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:05:34.930 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:34.927+0000 7f68b77fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f68b80b4620 0x7f68b81e8260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:52148/0 (socket says 172.21.15.167:52148) 2024-05-11T08:05:34.930 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:34.927+0000 7f68b77fe640 1 -- 172.21.15.167:0/281532334 learned_addr learned my addr 172.21.15.167:0/281532334 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:05:34.930 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:34.927+0000 7f68b7fff640 1 --2- 172.21.15.167:0/281532334 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f68b815f450 0x7f68b80cb030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:05:34.930 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:34.927+0000 7f68b6ffd640 1 --2- 172.21.15.167:0/281532334 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f68b80d3ca0 0x7f68b814cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:05:34.930 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:34.927+0000 7f68b77fe640 1 -- 172.21.15.167:0/281532334 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f68b815f450 msgr2=0x7f68b80cb030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:05:34.930 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:34.927+0000 7f68b77fe640 1 --2- 172.21.15.167:0/281532334 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f68b815f450 0x7f68b80cb030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:34.930 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:34.927+0000 7f68b77fe640 1 -- 172.21.15.167:0/281532334 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f68b80d3ca0 msgr2=0x7f68b814cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:05:34.930 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:34.927+0000 7f68b77fe640 1 --2- 172.21.15.167:0/281532334 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f68b80d3ca0 0x7f68b814cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:34.930 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:34.927+0000 7f68b77fe640 1 -- 172.21.15.167:0/281532334 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68b814d000 con 0x7f68b80b4620 2024-05-11T08:05:34.930 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:34.927+0000 7f68b77fe640 1 --2- 172.21.15.167:0/281532334 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f68b80b4620 0x7f68b81e8260 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f68a800e870 tx=0x7f68a800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:05:34.930 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:34.927+0000 7f689ffff640 1 -- 172.21.15.167:0/281532334 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68a8017070 con 0x7f68b80b4620 2024-05-11T08:05:34.930 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:34.927+0000 7f689ffff640 1 -- 172.21.15.167:0/281532334 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68a800c940 con 0x7f68b80b4620 2024-05-11T08:05:34.930 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:34.927+0000 7f68bdc54640 1 -- 172.21.15.167:0/281532334 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68b814d2d0 con 0x7f68b80b4620 2024-05-11T08:05:34.930 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:34.927+0000 7f689ffff640 1 -- 172.21.15.167:0/281532334 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68a800cd30 con 0x7f68b80b4620 2024-05-11T08:05:34.931 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:34.927+0000 7f68bdc54640 1 -- 172.21.15.167:0/281532334 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68b814d890 con 0x7f68b80b4620 2024-05-11T08:05:34.931 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:34.928+0000 7f689ffff640 1 -- 172.21.15.167:0/281532334 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f68a801a070 con 0x7f68b80b4620 2024-05-11T08:05:34.931 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:34.928+0000 7f689ffff640 1 -- 172.21.15.167:0/281532334 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f68a8013b40 con 0x7f68b80b4620 2024-05-11T08:05:34.931 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:34.928+0000 7f68bdc54640 1 -- 172.21.15.167:0/281532334 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68b8069de0 con 0x7f68b80b4620 2024-05-11T08:05:34.931 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:34.930+0000 7f689ffff640 1 -- 172.21.15.167:0/281532334 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f68a8013e20 con 0x7f68b80b4620 2024-05-11T08:05:34.992 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:34.992+0000 7f68bdc54640 1 -- 172.21.15.167:0/281532334 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f68b80ba230 con 0x7f68b80b4620 2024-05-11T08:05:34.993 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:34.993+0000 7f689ffff640 1 -- 172.21.15.167:0/281532334 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f68a8006520 con 0x7f68b80b4620 2024-05-11T08:05:34.993 INFO:teuthology.misc.health.smithi167.stdout: 2024-05-11T08:05:34.993 INFO:teuthology.misc.health.smithi167.stdout:{"epoch":10,"fsid":"2034778d-8ff1-4823-aa44-ded32b518241","created":"2024-05-11T08:05:31.612421+0000","modified":"2024-05-11T08:05:33.757311+0000","last_up_change":"0.000000","last_in_change":"2024-05-11T08:05:33.757311+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"587eaced-5ded-406d-884b-30c1caa8686b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"37c59104-2dd0-4cf9-85ea-56636f6b02a1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"2bb4f7f5-c698-451b-b4bb-78e716921291","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"9dd26759-a338-49d2-8103-45d068b9c000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"cb27250d-2a80-4c7b-8cef-3a7af05c8a16","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"92998541-ffec-430b-ab76-8c3cb7c2752d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"9543479c-2ec5-42a8-8933-707bfed2e048","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"c45a5647-8358-409e-af91-c05dda65f0b7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-11T08:05:34.994 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:34.993+0000 7f68bdc54640 1 -- 172.21.15.167:0/281532334 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f68b80b4620 msgr2=0x7f68b81e8260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:05:34.994 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:34.993+0000 7f68bdc54640 1 --2- 172.21.15.167:0/281532334 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f68b80b4620 0x7f68b81e8260 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f68a800e870 tx=0x7f68a800ed30 comp rx=0 tx=0).stop 2024-05-11T08:05:34.994 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:34.993+0000 7f68bdc54640 1 -- 172.21.15.167:0/281532334 shutdown_connections 2024-05-11T08:05:34.994 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:34.993+0000 7f68bdc54640 1 --2- 172.21.15.167:0/281532334 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f68b815f450 0x7f68b80cb030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:34.994 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:34.993+0000 7f68bdc54640 1 --2- 172.21.15.167:0/281532334 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f68b80d3ca0 0x7f68b814cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:34.994 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:34.993+0000 7f68bdc54640 1 --2- 172.21.15.167:0/281532334 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f68b80b4620 0x7f68b81e8260 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:34.994 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:34.993+0000 7f68bdc54640 1 -- 172.21.15.167:0/281532334 >> 172.21.15.167:0/281532334 conn(0x7f68b80b6bb0 msgr2=0x7f68b80b60a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:05:34.994 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:34.994+0000 7f68bdc54640 1 -- 172.21.15.167:0/281532334 shutdown_connections 2024-05-11T08:05:34.994 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:34.994+0000 7f68bdc54640 1 -- 172.21.15.167:0/281532334 wait complete. 2024-05-11T08:05:35.003 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-05-11T08:05:35.009 INFO:tasks.ceph.mgr.y.smithi167.stderr:2024-05-11T08:05:35.009+0000 7f84d39b8040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-11T08:05:35.082 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-05-11T08:05:35.082+0000 7fa318e13040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-11T08:05:35.147 INFO:tasks.ceph.mgr.y.smithi167.stderr:2024-05-11T08:05:35.147+0000 7f84d39b8040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-11T08:05:35.196 INFO:tasks.ceph.mgr.y.smithi167.stderr:2024-05-11T08:05:35.196+0000 7f84d39b8040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-11T08:05:35.217 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-05-11T08:05:35.217+0000 7fa318e13040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-11T08:05:35.267 INFO:tasks.ceph.mgr.y.smithi167.stderr:2024-05-11T08:05:35.267+0000 7f84d39b8040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-11T08:05:35.268 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-05-11T08:05:35.268+0000 7fa318e13040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-11T08:05:35.311 INFO:tasks.ceph.mgr.y.smithi167.stderr:2024-05-11T08:05:35.311+0000 7f84d39b8040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-11T08:05:35.337 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-05-11T08:05:35.337+0000 7fa318e13040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-11T08:05:35.382 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-05-11T08:05:35.382+0000 7fa318e13040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-11T08:05:35.790 INFO:tasks.ceph.mgr.y.smithi167.stderr:2024-05-11T08:05:35.790+0000 7f84d39b8040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-11T08:05:35.860 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-05-11T08:05:35.861+0000 7fa318e13040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-11T08:05:35.905 INFO:tasks.ceph.mgr.y.smithi167.stderr:2024-05-11T08:05:35.905+0000 7f84d39b8040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-11T08:05:35.981 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-05-11T08:05:35.981+0000 7fa318e13040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-11T08:05:36.919 INFO:tasks.ceph.mgr.y.smithi167.stderr:2024-05-11T08:05:36.918+0000 7f8473d72640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-05-11T08:05:36.919 INFO:tasks.ceph.mgr.y.smithi167.stderr:2024-05-11T08:05:36.919+0000 7f8473d72640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-05-11T08:05:37.865 INFO:tasks.ceph.osd.2.smithi167.stderr:2024-05-11T08:05:37.865+0000 7f81d7b68640 -1 osd.2 0 log_to_monitors true 2024-05-11T08:05:37.914 INFO:tasks.ceph.osd.4.smithi204.stderr:2024-05-11T08:05:37.914+0000 7f1577418640 -1 osd.4 0 log_to_monitors true 2024-05-11T08:05:37.928 INFO:tasks.ceph.osd.7.smithi204.stderr:2024-05-11T08:05:37.928+0000 7fb957018640 -1 osd.7 0 log_to_monitors true 2024-05-11T08:05:37.928 INFO:tasks.ceph.osd.3.smithi167.stderr:2024-05-11T08:05:37.928+0000 7f8f8e544640 -1 osd.3 0 log_to_monitors true 2024-05-11T08:05:37.936 INFO:tasks.ceph.osd.0.smithi167.stderr:2024-05-11T08:05:37.937+0000 7f1d57a15640 -1 osd.0 0 log_to_monitors true 2024-05-11T08:05:37.941 INFO:tasks.ceph.osd.1.smithi167.stderr:2024-05-11T08:05:37.942+0000 7fd83fda7640 -1 osd.1 0 log_to_monitors true 2024-05-11T08:05:37.947 INFO:tasks.ceph.osd.5.smithi204.stderr:2024-05-11T08:05:37.948+0000 7eff1112a640 -1 osd.5 0 log_to_monitors true 2024-05-11T08:05:37.948 INFO:tasks.ceph.osd.6.smithi204.stderr:2024-05-11T08:05:37.948+0000 7f995e133640 -1 osd.6 0 log_to_monitors true 2024-05-11T08:05:39.709 INFO:tasks.ceph.osd.2.smithi167.stderr:2024-05-11T08:05:39.709+0000 7f81d3afe640 -1 osd.2 0 waiting for initial osdmap 2024-05-11T08:05:39.720 INFO:tasks.ceph.osd.2.smithi167.stderr:2024-05-11T08:05:39.720+0000 7f81cb91e640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-11T08:05:41.553 INFO:tasks.ceph.osd.0.smithi167.stderr:2024-05-11T08:05:41.553+0000 7f1d532fb640 -1 osd.0 0 waiting for initial osdmap 2024-05-11T08:05:41.562 INFO:tasks.ceph.osd.0.smithi167.stderr:2024-05-11T08:05:41.562+0000 7f1d4b11b640 -1 osd.0 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-11T08:05:41.567 INFO:tasks.ceph.osd.1.smithi167.stderr:2024-05-11T08:05:41.567+0000 7fd83bd3d640 -1 osd.1 0 waiting for initial osdmap 2024-05-11T08:05:41.585 INFO:tasks.ceph.osd.1.smithi167.stderr:2024-05-11T08:05:41.585+0000 7fd833b5d640 -1 osd.1 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-11T08:05:41.608 INFO:tasks.ceph.osd.3.smithi167.stderr:2024-05-11T08:05:41.608+0000 7f8f8acee640 -1 osd.3 0 waiting for initial osdmap 2024-05-11T08:05:41.620 INFO:tasks.ceph.osd.3.smithi167.stderr:2024-05-11T08:05:41.621+0000 7f8f822fa640 -1 osd.3 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-11T08:05:41.809 DEBUG:teuthology.orchestra.run.smithi167:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-05-11T08:05:41.902 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:41.902+0000 7fe360ff0640 1 Processor -- start 2024-05-11T08:05:41.902 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:41.902+0000 7fe360ff0640 1 -- start start 2024-05-11T08:05:41.905 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:41.903+0000 7fe360ff0640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe35c0cf9b0 0x7fe35c0cfdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:05:41.905 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:41.903+0000 7fe360ff0640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe35c0d0310 con 0x7fe35c072830 2024-05-11T08:05:41.905 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:41.903+0000 7fe360ff0640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe35c162f50 con 0x7fe35c148bb0 2024-05-11T08:05:41.905 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:41.903+0000 7fe360ff0640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fe35c163e20 con 0x7fe35c0cf9b0 2024-05-11T08:05:41.906 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:41.903+0000 7fe351d9b640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7fe35c072830 0x7fe35c163710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:33140/0 (socket says 172.21.15.167:33140) 2024-05-11T08:05:41.906 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:41.903+0000 7fe351d9b640 1 -- 172.21.15.167:0/1729803972 learned_addr learned my addr 172.21.15.167:0/1729803972 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T08:05:41.906 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:41.903+0000 7fe35ad9d640 1 --2- 172.21.15.167:0/1729803972 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe35c0cf9b0 0x7fe35c0cfdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:05:41.906 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:41.903+0000 7fe359d9b640 1 -- 172.21.15.167:0/1729803972 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3557290826 0 0) 0x7fe35c0d0310 con 0x7fe35c072830 2024-05-11T08:05:41.906 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:41.903+0000 7fe359d9b640 1 -- 172.21.15.167:0/1729803972 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe348003520 con 0x7fe35c072830 2024-05-11T08:05:41.906 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:41.903+0000 7fe359d9b640 1 -- 172.21.15.167:0/1729803972 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3437436563 0 0) 0x7fe35c162f50 con 0x7fe35c148bb0 2024-05-11T08:05:41.906 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:41.903+0000 7fe359d9b640 1 -- 172.21.15.167:0/1729803972 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe35c0d0310 con 0x7fe35c148bb0 2024-05-11T08:05:41.906 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:41.903+0000 7fe359d9b640 1 -- 172.21.15.167:0/1729803972 <== mon.0 v1:172.21.15.167:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 820148996 0 0) 0x7fe348003520 con 0x7fe35c072830 2024-05-11T08:05:41.906 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:41.903+0000 7fe359d9b640 1 -- 172.21.15.167:0/1729803972 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe35c0cf9b0 msgr2=0x7fe35c0cfdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:05:41.906 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:41.903+0000 7fe359d9b640 1 --2- 172.21.15.167:0/1729803972 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe35c0cf9b0 0x7fe35c0cfdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:41.906 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:41.903+0000 7fe359d9b640 1 -- 172.21.15.167:0/1729803972 >> v1:172.21.15.204:6789/0 conn(0x7fe35c148bb0 legacy=0x7fe35c160310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:05:41.906 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:41.903+0000 7fe359d9b640 1 -- 172.21.15.167:0/1729803972 --> v1:172.21.15.167:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe35c148f90 con 0x7fe35c072830 2024-05-11T08:05:41.907 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:41.903+0000 7fe359d9b640 1 -- 172.21.15.167:0/1729803972 <== mon.0 v1:172.21.15.167:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3602277846 0 0) 0x7fe3440016f0 con 0x7fe35c072830 2024-05-11T08:05:41.907 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:41.903+0000 7fe359d9b640 1 -- 172.21.15.167:0/1729803972 >> v1:172.21.15.167:6789/0 conn(0x7fe35c072830 legacy=0x7fe35c163710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:05:41.907 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:41.903+0000 7fe359d9b640 1 --2- 172.21.15.167:0/1729803972 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe348003c10 0x7fe348004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:05:41.907 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:41.903+0000 7fe359d9b640 1 --2- 172.21.15.167:0/1729803972 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe35c0cf9b0 0x7fe348004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:05:41.907 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:41.903+0000 7fe359d9b640 1 --2- 172.21.15.167:0/1729803972 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe348004d40 0x7fe3480051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:05:41.907 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:41.903+0000 7fe359d9b640 1 -- 172.21.15.167:0/1729803972 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe35c163e20 con 0x7fe348003c10 2024-05-11T08:05:41.907 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:41.903+0000 7fe359d9b640 1 -- 172.21.15.167:0/1729803972 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe34800bb50 con 0x7fe348004d40 2024-05-11T08:05:41.907 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:41.903+0000 7fe359d9b640 1 -- 172.21.15.167:0/1729803972 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fe34800bd50 con 0x7fe35c0cf9b0 2024-05-11T08:05:41.907 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:41.903+0000 7fe35a59c640 1 --2- 172.21.15.167:0/1729803972 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe348003c10 0x7fe348004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:05:41.907 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:41.904+0000 7fe35ad9d640 1 --2- 172.21.15.167:0/1729803972 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe348004d40 0x7fe3480051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:05:41.907 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:41.904+0000 7fe35a59c640 1 -- 172.21.15.167:0/1729803972 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe35c0cf9b0 msgr2=0x7fe348004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:05:41.907 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:41.904+0000 7fe35a59c640 1 --2- 172.21.15.167:0/1729803972 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe35c0cf9b0 0x7fe348004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:41.907 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:41.904+0000 7fe35a59c640 1 -- 172.21.15.167:0/1729803972 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe348004d40 msgr2=0x7fe3480051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:05:41.907 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:41.904+0000 7fe35a59c640 1 --2- 172.21.15.167:0/1729803972 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe348004d40 0x7fe3480051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:41.907 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:41.904+0000 7fe35a59c640 1 -- 172.21.15.167:0/1729803972 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe348003520 con 0x7fe348003c10 2024-05-11T08:05:41.908 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:41.904+0000 7fe35a59c640 1 --2- 172.21.15.167:0/1729803972 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe348003c10 0x7fe348004010 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe34c001900 tx=0x7fe348005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1922ee0c23eb1a8 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:05:41.908 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:41.904+0000 7fe359d9b640 1 -- 172.21.15.167:0/1729803972 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe34c001520 con 0x7fe348003c10 2024-05-11T08:05:41.908 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:41.904+0000 7fe359d9b640 1 -- 172.21.15.167:0/1729803972 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe34c02acd0 con 0x7fe348003c10 2024-05-11T08:05:41.908 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:41.904+0000 7fe359d9b640 1 -- 172.21.15.167:0/1729803972 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe34c005310 con 0x7fe348003c10 2024-05-11T08:05:41.908 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:41.905+0000 7fe360ff0640 1 -- 172.21.15.167:0/1729803972 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe348003c10 msgr2=0x7fe348004010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:05:41.908 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:41.905+0000 7fe360ff0640 1 --2- 172.21.15.167:0/1729803972 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe348003c10 0x7fe348004010 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe34c001900 tx=0x7fe348005830 comp rx=0 tx=0).stop 2024-05-11T08:05:41.908 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:41.905+0000 7fe35ad9d640 1 -- 172.21.15.167:0/1729803972 reap_dead start 2024-05-11T08:05:41.908 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:41.905+0000 7fe360ff0640 1 -- 172.21.15.167:0/1729803972 shutdown_connections 2024-05-11T08:05:41.908 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:41.905+0000 7fe360ff0640 1 -- 172.21.15.167:0/1729803972 >> 172.21.15.167:0/1729803972 conn(0x7fe35c0d3680 msgr2=0x7fe35c06fed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:05:41.908 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:41.905+0000 7fe360ff0640 1 -- 172.21.15.167:0/1729803972 shutdown_connections 2024-05-11T08:05:41.908 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:41.905+0000 7fe360ff0640 1 -- 172.21.15.167:0/1729803972 wait complete. 2024-05-11T08:05:41.911 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:41.906+0000 7fe360ff0640 1 Processor -- start 2024-05-11T08:05:41.911 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:41.907+0000 7fe360ff0640 1 -- start start 2024-05-11T08:05:41.911 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:41.907+0000 7fe360ff0640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe348003c10 0x7fe35c06ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:05:41.911 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:41.907+0000 7fe360ff0640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe35c0cd0b0 0x7fe35c1613b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:05:41.911 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:41.907+0000 7fe360ff0640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe35c0b6710 0x7fe35c0ca530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:05:41.911 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:41.907+0000 7fe360ff0640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe35c1631f0 con 0x7fe35c0cd0b0 2024-05-11T08:05:41.911 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:41.907+0000 7fe360ff0640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe35c0c8280 con 0x7fe348003c10 2024-05-11T08:05:41.911 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:41.907+0000 7fe360ff0640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fe35c0c8560 con 0x7fe35c0b6710 2024-05-11T08:05:41.911 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:41.907+0000 7fe351d9b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe35c0cd0b0 0x7fe35c1613b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:05:41.911 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:41.907+0000 7fe351d9b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe35c0cd0b0 0x7fe35c1613b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:52274/0 (socket says 172.21.15.167:52274) 2024-05-11T08:05:41.912 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:41.907+0000 7fe351d9b640 1 -- 172.21.15.167:0/709726743 learned_addr learned my addr 172.21.15.167:0/709726743 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:05:41.912 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:41.907+0000 7fe35a59c640 1 --2- 172.21.15.167:0/709726743 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe348003c10 0x7fe35c06ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:05:41.912 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:41.907+0000 7fe35ad9d640 1 --2- 172.21.15.167:0/709726743 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe35c0b6710 0x7fe35c0ca530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:05:41.912 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:41.908+0000 7fe351d9b640 1 -- 172.21.15.167:0/709726743 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe35c0b6710 msgr2=0x7fe35c0ca530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:05:41.912 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:41.908+0000 7fe351d9b640 1 --2- 172.21.15.167:0/709726743 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe35c0b6710 0x7fe35c0ca530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:41.912 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:41.908+0000 7fe351d9b640 1 -- 172.21.15.167:0/709726743 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe348003c10 msgr2=0x7fe35c06ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:05:41.912 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:41.908+0000 7fe351d9b640 1 --2- 172.21.15.167:0/709726743 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe348003c10 0x7fe35c06ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:41.912 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:41.908+0000 7fe351d9b640 1 -- 172.21.15.167:0/709726743 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe35c161d20 con 0x7fe35c0cd0b0 2024-05-11T08:05:41.912 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:41.908+0000 7fe351d9b640 1 --2- 172.21.15.167:0/709726743 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe35c0cd0b0 0x7fe35c1613b0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fe34400c6c0 tx=0x7fe34400cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:05:41.912 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:41.908+0000 7fe3537fe640 1 -- 172.21.15.167:0/709726743 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3440072d0 con 0x7fe35c0cd0b0 2024-05-11T08:05:41.912 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:41.908+0000 7fe3537fe640 1 -- 172.21.15.167:0/709726743 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe344007490 con 0x7fe35c0cd0b0 2024-05-11T08:05:41.912 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:41.908+0000 7fe3537fe640 1 -- 172.21.15.167:0/709726743 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe344004a20 con 0x7fe35c0cd0b0 2024-05-11T08:05:41.912 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:41.908+0000 7fe360ff0640 1 -- 172.21.15.167:0/709726743 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe35c162000 con 0x7fe35c0cd0b0 2024-05-11T08:05:41.913 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:41.908+0000 7fe360ff0640 1 -- 172.21.15.167:0/709726743 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe35c0b61f0 con 0x7fe35c0cd0b0 2024-05-11T08:05:41.913 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:41.909+0000 7fe360ff0640 1 -- 172.21.15.167:0/709726743 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe35c069de0 con 0x7fe35c0cd0b0 2024-05-11T08:05:41.913 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:41.910+0000 7fe3537fe640 1 -- 172.21.15.167:0/709726743 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 4) ==== 98145+0+0 (secure 0 0 0) 0x7fe344011070 con 0x7fe35c0cd0b0 2024-05-11T08:05:41.913 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:41.910+0000 7fe3537fe640 1 --2- 172.21.15.167:0/709726743 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fe33c076270 0x7fe33c078730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:05:41.913 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:41.910+0000 7fe35a59c640 1 --2- 172.21.15.167:0/709726743 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fe33c076270 0x7fe33c078730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:05:41.913 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:41.910+0000 7fe3537fe640 1 -- 172.21.15.167:0/709726743 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2511+0+0 (secure 0 0 0) 0x7fe344096390 con 0x7fe35c0cd0b0 2024-05-11T08:05:41.913 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:41.911+0000 7fe35a59c640 1 --2- 172.21.15.167:0/709726743 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fe33c076270 0x7fe33c078730 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fe34c001900 tx=0x7fe34c02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:05:41.915 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:41.915+0000 7fe3537fe640 1 -- 172.21.15.167:0/709726743 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fe344060490 con 0x7fe35c0cd0b0 2024-05-11T08:05:41.970 INFO:tasks.ceph.osd.5.smithi204.stderr:2024-05-11T08:05:41.970+0000 7eff0d0c0640 -1 osd.5 0 waiting for initial osdmap 2024-05-11T08:05:41.986 INFO:tasks.ceph.osd.5.smithi204.stderr:2024-05-11T08:05:41.986+0000 7eff04ee0640 -1 osd.5 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-11T08:05:42.024 INFO:tasks.ceph.osd.7.smithi204.stderr:2024-05-11T08:05:42.025+0000 7fb9531b4640 -1 osd.7 0 waiting for initial osdmap 2024-05-11T08:05:42.047 INFO:tasks.ceph.osd.7.smithi204.stderr:2024-05-11T08:05:42.047+0000 7fb94a7c0640 -1 osd.7 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-11T08:05:42.065 INFO:tasks.ceph.osd.6.smithi204.stderr:2024-05-11T08:05:42.065+0000 7f995a0c9640 -1 osd.6 0 waiting for initial osdmap 2024-05-11T08:05:42.075 INFO:tasks.ceph.osd.4.smithi204.stderr:2024-05-11T08:05:42.075+0000 7f1572de7640 -1 osd.4 0 waiting for initial osdmap 2024-05-11T08:05:42.077 INFO:tasks.ceph.osd.6.smithi204.stderr:2024-05-11T08:05:42.077+0000 7f9951ee9640 -1 osd.6 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-11T08:05:42.085 INFO:tasks.ceph.osd.4.smithi204.stderr:2024-05-11T08:05:42.085+0000 7f156ac07640 -1 osd.4 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-11T08:05:42.106 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:42.107+0000 7fe360ff0640 1 -- 172.21.15.167:0/709726743 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe35c0725a0 con 0x7fe35c0cd0b0 2024-05-11T08:05:42.107 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:42.107+0000 7fe3537fe640 1 -- 172.21.15.167:0/709726743 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v14) ==== 74+0+7590 (secure 0 0 0) 0x7fe344064790 con 0x7fe35c0cd0b0 2024-05-11T08:05:42.107 INFO:teuthology.misc.health.smithi167.stdout: 2024-05-11T08:05:42.107 INFO:teuthology.misc.health.smithi167.stdout:{"epoch":14,"fsid":"2034778d-8ff1-4823-aa44-ded32b518241","created":"2024-05-11T08:05:31.612421+0000","modified":"2024-05-11T08:05:40.958503+0000","last_up_change":"2024-05-11T08:05:39.926890+0000","last_in_change":"2024-05-11T08:05:33.757311+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"587eaced-5ded-406d-884b-30c1caa8686b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"37c59104-2dd0-4cf9-85ea-56636f6b02a1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"2bb4f7f5-c698-451b-b4bb-78e716921291","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":3242166900},{"type":"v1","addr":"172.21.15.167:6801","nonce":3242166900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":3242166900},{"type":"v1","addr":"172.21.15.167:6803","nonce":3242166900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6809","nonce":3242166900},{"type":"v1","addr":"172.21.15.167:6811","nonce":3242166900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6805","nonce":3242166900},{"type":"v1","addr":"172.21.15.167:6807","nonce":3242166900}]},"public_addr":"172.21.15.167:6801/3242166900","cluster_addr":"172.21.15.167:6803/3242166900","heartbeat_back_addr":"172.21.15.167:6811/3242166900","heartbeat_front_addr":"172.21.15.167:6807/3242166900","state":["exists","up"]},{"osd":3,"uuid":"9dd26759-a338-49d2-8103-45d068b9c000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"cb27250d-2a80-4c7b-8cef-3a7af05c8a16","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"92998541-ffec-430b-ab76-8c3cb7c2752d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"9543479c-2ec5-42a8-8933-707bfed2e048","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"c45a5647-8358-409e-af91-c05dda65f0b7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.860682+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-11T08:05:42.109 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:42.109+0000 7fe360ff0640 1 -- 172.21.15.167:0/709726743 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fe33c076270 msgr2=0x7fe33c078730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:05:42.110 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:42.109+0000 7fe360ff0640 1 --2- 172.21.15.167:0/709726743 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fe33c076270 0x7fe33c078730 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fe34c001900 tx=0x7fe34c02e000 comp rx=0 tx=0).stop 2024-05-11T08:05:42.110 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:42.109+0000 7fe360ff0640 1 -- 172.21.15.167:0/709726743 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe35c0cd0b0 msgr2=0x7fe35c1613b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:05:42.110 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:42.109+0000 7fe360ff0640 1 --2- 172.21.15.167:0/709726743 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe35c0cd0b0 0x7fe35c1613b0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fe34400c6c0 tx=0x7fe34400cb80 comp rx=0 tx=0).stop 2024-05-11T08:05:42.110 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:42.109+0000 7fe360ff0640 1 -- 172.21.15.167:0/709726743 shutdown_connections 2024-05-11T08:05:42.110 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:42.109+0000 7fe360ff0640 1 --2- 172.21.15.167:0/709726743 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fe33c076270 0x7fe33c078730 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:42.110 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:42.109+0000 7fe360ff0640 1 --2- 172.21.15.167:0/709726743 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe35c0b6710 0x7fe35c0ca530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:42.110 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:42.109+0000 7fe360ff0640 1 --2- 172.21.15.167:0/709726743 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe35c0cd0b0 0x7fe35c1613b0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:42.110 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:42.109+0000 7fe360ff0640 1 --2- 172.21.15.167:0/709726743 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe348003c10 0x7fe35c06ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:42.110 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:42.109+0000 7fe360ff0640 1 -- 172.21.15.167:0/709726743 >> 172.21.15.167:0/709726743 conn(0x7fe35c0d3680 msgr2=0x7fe35c0d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:05:42.110 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:42.109+0000 7fe360ff0640 1 -- 172.21.15.167:0/709726743 shutdown_connections 2024-05-11T08:05:42.110 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:42.109+0000 7fe360ff0640 1 -- 172.21.15.167:0/709726743 wait complete. 2024-05-11T08:05:42.119 DEBUG:teuthology.misc:1 of 8 OSDs are up 2024-05-11T08:05:48.925 DEBUG:teuthology.orchestra.run.smithi167:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-05-11T08:05:49.028 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:49.028+0000 7f7a35e93640 1 Processor -- start 2024-05-11T08:05:49.028 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:49.028+0000 7f7a35e93640 1 -- start start 2024-05-11T08:05:49.029 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:49.029+0000 7f7a35e93640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f7a30148bb0 0x7f7a30160310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:05:49.029 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:49.029+0000 7f7a35e93640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7a30072830 0x7f7a30163710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:05:49.029 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:49.029+0000 7f7a35e93640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a30162f50 con 0x7f7a300cf9b0 2024-05-11T08:05:49.030 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:49.029+0000 7f7a35e93640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f7a30160850 con 0x7f7a30148bb0 2024-05-11T08:05:49.030 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:49.029+0000 7f7a35e93640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f7a30148f90 con 0x7f7a30072830 2024-05-11T08:05:49.030 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:49.029+0000 7f7a2f7fe640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f7a30148bb0 0x7f7a30160310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:05:49.030 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:49.029+0000 7f7a2f7fe640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f7a30148bb0 0x7f7a30160310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:42412/0 (socket says 172.21.15.167:42412) 2024-05-11T08:05:49.030 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:49.029+0000 7f7a2f7fe640 1 -- 172.21.15.167:0/63427693 learned_addr learned my addr 172.21.15.167:0/63427693 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:05:49.030 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:49.029+0000 7f7a2f7fe640 1 -- 172.21.15.167:0/63427693 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7a30072830 msgr2=0x7f7a30163710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:05:49.030 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:49.029+0000 7f7a2f7fe640 1 --2- 172.21.15.167:0/63427693 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7a30072830 0x7f7a30163710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:49.030 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:49.029+0000 7f7a2f7fe640 1 -- 172.21.15.167:0/63427693 >> v1:172.21.15.204:6789/0 conn(0x7f7a300cf9b0 legacy=0x7f7a300cfdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:05:49.030 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:49.029+0000 7f7a2f7fe640 1 -- 172.21.15.167:0/63427693 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a300c7fc0 con 0x7f7a30148bb0 2024-05-11T08:05:49.031 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:49.030+0000 7f7a2f7fe640 1 --2- 172.21.15.167:0/63427693 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7a30148bb0 0x7f7a30160310 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f7a20009e50 tx=0x7f7a2002a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a66a774715d2876a server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:05:49.031 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:49.030+0000 7f7a2e7fc640 1 -- 172.21.15.167:0/63427693 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a20036070 con 0x7f7a30148bb0 2024-05-11T08:05:49.031 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:49.030+0000 7f7a2e7fc640 1 -- 172.21.15.167:0/63427693 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a20031600 con 0x7f7a30148bb0 2024-05-11T08:05:49.032 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:49.030+0000 7f7a2e7fc640 1 -- 172.21.15.167:0/63427693 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a200319d0 con 0x7f7a30148bb0 2024-05-11T08:05:49.032 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:49.031+0000 7f7a35e93640 1 -- 172.21.15.167:0/63427693 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7a30148bb0 msgr2=0x7f7a30160310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:05:49.032 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:49.031+0000 7f7a35e93640 1 --2- 172.21.15.167:0/63427693 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7a30148bb0 0x7f7a30160310 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f7a20009e50 tx=0x7f7a2002a5f0 comp rx=0 tx=0).stop 2024-05-11T08:05:49.032 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:49.031+0000 7f7a35e93640 1 -- 172.21.15.167:0/63427693 shutdown_connections 2024-05-11T08:05:49.032 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:49.031+0000 7f7a35e93640 1 --2- 172.21.15.167:0/63427693 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7a30072830 0x7f7a30163710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:49.032 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:49.031+0000 7f7a35e93640 1 --2- 172.21.15.167:0/63427693 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7a30148bb0 0x7f7a30160310 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:49.032 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:49.031+0000 7f7a35e93640 1 -- 172.21.15.167:0/63427693 >> 172.21.15.167:0/63427693 conn(0x7f7a300d3680 msgr2=0x7f7a3006fed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:05:49.032 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:49.031+0000 7f7a35e93640 1 -- 172.21.15.167:0/63427693 shutdown_connections 2024-05-11T08:05:49.032 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:49.031+0000 7f7a35e93640 1 -- 172.21.15.167:0/63427693 wait complete. 2024-05-11T08:05:49.033 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:49.032+0000 7f7a35e93640 1 Processor -- start 2024-05-11T08:05:49.033 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:49.032+0000 7f7a35e93640 1 -- start start 2024-05-11T08:05:49.035 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:49.033+0000 7f7a35e93640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7a30072830 0x7f7a3015ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:05:49.035 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:49.033+0000 7f7a35e93640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a300cf9b0 0x7f7a3013dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:05:49.035 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:49.033+0000 7f7a35e93640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7a30148bb0 0x7f7a30161ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:05:49.035 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:49.033+0000 7f7a35e93640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7a300c8560 con 0x7f7a30148bb0 2024-05-11T08:05:49.035 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:49.033+0000 7f7a35e93640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7a301631f0 con 0x7f7a300cf9b0 2024-05-11T08:05:49.035 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:49.033+0000 7f7a35e93640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f7a300c86e0 con 0x7f7a30072830 2024-05-11T08:05:49.035 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:49.033+0000 7f7a2ffff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7a30148bb0 0x7f7a30161ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:05:49.035 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:49.033+0000 7f7a2ffff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7a30148bb0 0x7f7a30161ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:42420/0 (socket says 172.21.15.167:42420) 2024-05-11T08:05:49.035 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:49.033+0000 7f7a2ffff640 1 -- 172.21.15.167:0/1250873727 learned_addr learned my addr 172.21.15.167:0/1250873727 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:05:49.035 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:49.033+0000 7f7a2f7fe640 1 --2- 172.21.15.167:0/1250873727 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7a30072830 0x7f7a3015ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:05:49.036 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:49.033+0000 7f7a2effd640 1 --2- 172.21.15.167:0/1250873727 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a300cf9b0 0x7f7a3013dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:05:49.036 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:49.034+0000 7f7a2ffff640 1 -- 172.21.15.167:0/1250873727 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7a30072830 msgr2=0x7f7a3015ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:05:49.036 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:49.034+0000 7f7a2ffff640 1 --2- 172.21.15.167:0/1250873727 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7a30072830 0x7f7a3015ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:49.036 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:49.034+0000 7f7a2ffff640 1 -- 172.21.15.167:0/1250873727 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a300cf9b0 msgr2=0x7f7a3013dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:05:49.036 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:49.034+0000 7f7a2ffff640 1 --2- 172.21.15.167:0/1250873727 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a300cf9b0 0x7f7a3013dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:49.036 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:49.034+0000 7f7a2ffff640 1 -- 172.21.15.167:0/1250873727 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a30143380 con 0x7f7a30148bb0 2024-05-11T08:05:49.036 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:49.034+0000 7f7a2ffff640 1 --2- 172.21.15.167:0/1250873727 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7a30148bb0 0x7f7a30161ab0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f7a2800c390 tx=0x7f7a2800c790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:05:49.036 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:49.034+0000 7f7a17fff640 1 -- 172.21.15.167:0/1250873727 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a28017070 con 0x7f7a30148bb0 2024-05-11T08:05:49.036 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:49.034+0000 7f7a17fff640 1 -- 172.21.15.167:0/1250873727 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a280035f0 con 0x7f7a30148bb0 2024-05-11T08:05:49.036 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:49.034+0000 7f7a17fff640 1 -- 172.21.15.167:0/1250873727 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a280039e0 con 0x7f7a30148bb0 2024-05-11T08:05:49.036 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:49.034+0000 7f7a35e93640 1 -- 172.21.15.167:0/1250873727 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a3015e130 con 0x7f7a30148bb0 2024-05-11T08:05:49.036 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:49.034+0000 7f7a35e93640 1 -- 172.21.15.167:0/1250873727 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a3013e260 con 0x7f7a30148bb0 2024-05-11T08:05:49.036 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:49.035+0000 7f7a35e93640 1 -- 172.21.15.167:0/1250873727 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a3015df20 con 0x7f7a30148bb0 2024-05-11T08:05:49.037 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:49.036+0000 7f7a17fff640 1 -- 172.21.15.167:0/1250873727 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f7a2801a070 con 0x7f7a30148bb0 2024-05-11T08:05:49.037 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:49.036+0000 7f7a17fff640 1 --2- 172.21.15.167:0/1250873727 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f7a1c0762d0 0x7f7a1c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:05:49.037 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:49.036+0000 7f7a17fff640 1 -- 172.21.15.167:0/1250873727 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4857+0+0 (secure 0 0 0) 0x7f7a28096500 con 0x7f7a30148bb0 2024-05-11T08:05:49.037 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:49.036+0000 7f7a2f7fe640 1 --2- 172.21.15.167:0/1250873727 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f7a1c0762d0 0x7f7a1c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:05:49.037 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:49.037+0000 7f7a2f7fe640 1 --2- 172.21.15.167:0/1250873727 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f7a1c0762d0 0x7f7a1c078790 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7a20033000 tx=0x7f7a20009000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:05:49.041 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:49.041+0000 7f7a17fff640 1 -- 172.21.15.167:0/1250873727 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f7a2805fbf0 con 0x7f7a30148bb0 2024-05-11T08:05:49.226 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:49.226+0000 7f7a35e93640 1 -- 172.21.15.167:0/1250873727 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7a30069de0 con 0x7f7a30148bb0 2024-05-11T08:05:49.227 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:49.227+0000 7f7a17fff640 1 -- 172.21.15.167:0/1250873727 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) ==== 74+0+12932 (secure 0 0 0) 0x7f7a28063ef0 con 0x7f7a30148bb0 2024-05-11T08:05:49.227 INFO:teuthology.misc.health.smithi167.stdout: 2024-05-11T08:05:49.228 INFO:teuthology.misc.health.smithi167.stdout:{"epoch":19,"fsid":"2034778d-8ff1-4823-aa44-ded32b518241","created":"2024-05-11T08:05:31.612421+0000","modified":"2024-05-11T08:05:46.628709+0000","last_up_change":"2024-05-11T08:05:42.557382+0000","last_in_change":"2024-05-11T08:05:33.757311+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-11T08:05:43.931263+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"587eaced-5ded-406d-884b-30c1caa8686b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":3863307356},{"type":"v1","addr":"172.21.15.167:6806","nonce":3863307356}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":3863307356},{"type":"v1","addr":"172.21.15.167:6810","nonce":3863307356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":3863307356},{"type":"v1","addr":"172.21.15.167:6815","nonce":3863307356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":3863307356},{"type":"v1","addr":"172.21.15.167:6813","nonce":3863307356}]},"public_addr":"172.21.15.167:6806/3863307356","cluster_addr":"172.21.15.167:6810/3863307356","heartbeat_back_addr":"172.21.15.167:6815/3863307356","heartbeat_front_addr":"172.21.15.167:6813/3863307356","state":["exists","up"]},{"osd":1,"uuid":"37c59104-2dd0-4cf9-85ea-56636f6b02a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":1585024048},{"type":"v1","addr":"172.21.15.167:6817","nonce":1585024048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1585024048},{"type":"v1","addr":"172.21.15.167:6819","nonce":1585024048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1585024048},{"type":"v1","addr":"172.21.15.167:6823","nonce":1585024048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":1585024048},{"type":"v1","addr":"172.21.15.167:6821","nonce":1585024048}]},"public_addr":"172.21.15.167:6817/1585024048","cluster_addr":"172.21.15.167:6819/1585024048","heartbeat_back_addr":"172.21.15.167:6823/1585024048","heartbeat_front_addr":"172.21.15.167:6821/1585024048","state":["exists","up"]},{"osd":2,"uuid":"2bb4f7f5-c698-451b-b4bb-78e716921291","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":3242166900},{"type":"v1","addr":"172.21.15.167:6801","nonce":3242166900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":3242166900},{"type":"v1","addr":"172.21.15.167:6803","nonce":3242166900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6809","nonce":3242166900},{"type":"v1","addr":"172.21.15.167:6811","nonce":3242166900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6805","nonce":3242166900},{"type":"v1","addr":"172.21.15.167:6807","nonce":3242166900}]},"public_addr":"172.21.15.167:6801/3242166900","cluster_addr":"172.21.15.167:6803/3242166900","heartbeat_back_addr":"172.21.15.167:6811/3242166900","heartbeat_front_addr":"172.21.15.167:6807/3242166900","state":["exists","up"]},{"osd":3,"uuid":"9dd26759-a338-49d2-8103-45d068b9c000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":453689293},{"type":"v1","addr":"172.21.15.167:6825","nonce":453689293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":453689293},{"type":"v1","addr":"172.21.15.167:6827","nonce":453689293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":453689293},{"type":"v1","addr":"172.21.15.167:6831","nonce":453689293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":453689293},{"type":"v1","addr":"172.21.15.167:6829","nonce":453689293}]},"public_addr":"172.21.15.167:6825/453689293","cluster_addr":"172.21.15.167:6827/453689293","heartbeat_back_addr":"172.21.15.167:6831/453689293","heartbeat_front_addr":"172.21.15.167:6829/453689293","state":["exists","up"]},{"osd":4,"uuid":"cb27250d-2a80-4c7b-8cef-3a7af05c8a16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1311831500},{"type":"v1","addr":"172.21.15.204:6801","nonce":1311831500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1311831500},{"type":"v1","addr":"172.21.15.204:6803","nonce":1311831500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1311831500},{"type":"v1","addr":"172.21.15.204:6807","nonce":1311831500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1311831500},{"type":"v1","addr":"172.21.15.204:6805","nonce":1311831500}]},"public_addr":"172.21.15.204:6801/1311831500","cluster_addr":"172.21.15.204:6803/1311831500","heartbeat_back_addr":"172.21.15.204:6807/1311831500","heartbeat_front_addr":"172.21.15.204:6805/1311831500","state":["exists","up"]},{"osd":5,"uuid":"92998541-ffec-430b-ab76-8c3cb7c2752d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2921847521},{"type":"v1","addr":"172.21.15.204:6825","nonce":2921847521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2921847521},{"type":"v1","addr":"172.21.15.204:6827","nonce":2921847521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2921847521},{"type":"v1","addr":"172.21.15.204:6831","nonce":2921847521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2921847521},{"type":"v1","addr":"172.21.15.204:6829","nonce":2921847521}]},"public_addr":"172.21.15.204:6825/2921847521","cluster_addr":"172.21.15.204:6827/2921847521","heartbeat_back_addr":"172.21.15.204:6831/2921847521","heartbeat_front_addr":"172.21.15.204:6829/2921847521","state":["exists","up"]},{"osd":6,"uuid":"9543479c-2ec5-42a8-8933-707bfed2e048","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":2889851077},{"type":"v1","addr":"172.21.15.204:6817","nonce":2889851077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":2889851077},{"type":"v1","addr":"172.21.15.204:6819","nonce":2889851077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":2889851077},{"type":"v1","addr":"172.21.15.204:6823","nonce":2889851077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":2889851077},{"type":"v1","addr":"172.21.15.204:6821","nonce":2889851077}]},"public_addr":"172.21.15.204:6817/2889851077","cluster_addr":"172.21.15.204:6819/2889851077","heartbeat_back_addr":"172.21.15.204:6823/2889851077","heartbeat_front_addr":"172.21.15.204:6821/2889851077","state":["exists","up"]},{"osd":7,"uuid":"c45a5647-8358-409e-af91-c05dda65f0b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":1949534184},{"type":"v1","addr":"172.21.15.204:6809","nonce":1949534184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":1949534184},{"type":"v1","addr":"172.21.15.204:6811","nonce":1949534184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1949534184},{"type":"v1","addr":"172.21.15.204:6815","nonce":1949534184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1949534184},{"type":"v1","addr":"172.21.15.204:6813","nonce":1949534184}]},"public_addr":"172.21.15.204:6809/1949534184","cluster_addr":"172.21.15.204:6811/1949534184","heartbeat_back_addr":"172.21.15.204:6815/1949534184","heartbeat_front_addr":"172.21.15.204:6813/1949534184","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.952253+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.940689+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.860682+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.929350+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.901627+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.957451+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.987249+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.951197+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-11T08:05:49.230 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:49.229+0000 7f7a35e93640 1 -- 172.21.15.167:0/1250873727 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f7a1c0762d0 msgr2=0x7f7a1c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:05:49.230 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:49.229+0000 7f7a35e93640 1 --2- 172.21.15.167:0/1250873727 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f7a1c0762d0 0x7f7a1c078790 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7a20033000 tx=0x7f7a20009000 comp rx=0 tx=0).stop 2024-05-11T08:05:49.230 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:49.229+0000 7f7a35e93640 1 -- 172.21.15.167:0/1250873727 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7a30148bb0 msgr2=0x7f7a30161ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:05:49.230 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:49.229+0000 7f7a35e93640 1 --2- 172.21.15.167:0/1250873727 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7a30148bb0 0x7f7a30161ab0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f7a2800c390 tx=0x7f7a2800c790 comp rx=0 tx=0).stop 2024-05-11T08:05:49.230 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:49.229+0000 7f7a35e93640 1 -- 172.21.15.167:0/1250873727 shutdown_connections 2024-05-11T08:05:49.230 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:49.229+0000 7f7a35e93640 1 --2- 172.21.15.167:0/1250873727 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f7a1c0762d0 0x7f7a1c078790 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:49.230 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:49.229+0000 7f7a35e93640 1 --2- 172.21.15.167:0/1250873727 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7a30148bb0 0x7f7a30161ab0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:49.230 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:49.229+0000 7f7a35e93640 1 --2- 172.21.15.167:0/1250873727 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a300cf9b0 0x7f7a3013dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:49.230 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:49.229+0000 7f7a35e93640 1 --2- 172.21.15.167:0/1250873727 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7a30072830 0x7f7a3015ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:49.230 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:49.229+0000 7f7a35e93640 1 -- 172.21.15.167:0/1250873727 >> 172.21.15.167:0/1250873727 conn(0x7f7a300d3680 msgr2=0x7f7a300b4320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:05:49.230 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:49.230+0000 7f7a35e93640 1 -- 172.21.15.167:0/1250873727 shutdown_connections 2024-05-11T08:05:49.230 INFO:teuthology.misc.health.smithi167.stderr:2024-05-11T08:05:49.230+0000 7f7a35e93640 1 -- 172.21.15.167:0/1250873727 wait complete. 2024-05-11T08:05:49.239 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-05-11T08:05:49.239 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-05-11T08:05:49.240 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-11T08:05:49.240 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-11T08:05:49.242 INFO:tasks.ceph.mds.a:Started 2024-05-11T08:05:49.242 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-11T08:05:49.242 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-11T08:05:49.244 INFO:tasks.ceph.mds.c:Started 2024-05-11T08:05:49.244 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-11T08:05:49.244 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-11T08:05:49.247 INFO:tasks.ceph.mds.b:Started 2024-05-11T08:05:49.247 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-11T08:05:49.247 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-11T08:05:49.249 INFO:tasks.ceph.mds.d:Started 2024-05-11T08:05:49.249 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-05-11T08:05:49.250 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-11T08:05:49.317 INFO:tasks.ceph.mds.a.smithi167.stdout:starting mds.a at 2024-05-11T08:05:49.318 INFO:tasks.ceph.mds.c.smithi167.stdout:starting mds.c at 2024-05-11T08:05:49.318 INFO:tasks.ceph.mds.c.smithi167.stderr:2024-05-11T08:05:49.318+0000 7fa97194c640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2024-05-11T08:05:49.333 INFO:tasks.ceph.mds.d.smithi204.stdout:starting mds.d at 2024-05-11T08:05:49.333 INFO:tasks.ceph.mds.b.smithi204.stdout:starting mds.b at 2024-05-11T08:05:49.355 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:49.355+0000 7fa436c07640 1 Processor -- start 2024-05-11T08:05:49.355 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:49.355+0000 7fa436c07640 1 -- start start 2024-05-11T08:05:49.357 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:49.356+0000 7fa436c07640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa4300736a0 0x7fa430072b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:05:49.357 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:49.356+0000 7fa436c07640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa43015bc40 con 0x7fa43015b860 2024-05-11T08:05:49.357 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:49.356+0000 7fa436c07640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa43006e410 con 0x7fa430072640 2024-05-11T08:05:49.357 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:49.356+0000 7fa436c07640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fa43006db70 con 0x7fa4300736a0 2024-05-11T08:05:49.357 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:49.356+0000 7fa4349a3640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7fa43015b860 0x7fa43006f4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:42668/0 (socket says 172.21.15.167:42668) 2024-05-11T08:05:49.357 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:49.356+0000 7fa4349a3640 1 -- 172.21.15.167:0/304015862 learned_addr learned my addr 172.21.15.167:0/304015862 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T08:05:49.357 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:49.357+0000 7fa427fff640 1 --2- 172.21.15.167:0/304015862 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa4300736a0 0x7fa430072b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:05:49.357 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:49.357+0000 7fa4277fe640 1 -- 172.21.15.167:0/304015862 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3701235589 0 0) 0x7fa42c001110 con 0x7fa43015b860 2024-05-11T08:05:49.357 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:49.357+0000 7fa4277fe640 1 -- 172.21.15.167:0/304015862 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa418003520 con 0x7fa43015b860 2024-05-11T08:05:49.357 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:49.357+0000 7fa4277fe640 1 -- 172.21.15.167:0/304015862 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 754252537 0 0) 0x7fa43006e410 con 0x7fa430072640 2024-05-11T08:05:49.358 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:49.357+0000 7fa4277fe640 1 -- 172.21.15.167:0/304015862 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa42c001110 con 0x7fa430072640 2024-05-11T08:05:49.358 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:49.357+0000 7fa427fff640 1 -- 172.21.15.167:0/304015862 >> v1:172.21.15.204:6789/0 conn(0x7fa430072640 legacy=0x7fa43006d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:05:49.358 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:49.357+0000 7fa427fff640 1 -- 172.21.15.167:0/304015862 >> v1:172.21.15.167:6789/0 conn(0x7fa43015b860 legacy=0x7fa43006f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:05:49.358 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:49.357+0000 7fa427fff640 1 -- 172.21.15.167:0/304015862 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa43006e6f0 con 0x7fa4300736a0 2024-05-11T08:05:49.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:49.358+0000 7fa427fff640 1 --2- 172.21.15.167:0/304015862 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa4300736a0 0x7fa430072b00 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fa4200097d0 tx=0x7fa42002a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e9cf55a2aec4aa70 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:05:49.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:49.358+0000 7fa4277fe640 1 -- 172.21.15.167:0/304015862 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa420037070 con 0x7fa4300736a0 2024-05-11T08:05:49.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:49.358+0000 7fa4277fe640 1 -- 172.21.15.167:0/304015862 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa420032440 con 0x7fa4300736a0 2024-05-11T08:05:49.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:49.358+0000 7fa4277fe640 1 -- 172.21.15.167:0/304015862 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4200327f0 con 0x7fa4300736a0 2024-05-11T08:05:49.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:49.358+0000 7fa436c07640 1 -- 172.21.15.167:0/304015862 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa4300736a0 msgr2=0x7fa430072b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:05:49.360 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:49.358+0000 7fa436c07640 1 --2- 172.21.15.167:0/304015862 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa4300736a0 0x7fa430072b00 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fa4200097d0 tx=0x7fa42002a210 comp rx=0 tx=0).stop 2024-05-11T08:05:49.360 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:49.358+0000 7fa436c07640 1 -- 172.21.15.167:0/304015862 shutdown_connections 2024-05-11T08:05:49.360 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:49.358+0000 7fa436c07640 1 --2- 172.21.15.167:0/304015862 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa4300736a0 0x7fa430072b00 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:49.360 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:49.358+0000 7fa436c07640 1 -- 172.21.15.167:0/304015862 >> 172.21.15.167:0/304015862 conn(0x7fa4300cd8f0 msgr2=0x7fa4300cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:05:49.360 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:49.358+0000 7fa436c07640 1 -- 172.21.15.167:0/304015862 shutdown_connections 2024-05-11T08:05:49.360 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:49.358+0000 7fa436c07640 1 -- 172.21.15.167:0/304015862 wait complete. 2024-05-11T08:05:49.360 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:49.359+0000 7fa436c07640 1 Processor -- start 2024-05-11T08:05:49.360 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:49.359+0000 7fa436c07640 1 -- start start 2024-05-11T08:05:49.362 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:49.360+0000 7fa436c07640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa430072640 0x7fa4301f5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:05:49.362 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:49.360+0000 7fa436c07640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa4300736a0 0x7fa4301505b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:05:49.362 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:49.360+0000 7fa436c07640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa43015b860 0x7fa4301f8350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:05:49.363 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:49.360+0000 7fa436c07640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa43006ef90 con 0x7fa430072640 2024-05-11T08:05:49.363 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:49.360+0000 7fa436c07640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa43006ecb0 con 0x7fa4300736a0 2024-05-11T08:05:49.363 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:49.360+0000 7fa436c07640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fa43006fbc0 con 0x7fa43015b860 2024-05-11T08:05:49.363 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:49.360+0000 7fa4349a3640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa430072640 0x7fa4301f5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:05:49.363 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:49.360+0000 7fa4349a3640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa430072640 0x7fa4301f5a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:42470/0 (socket says 172.21.15.167:42470) 2024-05-11T08:05:49.363 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:49.360+0000 7fa4349a3640 1 -- 172.21.15.167:0/3967905795 learned_addr learned my addr 172.21.15.167:0/3967905795 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:05:49.363 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:49.360+0000 7fa4351a4640 1 --2- 172.21.15.167:0/3967905795 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa43015b860 0x7fa4301f8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:05:49.363 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:49.361+0000 7fa427fff640 1 --2- 172.21.15.167:0/3967905795 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa4300736a0 0x7fa4301505b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:05:49.363 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:49.361+0000 7fa4349a3640 1 -- 172.21.15.167:0/3967905795 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa43015b860 msgr2=0x7fa4301f8350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:05:49.363 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:49.361+0000 7fa4349a3640 1 --2- 172.21.15.167:0/3967905795 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa43015b860 0x7fa4301f8350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:49.363 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:49.361+0000 7fa4349a3640 1 -- 172.21.15.167:0/3967905795 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa4300736a0 msgr2=0x7fa4301505b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:05:49.363 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:49.361+0000 7fa4349a3640 1 --2- 172.21.15.167:0/3967905795 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa4300736a0 0x7fa4301505b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:49.363 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:49.361+0000 7fa4349a3640 1 -- 172.21.15.167:0/3967905795 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa430159c30 con 0x7fa430072640 2024-05-11T08:05:49.363 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:49.361+0000 7fa4349a3640 1 --2- 172.21.15.167:0/3967905795 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa430072640 0x7fa4301f5a20 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa42c00c730 tx=0x7fa42c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:05:49.364 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:49.361+0000 7fa4257fa640 1 -- 172.21.15.167:0/3967905795 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa42c017070 con 0x7fa430072640 2024-05-11T08:05:49.364 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:49.361+0000 7fa436c07640 1 -- 172.21.15.167:0/3967905795 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa43014cc00 con 0x7fa430072640 2024-05-11T08:05:49.364 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:49.361+0000 7fa436c07640 1 -- 172.21.15.167:0/3967905795 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa430159f10 con 0x7fa430072640 2024-05-11T08:05:49.364 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:49.361+0000 7fa4257fa640 1 -- 172.21.15.167:0/3967905795 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa42c005ab0 con 0x7fa430072640 2024-05-11T08:05:49.364 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:49.361+0000 7fa4257fa640 1 -- 172.21.15.167:0/3967905795 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa42c007070 con 0x7fa430072640 2024-05-11T08:05:49.364 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:49.362+0000 7fa436c07640 1 -- 172.21.15.167:0/3967905795 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa410005290 con 0x7fa430072640 2024-05-11T08:05:49.364 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:49.363+0000 7fa4257fa640 1 -- 172.21.15.167:0/3967905795 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fa42c01a070 con 0x7fa430072640 2024-05-11T08:05:49.364 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:49.363+0000 7fa4257fa640 1 --2- 172.21.15.167:0/3967905795 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fa41c0762d0 0x7fa41c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:05:49.365 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:49.363+0000 7fa4257fa640 1 -- 172.21.15.167:0/3967905795 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4857+0+0 (secure 0 0 0) 0x7fa42c096830 con 0x7fa430072640 2024-05-11T08:05:49.365 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:49.363+0000 7fa427fff640 1 --2- 172.21.15.167:0/3967905795 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fa41c0762d0 0x7fa41c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:05:49.365 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:49.363+0000 7fa427fff640 1 --2- 172.21.15.167:0/3967905795 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fa41c0762d0 0x7fa41c078790 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa420034000 tx=0x7fa42002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:05:49.367 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:49.367+0000 7fa4257fa640 1 -- 172.21.15.167:0/3967905795 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fa43015bc40 con 0x7fa430072640 2024-05-11T08:05:49.586 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:49.586+0000 7fa436c07640 1 -- 172.21.15.167:0/3967905795 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa410005850 con 0x7fa430072640 2024-05-11T08:05:49.587 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:49.587+0000 7fa4257fa640 1 -- 172.21.15.167:0/3967905795 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1392 (secure 0 0 0) 0x7fa42c05fea0 con 0x7fa430072640 2024-05-11T08:05:49.587 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T08:05:49.587 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4290,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/766834562","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":766834562},{"type":"v1","addr":"172.21.15.167:6836","nonce":766834562}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-05-11T08:05:49.587 INFO:teuthology.orchestra.run.smithi167.stderr:dumped fsmap epoch 2 2024-05-11T08:05:49.589 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:49.589+0000 7fa436c07640 1 -- 172.21.15.167:0/3967905795 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fa41c0762d0 msgr2=0x7fa41c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:05:49.589 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:49.589+0000 7fa436c07640 1 --2- 172.21.15.167:0/3967905795 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fa41c0762d0 0x7fa41c078790 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa420034000 tx=0x7fa42002e000 comp rx=0 tx=0).stop 2024-05-11T08:05:49.589 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:49.589+0000 7fa436c07640 1 -- 172.21.15.167:0/3967905795 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa430072640 msgr2=0x7fa4301f5a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:05:49.589 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:49.589+0000 7fa436c07640 1 --2- 172.21.15.167:0/3967905795 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa430072640 0x7fa4301f5a20 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa42c00c730 tx=0x7fa42c00cbf0 comp rx=0 tx=0).stop 2024-05-11T08:05:49.589 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:49.589+0000 7fa436c07640 1 -- 172.21.15.167:0/3967905795 shutdown_connections 2024-05-11T08:05:49.590 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:49.589+0000 7fa436c07640 1 --2- 172.21.15.167:0/3967905795 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fa41c0762d0 0x7fa41c078790 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:49.590 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:49.589+0000 7fa436c07640 1 --2- 172.21.15.167:0/3967905795 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa43015b860 0x7fa4301f8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:49.590 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:49.589+0000 7fa436c07640 1 --2- 172.21.15.167:0/3967905795 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa4300736a0 0x7fa4301505b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:49.590 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:49.589+0000 7fa436c07640 1 --2- 172.21.15.167:0/3967905795 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa430072640 0x7fa4301f5a20 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:49.590 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:49.589+0000 7fa436c07640 1 -- 172.21.15.167:0/3967905795 >> 172.21.15.167:0/3967905795 conn(0x7fa4300cd8f0 msgr2=0x7fa4300b2490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:05:49.590 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:49.589+0000 7fa436c07640 1 -- 172.21.15.167:0/3967905795 shutdown_connections 2024-05-11T08:05:49.590 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:49.589+0000 7fa436c07640 1 -- 172.21.15.167:0/3967905795 wait complete. 2024-05-11T08:05:51.600 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-11T08:05:51.713 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:51.713+0000 7f2357d91640 1 Processor -- start 2024-05-11T08:05:51.713 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:51.713+0000 7f2357d91640 1 -- start start 2024-05-11T08:05:51.715 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:51.714+0000 7f2357d91640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f23500b5e90 0x7f235015f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:05:51.715 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:51.714+0000 7f2357d91640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f235015fc80 0x7f23500cd610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:05:51.716 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:51.714+0000 7f2357d91640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23500d2410 con 0x7f235015f360 2024-05-11T08:05:51.716 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:51.714+0000 7f2357d91640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f23500d26f0 con 0x7f235015fc80 2024-05-11T08:05:51.716 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:51.714+0000 7f2357d91640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f23500d29d0 con 0x7f23500b5e90 2024-05-11T08:05:51.716 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:51.714+0000 7f235632e640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f235015fc80 0x7f23500cd610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:05:51.716 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:51.714+0000 7f235632e640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f235015fc80 0x7f23500cd610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:42482/0 (socket says 172.21.15.167:42482) 2024-05-11T08:05:51.716 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:51.714+0000 7f235632e640 1 -- 172.21.15.167:0/3119162691 learned_addr learned my addr 172.21.15.167:0/3119162691 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:05:51.716 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:51.714+0000 7f2355b2d640 1 --1- 172.21.15.167:0/3119162691 >> v1:172.21.15.167:6789/0 conn(0x7f235015f360 0x7f23500cfdc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:42676/0 (socket says 172.21.15.167:42676) 2024-05-11T08:05:51.716 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:51.715+0000 7f235532c640 1 --2- 172.21.15.167:0/3119162691 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f23500b5e90 0x7f235015f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:05:51.716 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:51.715+0000 7f235632e640 1 -- 172.21.15.167:0/3119162691 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f23500b5e90 msgr2=0x7f235015f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:05:51.716 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:51.715+0000 7f235632e640 1 --2- 172.21.15.167:0/3119162691 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f23500b5e90 0x7f235015f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:51.716 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:51.715+0000 7f235632e640 1 -- 172.21.15.167:0/3119162691 >> v1:172.21.15.167:6789/0 conn(0x7f235015f360 legacy=0x7f23500cfdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:05:51.716 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:51.715+0000 7f235632e640 1 -- 172.21.15.167:0/3119162691 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23500b7e20 con 0x7f235015fc80 2024-05-11T08:05:51.716 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:51.715+0000 7f2354b2b640 1 -- 172.21.15.167:0/3119162691 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4072762165 0 0) 0x7f23500d2410 con 0x7f235015f360 2024-05-11T08:05:51.717 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:51.715+0000 7f235632e640 1 --2- 172.21.15.167:0/3119162691 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f235015fc80 0x7f23500cd610 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2348009e50 tx=0x7f234802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb913c1089bba273 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:05:51.717 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:51.716+0000 7f2354b2b640 1 -- 172.21.15.167:0/3119162691 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2348036070 con 0x7f235015fc80 2024-05-11T08:05:51.717 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:51.716+0000 7f2354b2b640 1 -- 172.21.15.167:0/3119162691 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2348031600 con 0x7f235015fc80 2024-05-11T08:05:51.717 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:51.716+0000 7f2354b2b640 1 -- 172.21.15.167:0/3119162691 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23480319d0 con 0x7f235015fc80 2024-05-11T08:05:51.720 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:51.716+0000 7f2357d91640 1 -- 172.21.15.167:0/3119162691 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f235015fc80 msgr2=0x7f23500cd610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:05:51.720 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:51.716+0000 7f2357d91640 1 --2- 172.21.15.167:0/3119162691 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f235015fc80 0x7f23500cd610 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2348009e50 tx=0x7f234802a680 comp rx=0 tx=0).stop 2024-05-11T08:05:51.720 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:51.716+0000 7f2357d91640 1 -- 172.21.15.167:0/3119162691 shutdown_connections 2024-05-11T08:05:51.720 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:51.716+0000 7f2357d91640 1 --2- 172.21.15.167:0/3119162691 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f235015fc80 0x7f23500cd610 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:51.720 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:51.716+0000 7f2357d91640 1 --2- 172.21.15.167:0/3119162691 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f23500b5e90 0x7f235015f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:51.720 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:51.716+0000 7f2357d91640 1 -- 172.21.15.167:0/3119162691 >> 172.21.15.167:0/3119162691 conn(0x7f2350072730 msgr2=0x7f23500d2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:05:51.720 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:51.716+0000 7f2357d91640 1 -- 172.21.15.167:0/3119162691 shutdown_connections 2024-05-11T08:05:51.720 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:51.716+0000 7f2357d91640 1 -- 172.21.15.167:0/3119162691 wait complete. 2024-05-11T08:05:51.720 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:51.717+0000 7f2357d91640 1 Processor -- start 2024-05-11T08:05:51.720 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:51.717+0000 7f2357d91640 1 -- start start 2024-05-11T08:05:51.720 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:51.718+0000 7f2357d91640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f23500b5e90 0x7f235014d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:05:51.720 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:51.718+0000 7f2357d91640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f235015f360 0x7f23500b4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:05:51.720 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:51.718+0000 7f2357d91640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f235015fc80 0x7f23500b5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:05:51.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:51.718+0000 7f2357d91640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f23500cdcd0 con 0x7f23500b5e90 2024-05-11T08:05:51.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:51.718+0000 7f2357d91640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f23500cdb50 con 0x7f235015fc80 2024-05-11T08:05:51.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:51.718+0000 7f2357d91640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f23500cde50 con 0x7f235015f360 2024-05-11T08:05:51.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:51.718+0000 7f2355b2d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f23500b5e90 0x7f235014d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:05:51.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:51.718+0000 7f2355b2d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f23500b5e90 0x7f235014d390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:42486/0 (socket says 172.21.15.167:42486) 2024-05-11T08:05:51.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:51.718+0000 7f2355b2d640 1 -- 172.21.15.167:0/1670102409 learned_addr learned my addr 172.21.15.167:0/1670102409 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:05:51.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:51.718+0000 7f235532c640 1 --2- 172.21.15.167:0/1670102409 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f235015f360 0x7f23500b4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:05:51.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:51.718+0000 7f235632e640 1 --2- 172.21.15.167:0/1670102409 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f235015fc80 0x7f23500b5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:05:51.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:51.719+0000 7f2355b2d640 1 -- 172.21.15.167:0/1670102409 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f235015f360 msgr2=0x7f23500b4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:05:51.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:51.719+0000 7f2355b2d640 1 --2- 172.21.15.167:0/1670102409 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f235015f360 0x7f23500b4060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:51.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:51.719+0000 7f2355b2d640 1 -- 172.21.15.167:0/1670102409 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f235015fc80 msgr2=0x7f23500b5080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:05:51.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:51.719+0000 7f2355b2d640 1 --2- 172.21.15.167:0/1670102409 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f235015fc80 0x7f23500b5080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:51.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:51.719+0000 7f2355b2d640 1 -- 172.21.15.167:0/1670102409 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f235014d8d0 con 0x7f23500b5e90 2024-05-11T08:05:51.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:51.719+0000 7f2355b2d640 1 --2- 172.21.15.167:0/1670102409 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f23500b5e90 0x7f235014d390 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f234c00c490 tx=0x7f234c00c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:05:51.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:51.719+0000 7f23467fc640 1 -- 172.21.15.167:0/1670102409 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f234c018070 con 0x7f23500b5e90 2024-05-11T08:05:51.722 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:51.719+0000 7f23467fc640 1 -- 172.21.15.167:0/1670102409 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f234c007500 con 0x7f23500b5e90 2024-05-11T08:05:51.722 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:51.719+0000 7f2357d91640 1 -- 172.21.15.167:0/1670102409 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f235014de20 con 0x7f23500b5e90 2024-05-11T08:05:51.722 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:51.719+0000 7f23467fc640 1 -- 172.21.15.167:0/1670102409 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f234c004a20 con 0x7f23500b5e90 2024-05-11T08:05:51.722 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:51.719+0000 7f2357d91640 1 -- 172.21.15.167:0/1670102409 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f235014e3e0 con 0x7f23500b5e90 2024-05-11T08:05:51.722 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:51.720+0000 7f2357d91640 1 -- 172.21.15.167:0/1670102409 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2350069590 con 0x7f23500b5e90 2024-05-11T08:05:51.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:51.721+0000 7f23467fc640 1 -- 172.21.15.167:0/1670102409 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f234c01b070 con 0x7f23500b5e90 2024-05-11T08:05:51.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:51.721+0000 7f23467fc640 1 --2- 172.21.15.167:0/1670102409 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f23340762d0 0x7f2334078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:05:51.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:51.721+0000 7f235532c640 1 --2- 172.21.15.167:0/1670102409 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f23340762d0 0x7f2334078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:05:51.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:51.721+0000 7f23467fc640 1 -- 172.21.15.167:0/1670102409 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4857+0+0 (secure 0 0 0) 0x7f234c0964b0 con 0x7f23500b5e90 2024-05-11T08:05:51.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:51.722+0000 7f235532c640 1 --2- 172.21.15.167:0/1670102409 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f23340762d0 0x7f2334078790 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f23500ccb10 tx=0x7f2340008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:05:51.726 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:51.726+0000 7f23467fc640 1 -- 172.21.15.167:0/1670102409 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f234c05fb10 con 0x7f23500b5e90 2024-05-11T08:05:51.948 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:51.948+0000 7f2357d91640 1 -- 172.21.15.167:0/1670102409 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f23500d32e0 con 0x7f23500b5e90 2024-05-11T08:05:51.949 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:51.949+0000 7f23467fc640 1 -- 172.21.15.167:0/1670102409 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+3870 (secure 0 0 0) 0x7f234c063e10 con 0x7f23500b5e90 2024-05-11T08:05:51.949 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T08:05:51.949 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6834/506789040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":506789040},{"type":"v1","addr":"172.21.15.204:6834","nonce":506789040}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4287,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1448126000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":1448126000},{"type":"v1","addr":"172.21.15.167:6837","nonce":1448126000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4290,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/766834562","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":766834562},{"type":"v1","addr":"172.21.15.167:6836","nonce":766834562}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4305,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/1247774569","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6833","nonce":1247774569},{"type":"v1","addr":"172.21.15.204:6835","nonce":1247774569}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-05-11T08:05:51.949 INFO:teuthology.orchestra.run.smithi167.stderr:dumped fsmap epoch 3 2024-05-11T08:05:51.951 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:51.951+0000 7f2357d91640 1 -- 172.21.15.167:0/1670102409 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f23340762d0 msgr2=0x7f2334078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:05:51.951 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:51.951+0000 7f2357d91640 1 --2- 172.21.15.167:0/1670102409 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f23340762d0 0x7f2334078790 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f23500ccb10 tx=0x7f2340008040 comp rx=0 tx=0).stop 2024-05-11T08:05:51.951 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:51.951+0000 7f2357d91640 1 -- 172.21.15.167:0/1670102409 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f23500b5e90 msgr2=0x7f235014d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:05:51.951 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:51.951+0000 7f2357d91640 1 --2- 172.21.15.167:0/1670102409 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f23500b5e90 0x7f235014d390 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f234c00c490 tx=0x7f234c00c950 comp rx=0 tx=0).stop 2024-05-11T08:05:51.951 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:51.951+0000 7f2357d91640 1 -- 172.21.15.167:0/1670102409 shutdown_connections 2024-05-11T08:05:51.952 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:51.951+0000 7f2357d91640 1 --2- 172.21.15.167:0/1670102409 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f23340762d0 0x7f2334078790 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:51.952 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:51.951+0000 7f2357d91640 1 --2- 172.21.15.167:0/1670102409 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f235015fc80 0x7f23500b5080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:51.952 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:51.951+0000 7f2357d91640 1 --2- 172.21.15.167:0/1670102409 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f235015f360 0x7f23500b4060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:51.952 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:51.951+0000 7f2357d91640 1 --2- 172.21.15.167:0/1670102409 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f23500b5e90 0x7f235014d390 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:51.952 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:51.951+0000 7f2357d91640 1 -- 172.21.15.167:0/1670102409 >> 172.21.15.167:0/1670102409 conn(0x7f2350072730 msgr2=0x7f23500ceae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:05:51.952 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:51.951+0000 7f2357d91640 1 -- 172.21.15.167:0/1670102409 shutdown_connections 2024-05-11T08:05:51.952 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:51.951+0000 7f2357d91640 1 -- 172.21.15.167:0/1670102409 wait complete. 2024-05-11T08:05:51.962 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-11T08:05:52.063 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:52.063+0000 7f9c4eac4640 1 Processor -- start 2024-05-11T08:05:52.064 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:52.063+0000 7f9c4eac4640 1 -- start start 2024-05-11T08:05:52.067 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:52.064+0000 7f9c4eac4640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f9c4815f2a0 0x7f9c480b6d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:05:52.067 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:52.064+0000 7f9c4eac4640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f9c480b5f20 0x7f9c4806c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:05:52.067 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:52.064+0000 7f9c4eac4640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c480b1af0 con 0x7f9c480b1580 2024-05-11T08:05:52.067 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:52.064+0000 7f9c4eac4640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f9c480b1dd0 con 0x7f9c4815f2a0 2024-05-11T08:05:52.067 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:52.064+0000 7f9c4eac4640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f9c480b20b0 con 0x7f9c480b5f20 2024-05-11T08:05:52.067 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:52.064+0000 7f9c4c860640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f9c4815f2a0 0x7f9c480b6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:05:52.067 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:52.064+0000 7f9c4d061640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f9c480b1580 0x7f9c48162ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:42684/0 (socket says 172.21.15.167:42684) 2024-05-11T08:05:52.067 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:52.064+0000 7f9c4d061640 1 -- 172.21.15.167:0/255775288 learned_addr learned my addr 172.21.15.167:0/255775288 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T08:05:52.067 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:52.065+0000 7f9c4c860640 1 -- 172.21.15.167:0/255775288 >> v2:172.21.15.204:3300/0 conn(0x7f9c480b5f20 msgr2=0x7f9c4806c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:05:52.067 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:52.065+0000 7f9c47fff640 1 --2- 172.21.15.167:0/255775288 >> v2:172.21.15.204:3300/0 conn(0x7f9c480b5f20 0x7f9c4806c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:05:52.068 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:52.065+0000 7f9c4c860640 1 --2- 172.21.15.167:0/255775288 >> v2:172.21.15.204:3300/0 conn(0x7f9c480b5f20 0x7f9c4806c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:52.068 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:52.065+0000 7f9c4c860640 1 -- 172.21.15.167:0/255775288 >> v1:172.21.15.167:6789/0 conn(0x7f9c480b1580 legacy=0x7f9c48162ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:05:52.068 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:52.065+0000 7f9c4c860640 1 -- 172.21.15.167:0/255775288 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c480b2390 con 0x7f9c4815f2a0 2024-05-11T08:05:52.068 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:52.065+0000 7f9c477fe640 1 -- 172.21.15.167:0/255775288 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3750867976 0 0) 0x7f9c480b1af0 con 0x7f9c480b1580 2024-05-11T08:05:52.068 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:52.065+0000 7f9c4c860640 1 --2- 172.21.15.167:0/255775288 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9c4815f2a0 0x7f9c480b6d40 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f9c480b7280 tx=0x7f9c3802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef08a227037c9b99 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:05:52.068 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:52.065+0000 7f9c477fe640 1 -- 172.21.15.167:0/255775288 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c38037070 con 0x7f9c4815f2a0 2024-05-11T08:05:52.068 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:52.065+0000 7f9c477fe640 1 -- 172.21.15.167:0/255775288 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c38032440 con 0x7f9c4815f2a0 2024-05-11T08:05:52.068 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:52.065+0000 7f9c477fe640 1 -- 172.21.15.167:0/255775288 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c38032810 con 0x7f9c4815f2a0 2024-05-11T08:05:52.068 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:52.066+0000 7f9c4eac4640 1 -- 172.21.15.167:0/255775288 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9c4815f2a0 msgr2=0x7f9c480b6d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:05:52.068 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:52.066+0000 7f9c4eac4640 1 --2- 172.21.15.167:0/255775288 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9c4815f2a0 0x7f9c480b6d40 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f9c480b7280 tx=0x7f9c3802a1a0 comp rx=0 tx=0).stop 2024-05-11T08:05:52.068 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:52.066+0000 7f9c4eac4640 1 -- 172.21.15.167:0/255775288 shutdown_connections 2024-05-11T08:05:52.069 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:52.066+0000 7f9c4eac4640 1 --2- 172.21.15.167:0/255775288 >> v2:172.21.15.204:3300/0 conn(0x7f9c480b5f20 0x7f9c4806c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:52.069 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:52.066+0000 7f9c4eac4640 1 --2- 172.21.15.167:0/255775288 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9c4815f2a0 0x7f9c480b6d40 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:52.069 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:52.066+0000 7f9c4eac4640 1 -- 172.21.15.167:0/255775288 >> 172.21.15.167:0/255775288 conn(0x7f9c480738c0 msgr2=0x7f9c48073cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:05:52.069 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:52.066+0000 7f9c4eac4640 1 -- 172.21.15.167:0/255775288 shutdown_connections 2024-05-11T08:05:52.069 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:52.066+0000 7f9c4eac4640 1 -- 172.21.15.167:0/255775288 wait complete. 2024-05-11T08:05:52.072 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:52.067+0000 7f9c4eac4640 1 Processor -- start 2024-05-11T08:05:52.072 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:52.067+0000 7f9c4eac4640 1 -- start start 2024-05-11T08:05:52.072 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:52.068+0000 7f9c4eac4640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9c480b1580 0x7f9c480afb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:05:52.072 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:52.068+0000 7f9c4eac4640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9c480b5f20 0x7f9c4809a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:05:52.072 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:52.068+0000 7f9c4eac4640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9c4815f2a0 0x7f9c4809aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:05:52.073 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:52.068+0000 7f9c4eac4640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9c4815f800 con 0x7f9c4815f2a0 2024-05-11T08:05:52.073 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:52.068+0000 7f9c4eac4640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f9c4815f680 con 0x7f9c480b1580 2024-05-11T08:05:52.073 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:52.068+0000 7f9c4eac4640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f9c480aefb0 con 0x7f9c480b5f20 2024-05-11T08:05:52.073 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:52.068+0000 7f9c4d061640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9c4815f2a0 0x7f9c4809aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:05:52.073 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:52.068+0000 7f9c4d061640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9c4815f2a0 0x7f9c4809aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:42518/0 (socket says 172.21.15.167:42518) 2024-05-11T08:05:52.073 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:52.068+0000 7f9c4d061640 1 -- 172.21.15.167:0/687823779 learned_addr learned my addr 172.21.15.167:0/687823779 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:05:52.073 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:52.068+0000 7f9c4c860640 1 --2- 172.21.15.167:0/687823779 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9c480b1580 0x7f9c480afb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:05:52.073 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:52.068+0000 7f9c47fff640 1 --2- 172.21.15.167:0/687823779 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9c480b5f20 0x7f9c4809a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:05:52.073 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:52.069+0000 7f9c4d061640 1 -- 172.21.15.167:0/687823779 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9c480b5f20 msgr2=0x7f9c4809a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:05:52.073 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:52.069+0000 7f9c4d061640 1 --2- 172.21.15.167:0/687823779 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9c480b5f20 0x7f9c4809a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:52.073 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:52.069+0000 7f9c4d061640 1 -- 172.21.15.167:0/687823779 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9c480b1580 msgr2=0x7f9c480afb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:05:52.074 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:52.069+0000 7f9c4d061640 1 --2- 172.21.15.167:0/687823779 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9c480b1580 0x7f9c480afb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:52.074 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:52.069+0000 7f9c4d061640 1 -- 172.21.15.167:0/687823779 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c481542d0 con 0x7f9c4815f2a0 2024-05-11T08:05:52.074 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:52.069+0000 7f9c47fff640 1 --2- 172.21.15.167:0/687823779 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9c480b5f20 0x7f9c4809a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T08:05:52.074 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:52.069+0000 7f9c4d061640 1 --2- 172.21.15.167:0/687823779 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9c4815f2a0 0x7f9c4809aa10 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9c4000c440 tx=0x7f9c4000c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:05:52.074 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:52.069+0000 7f9c457fa640 1 -- 172.21.15.167:0/687823779 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c40018070 con 0x7f9c4815f2a0 2024-05-11T08:05:52.074 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:52.069+0000 7f9c457fa640 1 -- 172.21.15.167:0/687823779 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c400074e0 con 0x7f9c4815f2a0 2024-05-11T08:05:52.074 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:52.069+0000 7f9c457fa640 1 -- 172.21.15.167:0/687823779 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c40004a20 con 0x7f9c4815f2a0 2024-05-11T08:05:52.074 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:52.069+0000 7f9c4eac4640 1 -- 172.21.15.167:0/687823779 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c4815f9d0 con 0x7f9c4815f2a0 2024-05-11T08:05:52.074 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:52.069+0000 7f9c4eac4640 1 -- 172.21.15.167:0/687823779 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c4815ff90 con 0x7f9c4815f2a0 2024-05-11T08:05:52.074 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:52.070+0000 7f9c4eac4640 1 -- 172.21.15.167:0/687823779 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c48069bc0 con 0x7f9c4815f2a0 2024-05-11T08:05:52.075 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:52.070+0000 7f9c457fa640 1 -- 172.21.15.167:0/687823779 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f9c4001b070 con 0x7f9c4815f2a0 2024-05-11T08:05:52.075 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:52.071+0000 7f9c457fa640 1 --2- 172.21.15.167:0/687823779 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f9c340762d0 0x7f9c34078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:05:52.075 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:52.071+0000 7f9c4c860640 1 --2- 172.21.15.167:0/687823779 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f9c340762d0 0x7f9c34078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:05:52.075 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:52.071+0000 7f9c457fa640 1 -- 172.21.15.167:0/687823779 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4857+0+0 (secure 0 0 0) 0x7f9c40096420 con 0x7f9c4815f2a0 2024-05-11T08:05:52.075 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:52.071+0000 7f9c4c860640 1 --2- 172.21.15.167:0/687823779 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f9c340762d0 0x7f9c34078790 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f9c38034000 tx=0x7f9c3802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:05:52.075 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:52.075+0000 7f9c457fa640 1 -- 172.21.15.167:0/687823779 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f9c4005fa80 con 0x7f9c4815f2a0 2024-05-11T08:05:52.256 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:52.256+0000 7f9c4eac4640 1 -- 172.21.15.167:0/687823779 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f9c4806f8c0 con 0x7f9c4815f2a0 2024-05-11T08:05:52.256 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:52.256+0000 7f9c457fa640 1 -- 172.21.15.167:0/687823779 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v19) ==== 84+0+65 (secure 0 0 0) 0x7f9c40063d80 con 0x7f9c4815f2a0 2024-05-11T08:05:52.256 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T08:05:52.257 INFO:teuthology.orchestra.run.smithi167.stdout:[ 2024-05-11T08:05:52.257 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T08:05:52.257 INFO:teuthology.orchestra.run.smithi167.stdout: "poolnum": 1, 2024-05-11T08:05:52.257 INFO:teuthology.orchestra.run.smithi167.stdout: "poolname": ".mgr" 2024-05-11T08:05:52.257 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T08:05:52.257 INFO:teuthology.orchestra.run.smithi167.stdout:] 2024-05-11T08:05:52.259 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:52.258+0000 7f9c4eac4640 1 -- 172.21.15.167:0/687823779 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f9c340762d0 msgr2=0x7f9c34078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:05:52.259 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:52.258+0000 7f9c4eac4640 1 --2- 172.21.15.167:0/687823779 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f9c340762d0 0x7f9c34078790 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f9c38034000 tx=0x7f9c3802e000 comp rx=0 tx=0).stop 2024-05-11T08:05:52.259 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:52.258+0000 7f9c4eac4640 1 -- 172.21.15.167:0/687823779 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9c4815f2a0 msgr2=0x7f9c4809aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:05:52.259 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:52.258+0000 7f9c4eac4640 1 --2- 172.21.15.167:0/687823779 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9c4815f2a0 0x7f9c4809aa10 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9c4000c440 tx=0x7f9c4000c900 comp rx=0 tx=0).stop 2024-05-11T08:05:52.259 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:52.258+0000 7f9c4eac4640 1 -- 172.21.15.167:0/687823779 shutdown_connections 2024-05-11T08:05:52.259 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:52.258+0000 7f9c4eac4640 1 --2- 172.21.15.167:0/687823779 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f9c340762d0 0x7f9c34078790 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:52.259 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:52.258+0000 7f9c4eac4640 1 --2- 172.21.15.167:0/687823779 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9c4815f2a0 0x7f9c4809aa10 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:52.259 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:52.258+0000 7f9c4eac4640 1 --2- 172.21.15.167:0/687823779 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9c480b5f20 0x7f9c4809a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:52.259 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:52.258+0000 7f9c4eac4640 1 --2- 172.21.15.167:0/687823779 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9c480b1580 0x7f9c480afb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:52.259 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:52.258+0000 7f9c4eac4640 1 -- 172.21.15.167:0/687823779 >> 172.21.15.167:0/687823779 conn(0x7f9c480738c0 msgr2=0x7f9c480d1610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:05:52.259 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:52.258+0000 7f9c4eac4640 1 -- 172.21.15.167:0/687823779 shutdown_connections 2024-05-11T08:05:52.260 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:52.258+0000 7f9c4eac4640 1 -- 172.21.15.167:0/687823779 wait complete. 2024-05-11T08:05:52.269 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-11T08:05:52.269 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-11T08:05:52.374 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:52.374+0000 7fb15ac99640 1 Processor -- start 2024-05-11T08:05:52.374 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:52.374+0000 7fb15ac99640 1 -- start start 2024-05-11T08:05:52.375 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:52.375+0000 7fb15ac99640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fb15406fc30 0x7fb154070090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:05:52.375 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:52.375+0000 7fb15ac99640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb154072150 0x7fb1540725c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:05:52.376 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:52.375+0000 7fb15ac99640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb15406eb90 con 0x7fb154142740 2024-05-11T08:05:52.376 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:52.375+0000 7fb15ac99640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fb15406f7a0 con 0x7fb15406fc30 2024-05-11T08:05:52.376 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:52.375+0000 7fb15ac99640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fb15406ee70 con 0x7fb154072150 2024-05-11T08:05:52.376 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:52.375+0000 7fb159236640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb154072150 0x7fb1540725c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:05:52.376 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:52.375+0000 7fb14bfff640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fb15406fc30 0x7fb154070090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:05:52.376 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:52.375+0000 7fb159236640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb154072150 0x7fb1540725c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.167:36672/0 (socket says 172.21.15.167:36672) 2024-05-11T08:05:52.376 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:52.375+0000 7fb159236640 1 -- 172.21.15.167:0/3341661945 learned_addr learned my addr 172.21.15.167:0/3341661945 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:05:52.376 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:52.375+0000 7fb159236640 1 -- 172.21.15.167:0/3341661945 >> v2:172.21.15.204:3300/0 conn(0x7fb15406fc30 msgr2=0x7fb154070090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:05:52.376 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:52.375+0000 7fb159236640 1 --2- 172.21.15.167:0/3341661945 >> v2:172.21.15.204:3300/0 conn(0x7fb15406fc30 0x7fb154070090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:52.376 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:52.375+0000 7fb159236640 1 -- 172.21.15.167:0/3341661945 >> v1:172.21.15.204:6789/0 conn(0x7fb154142740 legacy=0x7fb1540b4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:05:52.376 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:52.376+0000 7fb159236640 1 -- 172.21.15.167:0/3341661945 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb15406f150 con 0x7fb154072150 2024-05-11T08:05:52.376 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:52.376+0000 7fb14bfff640 1 --2- 172.21.15.167:0/3341661945 >> v2:172.21.15.204:3300/0 conn(0x7fb15406fc30 0x7fb154070090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T08:05:52.378 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:52.376+0000 7fb159236640 1 --2- 172.21.15.167:0/3341661945 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb154072150 0x7fb1540725c0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb14c00bb10 tx=0x7fb14c02c890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e59870afdfbe1edf server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:05:52.378 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:52.376+0000 7fb14b7fe640 1 -- 172.21.15.167:0/3341661945 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb14c032790 con 0x7fb154072150 2024-05-11T08:05:52.378 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:52.376+0000 7fb14b7fe640 1 -- 172.21.15.167:0/3341661945 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb14c038070 con 0x7fb154072150 2024-05-11T08:05:52.378 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:52.376+0000 7fb14b7fe640 1 -- 172.21.15.167:0/3341661945 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb14c032b50 con 0x7fb154072150 2024-05-11T08:05:52.378 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:52.377+0000 7fb15ac99640 1 -- 172.21.15.167:0/3341661945 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb154072150 msgr2=0x7fb1540725c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:05:52.378 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:52.377+0000 7fb15ac99640 1 --2- 172.21.15.167:0/3341661945 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb154072150 0x7fb1540725c0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb14c00bb10 tx=0x7fb14c02c890 comp rx=0 tx=0).stop 2024-05-11T08:05:52.378 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:52.377+0000 7fb15ac99640 1 -- 172.21.15.167:0/3341661945 shutdown_connections 2024-05-11T08:05:52.378 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:52.377+0000 7fb15ac99640 1 --2- 172.21.15.167:0/3341661945 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb154072150 0x7fb1540725c0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:52.378 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:52.377+0000 7fb15ac99640 1 --2- 172.21.15.167:0/3341661945 >> v2:172.21.15.204:3300/0 conn(0x7fb15406fc30 0x7fb154070090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:52.379 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:52.377+0000 7fb15ac99640 1 -- 172.21.15.167:0/3341661945 >> 172.21.15.167:0/3341661945 conn(0x7fb1540d0c70 msgr2=0x7fb1540d1080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:05:52.379 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:52.377+0000 7fb15ac99640 1 -- 172.21.15.167:0/3341661945 shutdown_connections 2024-05-11T08:05:52.379 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:52.377+0000 7fb15ac99640 1 -- 172.21.15.167:0/3341661945 wait complete. 2024-05-11T08:05:52.379 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:52.378+0000 7fb15ac99640 1 Processor -- start 2024-05-11T08:05:52.379 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:52.378+0000 7fb15ac99640 1 -- start start 2024-05-11T08:05:52.381 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:52.379+0000 7fb15ac99640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb15406fc30 0x7fb15415ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:05:52.381 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:52.379+0000 7fb15ac99640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb154072150 0x7fb15415e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:05:52.381 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:52.379+0000 7fb15ac99640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb154142740 0x7fb15415eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:05:52.381 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:52.379+0000 7fb15ac99640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb154142b20 con 0x7fb154142740 2024-05-11T08:05:52.381 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:52.379+0000 7fb15ac99640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb1540b4e70 con 0x7fb154072150 2024-05-11T08:05:52.381 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:52.379+0000 7fb15ac99640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fb1541609a0 con 0x7fb15406fc30 2024-05-11T08:05:52.382 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:52.379+0000 7fb159236640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb154142740 0x7fb15415eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:05:52.382 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:52.379+0000 7fb159236640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb154142740 0x7fb15415eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:42528/0 (socket says 172.21.15.167:42528) 2024-05-11T08:05:52.382 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:52.379+0000 7fb159236640 1 -- 172.21.15.167:0/3379632090 learned_addr learned my addr 172.21.15.167:0/3379632090 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:05:52.382 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:52.379+0000 7fb158a35640 1 --2- 172.21.15.167:0/3379632090 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb15406fc30 0x7fb15415ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:05:52.382 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:52.379+0000 7fb14bfff640 1 --2- 172.21.15.167:0/3379632090 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb154072150 0x7fb15415e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:05:52.382 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:52.379+0000 7fb159236640 1 -- 172.21.15.167:0/3379632090 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb15406fc30 msgr2=0x7fb15415ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:05:52.382 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:52.379+0000 7fb159236640 1 --2- 172.21.15.167:0/3379632090 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb15406fc30 0x7fb15415ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:52.382 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:52.379+0000 7fb159236640 1 -- 172.21.15.167:0/3379632090 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb154072150 msgr2=0x7fb15415e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:05:52.382 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:52.379+0000 7fb159236640 1 --2- 172.21.15.167:0/3379632090 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb154072150 0x7fb15415e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:52.382 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:52.379+0000 7fb159236640 1 -- 172.21.15.167:0/3379632090 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb15413ff90 con 0x7fb154142740 2024-05-11T08:05:52.382 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:52.379+0000 7fb158a35640 1 --2- 172.21.15.167:0/3379632090 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb15406fc30 0x7fb15415ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T08:05:52.382 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:52.379+0000 7fb14bfff640 1 --2- 172.21.15.167:0/3379632090 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb154072150 0x7fb15415e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T08:05:52.382 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:52.380+0000 7fb159236640 1 --2- 172.21.15.167:0/3379632090 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb154142740 0x7fb15415eec0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb14c00b2c0 tx=0x7fb14c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:05:52.383 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:52.380+0000 7fb1497fa640 1 -- 172.21.15.167:0/3379632090 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb14c043070 con 0x7fb154142740 2024-05-11T08:05:52.383 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:52.380+0000 7fb1497fa640 1 -- 172.21.15.167:0/3379632090 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb14c03c620 con 0x7fb154142740 2024-05-11T08:05:52.383 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:52.380+0000 7fb15ac99640 1 -- 172.21.15.167:0/3379632090 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb154140270 con 0x7fb154142740 2024-05-11T08:05:52.383 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:52.380+0000 7fb1497fa640 1 -- 172.21.15.167:0/3379632090 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb14c03ca10 con 0x7fb154142740 2024-05-11T08:05:52.383 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:52.380+0000 7fb15ac99640 1 -- 172.21.15.167:0/3379632090 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb15415bfd0 con 0x7fb154142740 2024-05-11T08:05:52.383 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:52.381+0000 7fb15ac99640 1 -- 172.21.15.167:0/3379632090 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb154069590 con 0x7fb154142740 2024-05-11T08:05:52.383 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:52.381+0000 7fb1497fa640 1 -- 172.21.15.167:0/3379632090 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fb14c046070 con 0x7fb154142740 2024-05-11T08:05:52.384 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:52.382+0000 7fb1497fa640 1 --2- 172.21.15.167:0/3379632090 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fb1400762d0 0x7fb140078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:05:52.384 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:52.382+0000 7fb1497fa640 1 -- 172.21.15.167:0/3379632090 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4857+0+0 (secure 0 0 0) 0x7fb14c0b7800 con 0x7fb154142740 2024-05-11T08:05:52.384 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:52.382+0000 7fb158a35640 1 --2- 172.21.15.167:0/3379632090 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fb1400762d0 0x7fb140078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:05:52.384 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:52.382+0000 7fb158a35640 1 --2- 172.21.15.167:0/3379632090 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fb1400762d0 0x7fb140078790 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fb1540cd700 tx=0x7fb150009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:05:52.386 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:52.386+0000 7fb1497fa640 1 -- 172.21.15.167:0/3379632090 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fb14c081540 con 0x7fb154142740 2024-05-11T08:05:52.569 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:52.569+0000 7fb15ac99640 1 -- 172.21.15.167:0/3379632090 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fb154145d60 con 0x7fb154142740 2024-05-11T08:05:53.341 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:53.340+0000 7fb1497fa640 1 -- 172.21.15.167:0/3379632090 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v20) ==== 138+0+0 (secure 0 0 0) 0x7fb14c080e40 con 0x7fb154142740 2024-05-11T08:05:53.341 INFO:teuthology.orchestra.run.smithi167.stderr:pool 'cephfs_metadata' created 2024-05-11T08:05:53.345 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:53.345+0000 7fb15ac99640 1 -- 172.21.15.167:0/3379632090 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fb1400762d0 msgr2=0x7fb140078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:05:53.345 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:53.345+0000 7fb15ac99640 1 --2- 172.21.15.167:0/3379632090 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fb1400762d0 0x7fb140078790 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fb1540cd700 tx=0x7fb150009040 comp rx=0 tx=0).stop 2024-05-11T08:05:53.345 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:53.345+0000 7fb15ac99640 1 -- 172.21.15.167:0/3379632090 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb154142740 msgr2=0x7fb15415eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:05:53.346 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:53.345+0000 7fb15ac99640 1 --2- 172.21.15.167:0/3379632090 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb154142740 0x7fb15415eec0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb14c00b2c0 tx=0x7fb14c004730 comp rx=0 tx=0).stop 2024-05-11T08:05:53.346 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:53.345+0000 7fb15ac99640 1 -- 172.21.15.167:0/3379632090 shutdown_connections 2024-05-11T08:05:53.346 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:53.345+0000 7fb15ac99640 1 --2- 172.21.15.167:0/3379632090 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fb1400762d0 0x7fb140078790 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:53.346 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:53.345+0000 7fb15ac99640 1 --2- 172.21.15.167:0/3379632090 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb154142740 0x7fb15415eec0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:53.346 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:53.345+0000 7fb15ac99640 1 --2- 172.21.15.167:0/3379632090 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb154072150 0x7fb15415e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:53.346 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:53.345+0000 7fb15ac99640 1 --2- 172.21.15.167:0/3379632090 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb15406fc30 0x7fb15415ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:53.346 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:53.345+0000 7fb15ac99640 1 -- 172.21.15.167:0/3379632090 >> 172.21.15.167:0/3379632090 conn(0x7fb1540d0c70 msgr2=0x7fb1540cf680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:05:53.346 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:53.345+0000 7fb15ac99640 1 -- 172.21.15.167:0/3379632090 shutdown_connections 2024-05-11T08:05:53.346 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:53.345+0000 7fb15ac99640 1 -- 172.21.15.167:0/3379632090 wait complete. 2024-05-11T08:05:53.361 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-05-11T08:05:53.473 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:53.473+0000 7fa05f54e640 1 Processor -- start 2024-05-11T08:05:53.473 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:53.473+0000 7fa05f54e640 1 -- start start 2024-05-11T08:05:53.474 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:53.474+0000 7fa05f54e640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fa05815f2a0 0x7fa058162e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:05:53.474 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:53.474+0000 7fa05f54e640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fa0580d15c0 0x7fa0581637f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:05:53.475 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:53.474+0000 7fa05f54e640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa05806d1f0 con 0x7fa0581633d0 2024-05-11T08:05:53.475 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:53.474+0000 7fa05f54e640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7fa0580d1a10 con 0x7fa05815f2a0 2024-05-11T08:05:53.475 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:53.474+0000 7fa05f54e640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fa05806d4d0 con 0x7fa0580d15c0 2024-05-11T08:05:53.475 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:53.474+0000 7fa05d2ea640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fa05815f2a0 0x7fa058162e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:05:53.475 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:53.474+0000 7fa05cae9640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7fa0581633d0 0x7fa0580d3210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:42696/0 (socket says 172.21.15.167:42696) 2024-05-11T08:05:53.475 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:53.474+0000 7fa05cae9640 1 -- 172.21.15.167:0/2807160173 learned_addr learned my addr 172.21.15.167:0/2807160173 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T08:05:53.475 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:53.474+0000 7fa05daeb640 1 --2- 172.21.15.167:0/2807160173 >> v2:172.21.15.204:3300/0 conn(0x7fa0580d15c0 0x7fa0581637f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:05:53.475 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:53.474+0000 7fa05d2ea640 1 -- 172.21.15.167:0/2807160173 >> v2:172.21.15.204:3300/0 conn(0x7fa0580d15c0 msgr2=0x7fa0581637f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:05:53.475 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:53.474+0000 7fa05d2ea640 1 --2- 172.21.15.167:0/2807160173 >> v2:172.21.15.204:3300/0 conn(0x7fa0580d15c0 0x7fa0581637f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:53.475 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:53.474+0000 7fa05d2ea640 1 -- 172.21.15.167:0/2807160173 >> v1:172.21.15.167:6789/0 conn(0x7fa0581633d0 legacy=0x7fa0580d3210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:05:53.475 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:53.474+0000 7fa05d2ea640 1 -- 172.21.15.167:0/2807160173 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa05806d7b0 con 0x7fa05815f2a0 2024-05-11T08:05:53.475 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:53.474+0000 7fa04ffff640 1 -- 172.21.15.167:0/2807160173 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 50691788 0 0) 0x7fa05806d1f0 con 0x7fa0581633d0 2024-05-11T08:05:53.477 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:53.475+0000 7fa05d2ea640 1 --2- 172.21.15.167:0/2807160173 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa05815f2a0 0x7fa058162e90 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fa0580cfae0 tx=0x7fa05402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd9991a2e3ee5862 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:05:53.477 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:53.475+0000 7fa04ffff640 1 -- 172.21.15.167:0/2807160173 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa054007740 con 0x7fa05815f2a0 2024-05-11T08:05:53.477 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:53.475+0000 7fa04ffff640 1 -- 172.21.15.167:0/2807160173 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0540078e0 con 0x7fa05815f2a0 2024-05-11T08:05:53.477 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:53.475+0000 7fa04ffff640 1 -- 172.21.15.167:0/2807160173 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa054007cb0 con 0x7fa05815f2a0 2024-05-11T08:05:53.477 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:53.476+0000 7fa05f54e640 1 -- 172.21.15.167:0/2807160173 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa05815f2a0 msgr2=0x7fa058162e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:05:53.477 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:53.476+0000 7fa05f54e640 1 --2- 172.21.15.167:0/2807160173 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa05815f2a0 0x7fa058162e90 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fa0580cfae0 tx=0x7fa05402a710 comp rx=0 tx=0).stop 2024-05-11T08:05:53.477 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:53.476+0000 7fa05f54e640 1 -- 172.21.15.167:0/2807160173 shutdown_connections 2024-05-11T08:05:53.477 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:53.476+0000 7fa05f54e640 1 --2- 172.21.15.167:0/2807160173 >> v2:172.21.15.204:3300/0 conn(0x7fa0580d15c0 0x7fa0581637f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:53.477 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:53.476+0000 7fa05f54e640 1 --2- 172.21.15.167:0/2807160173 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa05815f2a0 0x7fa058162e90 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:53.477 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:53.476+0000 7fa05f54e640 1 -- 172.21.15.167:0/2807160173 >> 172.21.15.167:0/2807160173 conn(0x7fa0580d5120 msgr2=0x7fa0580d5530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:05:53.477 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:53.476+0000 7fa05f54e640 1 -- 172.21.15.167:0/2807160173 shutdown_connections 2024-05-11T08:05:53.477 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:53.476+0000 7fa05f54e640 1 -- 172.21.15.167:0/2807160173 wait complete. 2024-05-11T08:05:53.477 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:53.477+0000 7fa05f54e640 1 Processor -- start 2024-05-11T08:05:53.478 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:53.477+0000 7fa05f54e640 1 -- start start 2024-05-11T08:05:53.479 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:53.478+0000 7fa05f54e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa0580d15c0 0x7fa058162270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:05:53.480 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:53.478+0000 7fa05f54e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa05815f2a0 0x7fa05814f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:05:53.480 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:53.478+0000 7fa05f54e640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa0581633d0 0x7fa0580cd5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:05:53.480 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:53.478+0000 7fa05f54e640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa05815f680 con 0x7fa0580d15c0 2024-05-11T08:05:53.480 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:53.478+0000 7fa05f54e640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa0580d2cf0 con 0x7fa05815f2a0 2024-05-11T08:05:53.480 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:53.478+0000 7fa05f54e640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fa058160e40 con 0x7fa0581633d0 2024-05-11T08:05:53.480 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:53.478+0000 7fa05d2ea640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa0580d15c0 0x7fa058162270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:05:53.480 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:53.478+0000 7fa05d2ea640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa0580d15c0 0x7fa058162270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:42552/0 (socket says 172.21.15.167:42552) 2024-05-11T08:05:53.480 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:53.478+0000 7fa05d2ea640 1 -- 172.21.15.167:0/1419640611 learned_addr learned my addr 172.21.15.167:0/1419640611 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:05:53.480 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:53.478+0000 7fa05daeb640 1 --2- 172.21.15.167:0/1419640611 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa0581633d0 0x7fa0580cd5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:05:53.480 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:53.478+0000 7fa05cae9640 1 --2- 172.21.15.167:0/1419640611 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa05815f2a0 0x7fa05814f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:05:53.480 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:53.478+0000 7fa05d2ea640 1 -- 172.21.15.167:0/1419640611 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa0581633d0 msgr2=0x7fa0580cd5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:05:53.480 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:53.478+0000 7fa05d2ea640 1 --2- 172.21.15.167:0/1419640611 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa0581633d0 0x7fa0580cd5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:53.480 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:53.478+0000 7fa05d2ea640 1 -- 172.21.15.167:0/1419640611 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa05815f2a0 msgr2=0x7fa05814f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:05:53.480 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:53.478+0000 7fa05d2ea640 1 --2- 172.21.15.167:0/1419640611 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa05815f2a0 0x7fa05814f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:53.481 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:53.478+0000 7fa05d2ea640 1 -- 172.21.15.167:0/1419640611 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa05814faa0 con 0x7fa0580d15c0 2024-05-11T08:05:53.481 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:53.478+0000 7fa05d2ea640 1 --2- 172.21.15.167:0/1419640611 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa0580d15c0 0x7fa058162270 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa054009e80 tx=0x7fa054004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:05:53.481 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:53.478+0000 7fa04dffb640 1 -- 172.21.15.167:0/1419640611 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa054040070 con 0x7fa0580d15c0 2024-05-11T08:05:53.481 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:53.478+0000 7fa04dffb640 1 -- 172.21.15.167:0/1419640611 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa054031c40 con 0x7fa0580d15c0 2024-05-11T08:05:53.481 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:53.478+0000 7fa04dffb640 1 -- 172.21.15.167:0/1419640611 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa05403c420 con 0x7fa0580d15c0 2024-05-11T08:05:53.481 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:53.478+0000 7fa05f54e640 1 -- 172.21.15.167:0/1419640611 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0580cdae0 con 0x7fa0580d15c0 2024-05-11T08:05:53.481 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:53.479+0000 7fa05f54e640 1 -- 172.21.15.167:0/1419640611 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0580ce060 con 0x7fa0580d15c0 2024-05-11T08:05:53.481 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:53.480+0000 7fa05f54e640 1 -- 172.21.15.167:0/1419640611 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0580cea50 con 0x7fa0580d15c0 2024-05-11T08:05:53.482 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:53.480+0000 7fa04dffb640 1 -- 172.21.15.167:0/1419640611 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fa054005d10 con 0x7fa0580d15c0 2024-05-11T08:05:53.482 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:53.481+0000 7fa04dffb640 1 --2- 172.21.15.167:0/1419640611 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fa03c0762d0 0x7fa03c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:05:53.482 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:53.481+0000 7fa04dffb640 1 -- 172.21.15.167:0/1419640611 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5227+0+0 (secure 0 0 0) 0x7fa0540b83d0 con 0x7fa0580d15c0 2024-05-11T08:05:53.482 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:53.481+0000 7fa05cae9640 1 --2- 172.21.15.167:0/1419640611 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fa03c0762d0 0x7fa03c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:05:53.482 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:53.481+0000 7fa05cae9640 1 --2- 172.21.15.167:0/1419640611 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fa03c0762d0 0x7fa03c078790 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa05806c860 tx=0x7fa048007c00 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:05:53.485 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:53.485+0000 7fa04dffb640 1 -- 172.21.15.167:0/1419640611 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fa05404b020 con 0x7fa0580d15c0 2024-05-11T08:05:53.666 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:53.666+0000 7fa05f54e640 1 -- 172.21.15.167:0/1419640611 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7fa058069630 con 0x7fa0580d15c0 2024-05-11T08:05:54.348 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:54.348+0000 7fa04dffb640 1 -- 172.21.15.167:0/1419640611 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v21) ==== 171+0+0 (secure 0 0 0) 0x7fa05814faa0 con 0x7fa0580d15c0 2024-05-11T08:05:54.348 INFO:teuthology.orchestra.run.smithi167.stderr:pool 'cephfs_data' created 2024-05-11T08:05:54.351 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:54.350+0000 7fa05f54e640 1 -- 172.21.15.167:0/1419640611 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fa03c0762d0 msgr2=0x7fa03c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:05:54.351 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:54.350+0000 7fa05f54e640 1 --2- 172.21.15.167:0/1419640611 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fa03c0762d0 0x7fa03c078790 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa05806c860 tx=0x7fa048007c00 comp rx=0 tx=0).stop 2024-05-11T08:05:54.351 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:54.350+0000 7fa05f54e640 1 -- 172.21.15.167:0/1419640611 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa0580d15c0 msgr2=0x7fa058162270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:05:54.351 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:54.350+0000 7fa05f54e640 1 --2- 172.21.15.167:0/1419640611 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa0580d15c0 0x7fa058162270 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa054009e80 tx=0x7fa054004a40 comp rx=0 tx=0).stop 2024-05-11T08:05:54.351 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:54.351+0000 7fa05f54e640 1 -- 172.21.15.167:0/1419640611 shutdown_connections 2024-05-11T08:05:54.351 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:54.351+0000 7fa05f54e640 1 --2- 172.21.15.167:0/1419640611 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fa03c0762d0 0x7fa03c078790 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:54.351 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:54.351+0000 7fa05f54e640 1 --2- 172.21.15.167:0/1419640611 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa0581633d0 0x7fa0580cd5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:54.351 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:54.351+0000 7fa05f54e640 1 --2- 172.21.15.167:0/1419640611 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa05815f2a0 0x7fa05814f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:54.351 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:54.351+0000 7fa05f54e640 1 --2- 172.21.15.167:0/1419640611 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa0580d15c0 0x7fa058162270 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:54.351 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:54.351+0000 7fa05f54e640 1 -- 172.21.15.167:0/1419640611 >> 172.21.15.167:0/1419640611 conn(0x7fa0580d5120 msgr2=0x7fa05815e5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:05:54.351 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:54.351+0000 7fa05f54e640 1 -- 172.21.15.167:0/1419640611 shutdown_connections 2024-05-11T08:05:54.351 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:54.351+0000 7fa05f54e640 1 -- 172.21.15.167:0/1419640611 wait complete. 2024-05-11T08:05:54.376 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-11T08:05:54.480 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:54.480+0000 7ff07ab32640 1 Processor -- start 2024-05-11T08:05:54.480 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:54.480+0000 7ff07ab32640 1 -- start start 2024-05-11T08:05:54.482 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:54.481+0000 7ff07ab32640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7ff07406d200 0x7ff07406d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:05:54.482 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:54.481+0000 7ff07ab32640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7ff0740d2db0 0x7ff0740d3240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:05:54.482 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:54.481+0000 7ff07ab32640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff074159d50 con 0x7ff074159970 2024-05-11T08:05:54.482 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:54.481+0000 7ff07ab32640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7ff0740d04b0 con 0x7ff0740d2db0 2024-05-11T08:05:54.482 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:54.481+0000 7ff07ab32640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7ff0740d0790 con 0x7ff07406d200 2024-05-11T08:05:54.482 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:54.481+0000 7ff0790cf640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7ff0740d2db0 0x7ff0740d3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:05:54.482 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:54.481+0000 7ff0790cf640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7ff0740d2db0 0x7ff0740d3240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:58902/0 (socket says 172.21.15.167:58902) 2024-05-11T08:05:54.482 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:54.481+0000 7ff0790cf640 1 -- 172.21.15.167:0/1181377192 learned_addr learned my addr 172.21.15.167:0/1181377192 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:05:54.483 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:54.481+0000 7ff06bfff640 1 --2- 172.21.15.167:0/1181377192 >> v2:172.21.15.204:3300/0 conn(0x7ff07406d200 0x7ff07406d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:05:54.483 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:54.481+0000 7ff0790cf640 1 -- 172.21.15.167:0/1181377192 >> v2:172.21.15.204:3300/0 conn(0x7ff07406d200 msgr2=0x7ff07406d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:05:54.483 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:54.481+0000 7ff0790cf640 1 --2- 172.21.15.167:0/1181377192 >> v2:172.21.15.204:3300/0 conn(0x7ff07406d200 0x7ff07406d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:54.483 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:54.481+0000 7ff0790cf640 1 -- 172.21.15.167:0/1181377192 >> v1:172.21.15.204:6789/0 conn(0x7ff074159970 legacy=0x7ff0740d1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:05:54.483 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:54.481+0000 7ff0790cf640 1 -- 172.21.15.167:0/1181377192 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0740d0a70 con 0x7ff0740d2db0 2024-05-11T08:05:54.483 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:54.482+0000 7ff0790cf640 1 --2- 172.21.15.167:0/1181377192 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff0740d2db0 0x7ff0740d3240 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff0740cfe20 tx=0x7ff06c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57f7d59d3ff4b8dc server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:05:54.483 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:54.482+0000 7ff06b7fe640 1 -- 172.21.15.167:0/1181377192 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff06c037070 con 0x7ff0740d2db0 2024-05-11T08:05:54.483 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:54.482+0000 7ff06b7fe640 1 -- 172.21.15.167:0/1181377192 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff06c032440 con 0x7ff0740d2db0 2024-05-11T08:05:54.483 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:54.482+0000 7ff06b7fe640 1 -- 172.21.15.167:0/1181377192 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff06c032810 con 0x7ff0740d2db0 2024-05-11T08:05:54.486 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:54.482+0000 7ff07ab32640 1 -- 172.21.15.167:0/1181377192 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff0740d2db0 msgr2=0x7ff0740d3240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:05:54.486 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:54.482+0000 7ff07ab32640 1 --2- 172.21.15.167:0/1181377192 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff0740d2db0 0x7ff0740d3240 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff0740cfe20 tx=0x7ff06c02a1e0 comp rx=0 tx=0).stop 2024-05-11T08:05:54.486 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:54.482+0000 7ff07ab32640 1 -- 172.21.15.167:0/1181377192 shutdown_connections 2024-05-11T08:05:54.487 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:54.482+0000 7ff07ab32640 1 --2- 172.21.15.167:0/1181377192 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff0740d2db0 0x7ff0740d3240 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:54.487 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:54.482+0000 7ff07ab32640 1 --2- 172.21.15.167:0/1181377192 >> v2:172.21.15.204:3300/0 conn(0x7ff07406d200 0x7ff07406d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:54.487 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:54.483+0000 7ff07ab32640 1 -- 172.21.15.167:0/1181377192 >> 172.21.15.167:0/1181377192 conn(0x7ff0740d51e0 msgr2=0x7ff0740d55f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:05:54.487 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:54.483+0000 7ff07ab32640 1 -- 172.21.15.167:0/1181377192 shutdown_connections 2024-05-11T08:05:54.487 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:54.483+0000 7ff07ab32640 1 -- 172.21.15.167:0/1181377192 wait complete. 2024-05-11T08:05:54.487 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:54.484+0000 7ff07ab32640 1 Processor -- start 2024-05-11T08:05:54.487 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:54.484+0000 7ff07ab32640 1 -- start start 2024-05-11T08:05:54.487 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:54.485+0000 7ff07ab32640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff07406d200 0x7ff074153880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:05:54.487 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:54.485+0000 7ff07ab32640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff0740d2db0 0x7ff0740cb1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:05:54.487 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:54.485+0000 7ff07ab32640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff074159970 0x7ff0740cb730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:05:54.487 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:54.485+0000 7ff07ab32640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff0740d1980 con 0x7ff07406d200 2024-05-11T08:05:54.488 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:54.485+0000 7ff07ab32640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff074163d00 con 0x7ff074159970 2024-05-11T08:05:54.488 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:54.485+0000 7ff07ab32640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7ff074162f10 con 0x7ff0740d2db0 2024-05-11T08:05:54.488 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:54.485+0000 7ff0788ce640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff07406d200 0x7ff074153880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:05:54.488 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:54.485+0000 7ff0788ce640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff07406d200 0x7ff074153880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:58908/0 (socket says 172.21.15.167:58908) 2024-05-11T08:05:54.488 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:54.485+0000 7ff0788ce640 1 -- 172.21.15.167:0/2756967404 learned_addr learned my addr 172.21.15.167:0/2756967404 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:05:54.488 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:54.485+0000 7ff06bfff640 1 --2- 172.21.15.167:0/2756967404 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff0740d2db0 0x7ff0740cb1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:05:54.488 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:54.485+0000 7ff0790cf640 1 --2- 172.21.15.167:0/2756967404 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff074159970 0x7ff0740cb730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:05:54.488 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:54.485+0000 7ff0788ce640 1 -- 172.21.15.167:0/2756967404 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff0740d2db0 msgr2=0x7ff0740cb1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:05:54.488 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:54.485+0000 7ff0788ce640 1 --2- 172.21.15.167:0/2756967404 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff0740d2db0 0x7ff0740cb1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:54.488 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:54.485+0000 7ff0788ce640 1 -- 172.21.15.167:0/2756967404 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff074159970 msgr2=0x7ff0740cb730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:05:54.489 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:54.485+0000 7ff0788ce640 1 --2- 172.21.15.167:0/2756967404 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff074159970 0x7ff0740cb730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:54.489 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:54.485+0000 7ff0788ce640 1 -- 172.21.15.167:0/2756967404 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0741542e0 con 0x7ff07406d200 2024-05-11T08:05:54.489 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:54.485+0000 7ff0788ce640 1 --2- 172.21.15.167:0/2756967404 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff07406d200 0x7ff074153880 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff07000ba50 tx=0x7ff07000bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:05:54.489 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:54.485+0000 7ff0697fa640 1 -- 172.21.15.167:0/2756967404 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff070019070 con 0x7ff07406d200 2024-05-11T08:05:54.489 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:54.485+0000 7ff0697fa640 1 -- 172.21.15.167:0/2756967404 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff07000cc10 con 0x7ff07406d200 2024-05-11T08:05:54.489 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:54.485+0000 7ff07ab32640 1 -- 172.21.15.167:0/2756967404 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0741545c0 con 0x7ff07406d200 2024-05-11T08:05:54.489 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:54.485+0000 7ff0697fa640 1 -- 172.21.15.167:0/2756967404 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff070014440 con 0x7ff07406d200 2024-05-11T08:05:54.489 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:54.486+0000 7ff07ab32640 1 -- 172.21.15.167:0/2756967404 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff074154b80 con 0x7ff07406d200 2024-05-11T08:05:54.489 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:54.486+0000 7ff07ab32640 1 -- 172.21.15.167:0/2756967404 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff074069520 con 0x7ff07406d200 2024-05-11T08:05:54.490 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:54.487+0000 7ff0697fa640 1 -- 172.21.15.167:0/2756967404 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7ff070007370 con 0x7ff07406d200 2024-05-11T08:05:54.490 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:54.488+0000 7ff0697fa640 1 --2- 172.21.15.167:0/2756967404 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7ff060076300 0x7ff0600787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:05:54.490 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:54.488+0000 7ff06bfff640 1 --2- 172.21.15.167:0/2756967404 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7ff060076300 0x7ff0600787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:05:54.490 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:54.488+0000 7ff0697fa640 1 -- 172.21.15.167:0/2756967404 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5609+0+0 (secure 0 0 0) 0x7ff0700970a0 con 0x7ff07406d200 2024-05-11T08:05:54.490 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:54.488+0000 7ff06bfff640 1 --2- 172.21.15.167:0/2756967404 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7ff060076300 0x7ff0600787c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff0741617c0 tx=0x7ff064008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:05:54.492 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:54.492+0000 7ff0697fa640 1 -- 172.21.15.167:0/2756967404 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7ff0700604a0 con 0x7ff07406d200 2024-05-11T08:05:54.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:54.710+0000 7ff07ab32640 1 -- 172.21.15.167:0/2756967404 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7ff0740d00f0 con 0x7ff07406d200 2024-05-11T08:05:55.349 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:55.349+0000 7ff0697fa640 1 -- 172.21.15.167:0/2756967404 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7ff0700647a0 con 0x7ff07406d200 2024-05-11T08:05:55.349 INFO:teuthology.orchestra.run.smithi167.stderr:new fs with metadata pool 2 and data pool 3 2024-05-11T08:05:55.353 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:55.352+0000 7ff07ab32640 1 -- 172.21.15.167:0/2756967404 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7ff060076300 msgr2=0x7ff0600787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:05:55.353 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:55.352+0000 7ff07ab32640 1 --2- 172.21.15.167:0/2756967404 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7ff060076300 0x7ff0600787c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff0741617c0 tx=0x7ff064008040 comp rx=0 tx=0).stop 2024-05-11T08:05:55.353 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:55.352+0000 7ff07ab32640 1 -- 172.21.15.167:0/2756967404 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff07406d200 msgr2=0x7ff074153880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:05:55.353 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:55.352+0000 7ff07ab32640 1 --2- 172.21.15.167:0/2756967404 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff07406d200 0x7ff074153880 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff07000ba50 tx=0x7ff07000bf10 comp rx=0 tx=0).stop 2024-05-11T08:05:55.353 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:55.352+0000 7ff07ab32640 1 -- 172.21.15.167:0/2756967404 shutdown_connections 2024-05-11T08:05:55.353 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:55.352+0000 7ff07ab32640 1 --2- 172.21.15.167:0/2756967404 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7ff060076300 0x7ff0600787c0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:55.353 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:55.352+0000 7ff07ab32640 1 --2- 172.21.15.167:0/2756967404 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff074159970 0x7ff0740cb730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:55.353 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:55.352+0000 7ff07ab32640 1 --2- 172.21.15.167:0/2756967404 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff0740d2db0 0x7ff0740cb1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:55.353 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:55.352+0000 7ff07ab32640 1 --2- 172.21.15.167:0/2756967404 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff07406d200 0x7ff074153880 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:55.353 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:55.352+0000 7ff07ab32640 1 -- 172.21.15.167:0/2756967404 >> 172.21.15.167:0/2756967404 conn(0x7ff0740d51e0 msgr2=0x7ff0740d55c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:05:55.353 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:55.353+0000 7ff07ab32640 1 -- 172.21.15.167:0/2756967404 shutdown_connections 2024-05-11T08:05:55.353 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:55.353+0000 7ff07ab32640 1 -- 172.21.15.167:0/2756967404 wait complete. 2024-05-11T08:05:55.383 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-05-11T08:05:55.383 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-05-11T08:05:55.488 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:55.488+0000 7ff52f2a7640 1 Processor -- start 2024-05-11T08:05:55.488 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:55.488+0000 7ff52f2a7640 1 -- start start 2024-05-11T08:05:55.489 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:55.489+0000 7ff52f2a7640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7ff528148d50 0x7ff5280cd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:05:55.489 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:55.489+0000 7ff52f2a7640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5280cfad0 con 0x7ff528163580 2024-05-11T08:05:55.489 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:55.489+0000 7ff52f2a7640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5280cfdb0 con 0x7ff5280d2630 2024-05-11T08:05:55.489 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:55.489+0000 7ff52f2a7640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7ff5280cda40 con 0x7ff528148d50 2024-05-11T08:05:55.489 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:55.489+0000 7ff52d043640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7ff528148d50 0x7ff5280cd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:05:55.490 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:55.489+0000 7ff52d043640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7ff528148d50 0x7ff5280cd500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:58918/0 (socket says 172.21.15.167:58918) 2024-05-11T08:05:55.490 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:55.489+0000 7ff52d043640 1 -- 172.21.15.167:0/1184202803 learned_addr learned my addr 172.21.15.167:0/1184202803 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:05:55.490 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:55.489+0000 7ff52d043640 1 -- 172.21.15.167:0/1184202803 >> v1:172.21.15.204:6789/0 conn(0x7ff5280d2630 legacy=0x7ff528162e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:05:55.490 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:55.489+0000 7ff52d043640 1 -- 172.21.15.167:0/1184202803 >> v1:172.21.15.167:6789/0 conn(0x7ff528163580 legacy=0x7ff5280d0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:05:55.490 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:55.489+0000 7ff51ffff640 1 -- 172.21.15.167:0/1184202803 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3794686653 0 0) 0x7ff5280cfad0 con 0x7ff528163580 2024-05-11T08:05:55.490 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:55.489+0000 7ff52d043640 1 -- 172.21.15.167:0/1184202803 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5280d0090 con 0x7ff528148d50 2024-05-11T08:05:55.491 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:55.490+0000 7ff52d043640 1 --2- 172.21.15.167:0/1184202803 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff528148d50 0x7ff5280cd500 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff524009870 tx=0x7ff52402a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8431f44e4676dbb0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:05:55.491 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:55.490+0000 7ff51ffff640 1 -- 172.21.15.167:0/1184202803 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff524037070 con 0x7ff528148d50 2024-05-11T08:05:55.492 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:55.490+0000 7ff51ffff640 1 -- 172.21.15.167:0/1184202803 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff524032440 con 0x7ff528148d50 2024-05-11T08:05:55.492 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:55.490+0000 7ff51ffff640 1 -- 172.21.15.167:0/1184202803 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff524032810 con 0x7ff528148d50 2024-05-11T08:05:55.492 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:55.490+0000 7ff52f2a7640 1 -- 172.21.15.167:0/1184202803 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff528148d50 msgr2=0x7ff5280cd500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:05:55.492 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:55.490+0000 7ff52f2a7640 1 --2- 172.21.15.167:0/1184202803 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff528148d50 0x7ff5280cd500 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff524009870 tx=0x7ff52402a130 comp rx=0 tx=0).stop 2024-05-11T08:05:55.492 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:55.490+0000 7ff52f2a7640 1 -- 172.21.15.167:0/1184202803 shutdown_connections 2024-05-11T08:05:55.492 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:55.490+0000 7ff52f2a7640 1 --2- 172.21.15.167:0/1184202803 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff528148d50 0x7ff5280cd500 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:55.492 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:55.490+0000 7ff52f2a7640 1 -- 172.21.15.167:0/1184202803 >> 172.21.15.167:0/1184202803 conn(0x7ff5280b4ae0 msgr2=0x7ff5280b4ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:05:55.492 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:55.491+0000 7ff52f2a7640 1 -- 172.21.15.167:0/1184202803 shutdown_connections 2024-05-11T08:05:55.492 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:55.491+0000 7ff52f2a7640 1 -- 172.21.15.167:0/1184202803 wait complete. 2024-05-11T08:05:55.492 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:55.492+0000 7ff52f2a7640 1 Processor -- start 2024-05-11T08:05:55.492 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:55.492+0000 7ff52f2a7640 1 -- start start 2024-05-11T08:05:55.494 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:55.492+0000 7ff52f2a7640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff5280d2630 0x7ff528146d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:05:55.495 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:55.492+0000 7ff52f2a7640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff528148d50 0x7ff5281457d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:05:55.495 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:55.492+0000 7ff52f2a7640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff528163580 0x7ff52815e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:05:55.495 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:55.492+0000 7ff52f2a7640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff5280d1550 con 0x7ff528148d50 2024-05-11T08:05:55.495 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:55.492+0000 7ff52f2a7640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff5280b5de0 con 0x7ff5280d2630 2024-05-11T08:05:55.495 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:55.492+0000 7ff52f2a7640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7ff5280d16d0 con 0x7ff528163580 2024-05-11T08:05:55.495 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:55.492+0000 7ff52c842640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff528148d50 0x7ff5281457d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:05:55.495 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:55.492+0000 7ff52c842640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff528148d50 0x7ff5281457d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:58932/0 (socket says 172.21.15.167:58932) 2024-05-11T08:05:55.495 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:55.492+0000 7ff52c842640 1 -- 172.21.15.167:0/2914758125 learned_addr learned my addr 172.21.15.167:0/2914758125 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:05:55.495 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:55.493+0000 7ff52d844640 1 --2- 172.21.15.167:0/2914758125 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff528163580 0x7ff52815e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:05:55.495 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:55.493+0000 7ff52d043640 1 --2- 172.21.15.167:0/2914758125 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff5280d2630 0x7ff528146d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:05:55.495 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:55.493+0000 7ff52c842640 1 -- 172.21.15.167:0/2914758125 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff528163580 msgr2=0x7ff52815e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:05:55.495 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:55.493+0000 7ff52c842640 1 --2- 172.21.15.167:0/2914758125 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff528163580 0x7ff52815e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:55.496 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:55.493+0000 7ff52c842640 1 -- 172.21.15.167:0/2914758125 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff5280d2630 msgr2=0x7ff528146d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:05:55.496 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:55.493+0000 7ff52c842640 1 --2- 172.21.15.167:0/2914758125 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff5280d2630 0x7ff528146d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:55.496 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:55.493+0000 7ff52c842640 1 -- 172.21.15.167:0/2914758125 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff528146170 con 0x7ff528148d50 2024-05-11T08:05:55.496 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:55.493+0000 7ff52d844640 1 --2- 172.21.15.167:0/2914758125 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff528163580 0x7ff52815e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T08:05:55.496 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:55.493+0000 7ff52c842640 1 --2- 172.21.15.167:0/2914758125 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff528148d50 0x7ff5281457d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff51800c430 tx=0x7ff51800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:05:55.496 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:55.493+0000 7ff51dffb640 1 -- 172.21.15.167:0/2914758125 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff518011790 con 0x7ff528148d50 2024-05-11T08:05:55.496 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:55.493+0000 7ff51dffb640 1 -- 172.21.15.167:0/2914758125 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff518017070 con 0x7ff528148d50 2024-05-11T08:05:55.496 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:55.493+0000 7ff52f2a7640 1 -- 172.21.15.167:0/2914758125 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff528146450 con 0x7ff528148d50 2024-05-11T08:05:55.496 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:55.493+0000 7ff51dffb640 1 -- 172.21.15.167:0/2914758125 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff518011bb0 con 0x7ff528148d50 2024-05-11T08:05:55.496 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:55.493+0000 7ff52f2a7640 1 -- 172.21.15.167:0/2914758125 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5280ca020 con 0x7ff528148d50 2024-05-11T08:05:55.496 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:55.494+0000 7ff52f2a7640 1 -- 172.21.15.167:0/2914758125 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5280752e0 con 0x7ff528148d50 2024-05-11T08:05:55.496 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:55.495+0000 7ff51dffb640 1 -- 172.21.15.167:0/2914758125 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7ff51801c460 con 0x7ff528148d50 2024-05-11T08:05:55.497 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:55.495+0000 7ff51dffb640 1 --2- 172.21.15.167:0/2914758125 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7ff50c076300 0x7ff50c0787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:05:55.497 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:55.495+0000 7ff52d043640 1 --2- 172.21.15.167:0/2914758125 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7ff50c076300 0x7ff50c0787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:05:55.497 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:55.495+0000 7ff51dffb640 1 -- 172.21.15.167:0/2914758125 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7ff518097480 con 0x7ff528148d50 2024-05-11T08:05:55.497 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:55.496+0000 7ff52d043640 1 --2- 172.21.15.167:0/2914758125 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7ff50c076300 0x7ff50c0787c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff524034000 tx=0x7ff52402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:05:55.500 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:55.500+0000 7ff51dffb640 1 -- 172.21.15.167:0/2914758125 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7ff518060de0 con 0x7ff528148d50 2024-05-11T08:05:55.590 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-11T08:05:55.593 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-11T08:05:55.693 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:55.693+0000 7ff52f2a7640 1 -- 172.21.15.167:0/2914758125 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7ff5280b8e60 con 0x7ff528148d50 2024-05-11T08:05:56.351 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:56.351+0000 7ff51dffb640 1 -- 172.21.15.167:0/2914758125 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v23) ==== 157+0+0 (secure 0 0 0) 0x7ff5180606e0 con 0x7ff528148d50 2024-05-11T08:05:56.354 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:56.354+0000 7ff52f2a7640 1 -- 172.21.15.167:0/2914758125 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7ff50c076300 msgr2=0x7ff50c0787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:05:56.354 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:56.354+0000 7ff52f2a7640 1 --2- 172.21.15.167:0/2914758125 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7ff50c076300 0x7ff50c0787c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff524034000 tx=0x7ff52402e000 comp rx=0 tx=0).stop 2024-05-11T08:05:56.354 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:56.354+0000 7ff52f2a7640 1 -- 172.21.15.167:0/2914758125 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff528148d50 msgr2=0x7ff5281457d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:05:56.354 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:56.354+0000 7ff52f2a7640 1 --2- 172.21.15.167:0/2914758125 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff528148d50 0x7ff5281457d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff51800c430 tx=0x7ff51800c8f0 comp rx=0 tx=0).stop 2024-05-11T08:05:56.355 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:56.354+0000 7ff52f2a7640 1 -- 172.21.15.167:0/2914758125 shutdown_connections 2024-05-11T08:05:56.355 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:56.354+0000 7ff52f2a7640 1 --2- 172.21.15.167:0/2914758125 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7ff50c076300 0x7ff50c0787c0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:56.355 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:56.354+0000 7ff52f2a7640 1 --2- 172.21.15.167:0/2914758125 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff528163580 0x7ff52815e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:56.355 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:56.354+0000 7ff52f2a7640 1 --2- 172.21.15.167:0/2914758125 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff528148d50 0x7ff5281457d0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:56.355 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:56.354+0000 7ff52f2a7640 1 --2- 172.21.15.167:0/2914758125 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff5280d2630 0x7ff528146d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:56.355 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:56.354+0000 7ff52f2a7640 1 -- 172.21.15.167:0/2914758125 >> 172.21.15.167:0/2914758125 conn(0x7ff5280b4ae0 msgr2=0x7ff528143740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:05:56.355 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:56.354+0000 7ff52f2a7640 1 -- 172.21.15.167:0/2914758125 shutdown_connections 2024-05-11T08:05:56.355 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:56.354+0000 7ff52f2a7640 1 -- 172.21.15.167:0/2914758125 wait complete. 2024-05-11T08:05:56.368 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-05-11T08:05:56.482 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:56.482+0000 7fb8fd254640 1 Processor -- start 2024-05-11T08:05:56.482 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:56.482+0000 7fb8fd254640 1 -- start start 2024-05-11T08:05:56.484 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:56.482+0000 7fb8fd254640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb8f806f140 0x7fb8f80b4f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:05:56.484 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:56.483+0000 7fb8fd254640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fb8f8160c00 0x7fb8f8161070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:05:56.484 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:56.483+0000 7fb8fd254640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8f8070890 con 0x7fb8f806ed60 2024-05-11T08:05:56.484 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:56.483+0000 7fb8fd254640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7fb8f806d3f0 con 0x7fb8f8160c00 2024-05-11T08:05:56.484 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:56.483+0000 7fb8fd254640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fb8f81615b0 con 0x7fb8f806f140 2024-05-11T08:05:56.484 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:56.483+0000 7fb8f759e640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fb8f8160c00 0x7fb8f8161070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:05:56.484 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:56.483+0000 7fb8f759e640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fb8f8160c00 0x7fb8f8161070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:58938/0 (socket says 172.21.15.167:58938) 2024-05-11T08:05:56.484 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:56.483+0000 7fb8f6d9d640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7fb8f806ed60 0x7fb8f806cce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:40616/0 (socket says 172.21.15.167:40616) 2024-05-11T08:05:56.484 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:56.483+0000 7fb8f759e640 1 -- 172.21.15.167:0/3531964639 learned_addr learned my addr 172.21.15.167:0/3531964639 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:05:56.484 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:56.483+0000 7fb8effff640 1 --2- 172.21.15.167:0/3531964639 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb8f806f140 0x7fb8f80b4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:05:56.484 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:56.483+0000 7fb8f759e640 1 -- 172.21.15.167:0/3531964639 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb8f806f140 msgr2=0x7fb8f80b4f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:05:56.484 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:56.483+0000 7fb8f759e640 1 --2- 172.21.15.167:0/3531964639 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb8f806f140 0x7fb8f80b4f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:56.485 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:56.483+0000 7fb8f759e640 1 -- 172.21.15.167:0/3531964639 >> v1:172.21.15.167:6789/0 conn(0x7fb8f806ed60 legacy=0x7fb8f806cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:05:56.485 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:56.483+0000 7fb8f759e640 1 -- 172.21.15.167:0/3531964639 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8f806d610 con 0x7fb8f8160c00 2024-05-11T08:05:56.485 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:56.483+0000 7fb8f659c640 1 -- 172.21.15.167:0/3531964639 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3190900642 0 0) 0x7fb8f8070890 con 0x7fb8f806ed60 2024-05-11T08:05:56.485 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:56.483+0000 7fb8effff640 1 --2- 172.21.15.167:0/3531964639 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb8f806f140 0x7fb8f80b4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T08:05:56.485 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:56.484+0000 7fb8f759e640 1 --2- 172.21.15.167:0/3531964639 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb8f8160c00 0x7fb8f8161070 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb8f0009e50 tx=0x7fb8f002a700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ffc094db79f039a server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:05:56.485 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:56.484+0000 7fb8f659c640 1 -- 172.21.15.167:0/3531964639 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8f0036070 con 0x7fb8f8160c00 2024-05-11T08:05:56.485 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:56.484+0000 7fb8f659c640 1 -- 172.21.15.167:0/3531964639 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8f0031600 con 0x7fb8f8160c00 2024-05-11T08:05:56.485 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:56.484+0000 7fb8f659c640 1 -- 172.21.15.167:0/3531964639 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8f00319d0 con 0x7fb8f8160c00 2024-05-11T08:05:56.488 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:56.484+0000 7fb8fd254640 1 -- 172.21.15.167:0/3531964639 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb8f8160c00 msgr2=0x7fb8f8161070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:05:56.488 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:56.484+0000 7fb8fd254640 1 --2- 172.21.15.167:0/3531964639 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb8f8160c00 0x7fb8f8161070 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb8f0009e50 tx=0x7fb8f002a700 comp rx=0 tx=0).stop 2024-05-11T08:05:56.488 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:56.484+0000 7fb8fd254640 1 -- 172.21.15.167:0/3531964639 shutdown_connections 2024-05-11T08:05:56.488 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:56.484+0000 7fb8fd254640 1 --2- 172.21.15.167:0/3531964639 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb8f8160c00 0x7fb8f8161070 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:56.488 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:56.484+0000 7fb8fd254640 1 --2- 172.21.15.167:0/3531964639 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb8f806f140 0x7fb8f80b4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:56.489 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:56.484+0000 7fb8fd254640 1 -- 172.21.15.167:0/3531964639 >> 172.21.15.167:0/3531964639 conn(0x7fb8f80d1ed0 msgr2=0x7fb8f80d22e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:05:56.489 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:56.484+0000 7fb8fd254640 1 -- 172.21.15.167:0/3531964639 shutdown_connections 2024-05-11T08:05:56.489 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:56.484+0000 7fb8fd254640 1 -- 172.21.15.167:0/3531964639 wait complete. 2024-05-11T08:05:56.489 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:56.486+0000 7fb8fd254640 1 Processor -- start 2024-05-11T08:05:56.489 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:56.486+0000 7fb8fd254640 1 -- start start 2024-05-11T08:05:56.489 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:56.486+0000 7fb8fd254640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb8f806ed60 0x7fb8f80cc550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:05:56.489 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:56.486+0000 7fb8fd254640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb8f806f140 0x7fb8f80cddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:05:56.489 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:56.486+0000 7fb8fd254640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb8f8160c00 0x7fb8f8147720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:05:56.489 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:56.486+0000 7fb8fd254640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb8f806dbd0 con 0x7fb8f806ed60 2024-05-11T08:05:56.489 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:56.486+0000 7fb8fd254640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb8f8070bc0 con 0x7fb8f8160c00 2024-05-11T08:05:56.489 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:56.486+0000 7fb8fd254640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fb8f8071050 con 0x7fb8f806f140 2024-05-11T08:05:56.489 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:56.487+0000 7fb8f6d9d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb8f806ed60 0x7fb8f80cc550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:05:56.489 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:56.487+0000 7fb8f6d9d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb8f806ed60 0x7fb8f80cc550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:58940/0 (socket says 172.21.15.167:58940) 2024-05-11T08:05:56.489 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:56.487+0000 7fb8f6d9d640 1 -- 172.21.15.167:0/3696679808 learned_addr learned my addr 172.21.15.167:0/3696679808 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:05:56.490 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:56.487+0000 7fb8effff640 1 --2- 172.21.15.167:0/3696679808 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb8f806f140 0x7fb8f80cddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:05:56.490 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:56.487+0000 7fb8f759e640 1 --2- 172.21.15.167:0/3696679808 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb8f8160c00 0x7fb8f8147720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:05:56.490 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:56.487+0000 7fb8f6d9d640 1 -- 172.21.15.167:0/3696679808 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb8f806f140 msgr2=0x7fb8f80cddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:05:56.490 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:56.487+0000 7fb8f6d9d640 1 --2- 172.21.15.167:0/3696679808 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb8f806f140 0x7fb8f80cddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:56.490 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:56.487+0000 7fb8f6d9d640 1 -- 172.21.15.167:0/3696679808 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb8f8160c00 msgr2=0x7fb8f8147720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:05:56.490 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:56.487+0000 7fb8f6d9d640 1 --2- 172.21.15.167:0/3696679808 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb8f8160c00 0x7fb8f8147720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:56.490 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:56.487+0000 7fb8f6d9d640 1 -- 172.21.15.167:0/3696679808 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8f80cca90 con 0x7fb8f806ed60 2024-05-11T08:05:56.490 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:56.487+0000 7fb8effff640 1 --2- 172.21.15.167:0/3696679808 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb8f806f140 0x7fb8f80cddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T08:05:56.490 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:56.487+0000 7fb8f759e640 1 --2- 172.21.15.167:0/3696679808 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb8f8160c00 0x7fb8f8147720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T08:05:56.490 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:56.487+0000 7fb8f6d9d640 1 --2- 172.21.15.167:0/3696679808 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb8f806ed60 0x7fb8f80cc550 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb8e800c490 tx=0x7fb8e800c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:05:56.490 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:56.487+0000 7fb8ef7fe640 1 -- 172.21.15.167:0/3696679808 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8e8018070 con 0x7fb8f806ed60 2024-05-11T08:05:56.490 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:56.487+0000 7fb8ef7fe640 1 -- 172.21.15.167:0/3696679808 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8e8007500 con 0x7fb8f806ed60 2024-05-11T08:05:56.491 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:56.487+0000 7fb8fd254640 1 -- 172.21.15.167:0/3696679808 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8f80ccd10 con 0x7fb8f806ed60 2024-05-11T08:05:56.491 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:56.487+0000 7fb8ef7fe640 1 -- 172.21.15.167:0/3696679808 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8e8004a20 con 0x7fb8f806ed60 2024-05-11T08:05:56.491 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:56.487+0000 7fb8fd254640 1 -- 172.21.15.167:0/3696679808 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8f80cd2d0 con 0x7fb8f806ed60 2024-05-11T08:05:56.491 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:56.488+0000 7fb8fd254640 1 -- 172.21.15.167:0/3696679808 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8f8069630 con 0x7fb8f806ed60 2024-05-11T08:05:56.491 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:56.489+0000 7fb8ef7fe640 1 -- 172.21.15.167:0/3696679808 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fb8e801b070 con 0x7fb8f806ed60 2024-05-11T08:05:56.492 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:56.489+0000 7fb8ef7fe640 1 --2- 172.21.15.167:0/3696679808 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fb8d80762d0 0x7fb8d8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:05:56.492 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:56.489+0000 7fb8ef7fe640 1 -- 172.21.15.167:0/3696679808 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 6007+0+0 (secure 0 0 0) 0x7fb8e8096980 con 0x7fb8f806ed60 2024-05-11T08:05:56.492 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:56.489+0000 7fb8effff640 1 --2- 172.21.15.167:0/3696679808 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fb8d80762d0 0x7fb8d8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:05:56.492 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:56.490+0000 7fb8effff640 1 --2- 172.21.15.167:0/3696679808 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fb8d80762d0 0x7fb8d8078790 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb8f8152e80 tx=0x7fb8e0008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:05:56.493 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:56.494+0000 7fb8ef7fe640 1 -- 172.21.15.167:0/3696679808 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fb8f80cca90 con 0x7fb8f806ed60 2024-05-11T08:05:56.676 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:56.676+0000 7fb8fd254640 1 -- 172.21.15.167:0/3696679808 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fb8f8153450 con 0x7fb8f806ed60 2024-05-11T08:05:58.378 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:58.378+0000 7fb8ef7fe640 1 -- 172.21.15.167:0/3696679808 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v25) ==== 228+0+0 (secure 0 0 0) 0x7fb8e8060240 con 0x7fb8f806ed60 2024-05-11T08:05:58.378 INFO:teuthology.orchestra.run.smithi167.stderr:pool 'cephfs_data_ec' created 2024-05-11T08:05:58.381 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:58.381+0000 7fb8fd254640 1 -- 172.21.15.167:0/3696679808 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fb8d80762d0 msgr2=0x7fb8d8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:05:58.381 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:58.381+0000 7fb8fd254640 1 --2- 172.21.15.167:0/3696679808 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fb8d80762d0 0x7fb8d8078790 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb8f8152e80 tx=0x7fb8e0008040 comp rx=0 tx=0).stop 2024-05-11T08:05:58.381 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:58.381+0000 7fb8fd254640 1 -- 172.21.15.167:0/3696679808 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb8f806ed60 msgr2=0x7fb8f80cc550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:05:58.381 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:58.381+0000 7fb8fd254640 1 --2- 172.21.15.167:0/3696679808 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb8f806ed60 0x7fb8f80cc550 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb8e800c490 tx=0x7fb8e800c950 comp rx=0 tx=0).stop 2024-05-11T08:05:58.381 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:58.381+0000 7fb8fd254640 1 -- 172.21.15.167:0/3696679808 shutdown_connections 2024-05-11T08:05:58.381 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:58.381+0000 7fb8fd254640 1 --2- 172.21.15.167:0/3696679808 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fb8d80762d0 0x7fb8d8078790 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:58.382 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:58.381+0000 7fb8fd254640 1 --2- 172.21.15.167:0/3696679808 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb8f8160c00 0x7fb8f8147720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:58.382 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:58.381+0000 7fb8fd254640 1 --2- 172.21.15.167:0/3696679808 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb8f806f140 0x7fb8f80cddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:58.382 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:58.381+0000 7fb8fd254640 1 --2- 172.21.15.167:0/3696679808 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb8f806ed60 0x7fb8f80cc550 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:58.382 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:58.381+0000 7fb8fd254640 1 -- 172.21.15.167:0/3696679808 >> 172.21.15.167:0/3696679808 conn(0x7fb8f80d1ed0 msgr2=0x7fb8f815e750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:05:58.382 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:58.381+0000 7fb8fd254640 1 -- 172.21.15.167:0/3696679808 shutdown_connections 2024-05-11T08:05:58.382 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:58.381+0000 7fb8fd254640 1 -- 172.21.15.167:0/3696679808 wait complete. 2024-05-11T08:05:58.400 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-05-11T08:05:58.509 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:58.509+0000 7f3516990640 1 Processor -- start 2024-05-11T08:05:58.509 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:58.509+0000 7f3516990640 1 -- start start 2024-05-11T08:05:58.512 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:58.510+0000 7f3516990640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f351015f430 0x7f35100b8390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:05:58.512 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:58.510+0000 7f3516990640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f3510163a20 0x7f351006d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:05:58.512 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:58.510+0000 7f3516990640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f3510162df0 0x7f35101632a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:05:58.512 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:58.510+0000 7f3516990640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f35100d2cf0 con 0x7f3510162df0 2024-05-11T08:05:58.513 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:58.510+0000 7f3516990640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f35100b88d0 con 0x7f3510163a20 2024-05-11T08:05:58.513 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:58.510+0000 7f3516990640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f351006dba0 con 0x7f351015f430 2024-05-11T08:05:58.513 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:58.510+0000 7f3514f2d640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f3510162df0 0x7f35101632a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:05:58.513 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:58.510+0000 7f3514f2d640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f3510162df0 0x7f35101632a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:58948/0 (socket says 172.21.15.167:58948) 2024-05-11T08:05:58.513 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:58.510+0000 7f3514f2d640 1 -- 172.21.15.167:0/3851151543 learned_addr learned my addr 172.21.15.167:0/3851151543 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:05:58.513 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:58.510+0000 7f350ffff640 1 --2- 172.21.15.167:0/3851151543 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f351015f430 0x7f35100b8390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:05:58.513 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:58.510+0000 7f3514f2d640 1 -- 172.21.15.167:0/3851151543 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f351015f430 msgr2=0x7f35100b8390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:05:58.513 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:58.510+0000 7f350f7fe640 1 --2- 172.21.15.167:0/3851151543 >> v2:172.21.15.204:3300/0 conn(0x7f3510163a20 0x7f351006d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:05:58.513 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:58.510+0000 7f3514f2d640 1 --2- 172.21.15.167:0/3851151543 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f351015f430 0x7f35100b8390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:58.513 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:58.510+0000 7f3514f2d640 1 -- 172.21.15.167:0/3851151543 >> v2:172.21.15.204:3300/0 conn(0x7f3510163a20 msgr2=0x7f351006d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:05:58.513 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:58.510+0000 7f3514f2d640 1 --2- 172.21.15.167:0/3851151543 >> v2:172.21.15.204:3300/0 conn(0x7f3510163a20 0x7f351006d200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:58.513 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:58.510+0000 7f3514f2d640 1 -- 172.21.15.167:0/3851151543 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35100d2f40 con 0x7f3510162df0 2024-05-11T08:05:58.513 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:58.510+0000 7f350ffff640 1 --2- 172.21.15.167:0/3851151543 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f351015f430 0x7f35100b8390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T08:05:58.513 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:58.511+0000 7f3514f2d640 1 --2- 172.21.15.167:0/3851151543 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3510162df0 0x7f35101632a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f35100b7d70 tx=0x7f350802a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1edf63cc3ca4dfef server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:05:58.513 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:58.511+0000 7f350effd640 1 -- 172.21.15.167:0/3851151543 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3508036070 con 0x7f3510162df0 2024-05-11T08:05:58.514 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:58.511+0000 7f350effd640 1 -- 172.21.15.167:0/3851151543 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3508031600 con 0x7f3510162df0 2024-05-11T08:05:58.514 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:58.511+0000 7f350effd640 1 -- 172.21.15.167:0/3851151543 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35080319d0 con 0x7f3510162df0 2024-05-11T08:05:58.514 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:58.512+0000 7f3516990640 1 -- 172.21.15.167:0/3851151543 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3510162df0 msgr2=0x7f35101632a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:05:58.514 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:58.512+0000 7f3516990640 1 --2- 172.21.15.167:0/3851151543 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3510162df0 0x7f35101632a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f35100b7d70 tx=0x7f350802a610 comp rx=0 tx=0).stop 2024-05-11T08:05:58.514 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:58.512+0000 7f3516990640 1 -- 172.21.15.167:0/3851151543 shutdown_connections 2024-05-11T08:05:58.514 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:58.512+0000 7f3516990640 1 --2- 172.21.15.167:0/3851151543 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3510162df0 0x7f35101632a0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:58.514 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:58.512+0000 7f3516990640 1 --2- 172.21.15.167:0/3851151543 >> v2:172.21.15.204:3300/0 conn(0x7f3510163a20 0x7f351006d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:58.514 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:58.512+0000 7f3516990640 1 --2- 172.21.15.167:0/3851151543 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f351015f430 0x7f35100b8390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:58.514 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:58.512+0000 7f3516990640 1 -- 172.21.15.167:0/3851151543 >> 172.21.15.167:0/3851151543 conn(0x7f35100d5300 msgr2=0x7f35100d5710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:05:58.514 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:58.512+0000 7f3516990640 1 -- 172.21.15.167:0/3851151543 shutdown_connections 2024-05-11T08:05:58.514 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:58.512+0000 7f3516990640 1 -- 172.21.15.167:0/3851151543 wait complete. 2024-05-11T08:05:58.517 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:58.513+0000 7f3516990640 1 Processor -- start 2024-05-11T08:05:58.517 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:58.513+0000 7f3516990640 1 -- start start 2024-05-11T08:05:58.517 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:58.514+0000 7f3516990640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f351015f430 0x7f351015bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:05:58.517 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:58.514+0000 7f3516990640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3510162df0 0x7f35100ce3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:05:58.517 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:58.514+0000 7f3516990640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3510163a20 0x7f35101eaea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:05:58.517 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:58.514+0000 7f3516990640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f351006d740 con 0x7f3510162df0 2024-05-11T08:05:58.517 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:58.514+0000 7f3516990640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f35100d3500 con 0x7f3510163a20 2024-05-11T08:05:58.517 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:58.514+0000 7f3516990640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f351006d8c0 con 0x7f351015f430 2024-05-11T08:05:58.517 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:58.514+0000 7f350f7fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3510162df0 0x7f35100ce3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:05:58.518 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:58.514+0000 7f350f7fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3510162df0 0x7f35100ce3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:58952/0 (socket says 172.21.15.167:58952) 2024-05-11T08:05:58.518 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:58.514+0000 7f350f7fe640 1 -- 172.21.15.167:0/2488409180 learned_addr learned my addr 172.21.15.167:0/2488409180 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:05:58.518 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:58.514+0000 7f350ffff640 1 --2- 172.21.15.167:0/2488409180 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f351015f430 0x7f351015bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:05:58.518 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:58.514+0000 7f3514f2d640 1 --2- 172.21.15.167:0/2488409180 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3510163a20 0x7f35101eaea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:05:58.518 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:58.514+0000 7f350f7fe640 1 -- 172.21.15.167:0/2488409180 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f351015f430 msgr2=0x7f351015bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:05:58.518 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:58.514+0000 7f350f7fe640 1 --2- 172.21.15.167:0/2488409180 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f351015f430 0x7f351015bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:58.518 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:58.514+0000 7f350f7fe640 1 -- 172.21.15.167:0/2488409180 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3510163a20 msgr2=0x7f35101eaea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:05:58.518 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:58.514+0000 7f350f7fe640 1 --2- 172.21.15.167:0/2488409180 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3510163a20 0x7f35101eaea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:58.518 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:58.514+0000 7f350f7fe640 1 -- 172.21.15.167:0/2488409180 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35100cf1b0 con 0x7f3510162df0 2024-05-11T08:05:58.518 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:58.514+0000 7f350f7fe640 1 --2- 172.21.15.167:0/2488409180 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3510162df0 0x7f35100ce3a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f350400c820 tx=0x7f350400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:05:58.518 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:58.514+0000 7f350cff9640 1 -- 172.21.15.167:0/2488409180 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3504015070 con 0x7f3510162df0 2024-05-11T08:05:58.518 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:58.514+0000 7f350cff9640 1 -- 172.21.15.167:0/2488409180 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3504004240 con 0x7f3510162df0 2024-05-11T08:05:58.518 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:58.514+0000 7f350cff9640 1 -- 172.21.15.167:0/2488409180 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f350400ce10 con 0x7f3510162df0 2024-05-11T08:05:58.519 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:58.514+0000 7f3516990640 1 -- 172.21.15.167:0/2488409180 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35100cebf0 con 0x7f3510162df0 2024-05-11T08:05:58.519 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:58.515+0000 7f3516990640 1 -- 172.21.15.167:0/2488409180 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3510160280 con 0x7f3510162df0 2024-05-11T08:05:58.519 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:58.516+0000 7f3516990640 1 -- 172.21.15.167:0/2488409180 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3510069590 con 0x7f3510162df0 2024-05-11T08:05:58.519 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:58.516+0000 7f350cff9640 1 -- 172.21.15.167:0/2488409180 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f3504007830 con 0x7f3510162df0 2024-05-11T08:05:58.519 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:58.517+0000 7f350cff9640 1 --2- 172.21.15.167:0/2488409180 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f34fc076300 0x7f34fc0787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:05:58.519 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:58.517+0000 7f350ffff640 1 --2- 172.21.15.167:0/2488409180 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f34fc076300 0x7f34fc0787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:05:58.519 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:58.517+0000 7f350cff9640 1 -- 172.21.15.167:0/2488409180 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6500+0+0 (secure 0 0 0) 0x7f3504097600 con 0x7f3510162df0 2024-05-11T08:05:58.519 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:58.517+0000 7f350ffff640 1 --2- 172.21.15.167:0/2488409180 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f34fc076300 0x7f34fc0787c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3500006fd0 tx=0x7f3500008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:05:58.521 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:58.522+0000 7f350cff9640 1 -- 172.21.15.167:0/2488409180 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f3504060680 con 0x7f3510162df0 2024-05-11T08:05:58.704 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:58.704+0000 7f3516990640 1 -- 172.21.15.167:0/2488409180 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7f35100b8350 con 0x7f3510162df0 2024-05-11T08:05:59.383 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:59.383+0000 7f350cff9640 1 -- 172.21.15.167:0/2488409180 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v26) ==== 169+0+0 (secure 0 0 0) 0x7f3504064980 con 0x7f3510162df0 2024-05-11T08:05:59.383 INFO:teuthology.orchestra.run.smithi167.stderr:set pool 4 allow_ec_overwrites to true 2024-05-11T08:05:59.388 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:59.388+0000 7f3516990640 1 -- 172.21.15.167:0/2488409180 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f34fc076300 msgr2=0x7f34fc0787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:05:59.388 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:59.388+0000 7f3516990640 1 --2- 172.21.15.167:0/2488409180 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f34fc076300 0x7f34fc0787c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3500006fd0 tx=0x7f3500008040 comp rx=0 tx=0).stop 2024-05-11T08:05:59.388 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:59.388+0000 7f3516990640 1 -- 172.21.15.167:0/2488409180 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3510162df0 msgr2=0x7f35100ce3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:05:59.388 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:59.388+0000 7f3516990640 1 --2- 172.21.15.167:0/2488409180 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3510162df0 0x7f35100ce3a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f350400c820 tx=0x7f350400cce0 comp rx=0 tx=0).stop 2024-05-11T08:05:59.388 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:59.388+0000 7f3516990640 1 -- 172.21.15.167:0/2488409180 shutdown_connections 2024-05-11T08:05:59.388 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:59.388+0000 7f3516990640 1 --2- 172.21.15.167:0/2488409180 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f34fc076300 0x7f34fc0787c0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:59.388 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:59.388+0000 7f3516990640 1 --2- 172.21.15.167:0/2488409180 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3510163a20 0x7f35101eaea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:59.388 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:59.388+0000 7f3516990640 1 --2- 172.21.15.167:0/2488409180 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3510162df0 0x7f35100ce3a0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:59.388 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:59.388+0000 7f3516990640 1 --2- 172.21.15.167:0/2488409180 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f351015f430 0x7f351015bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:59.388 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:59.388+0000 7f3516990640 1 -- 172.21.15.167:0/2488409180 >> 172.21.15.167:0/2488409180 conn(0x7f35100d5300 msgr2=0x7f35100d56e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:05:59.389 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:59.388+0000 7f3516990640 1 -- 172.21.15.167:0/2488409180 shutdown_connections 2024-05-11T08:05:59.389 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:59.388+0000 7f3516990640 1 -- 172.21.15.167:0/2488409180 wait complete. 2024-05-11T08:05:59.414 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-05-11T08:05:59.519 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:59.519+0000 7f510899d640 1 Processor -- start 2024-05-11T08:05:59.519 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:59.519+0000 7f510899d640 1 -- start start 2024-05-11T08:05:59.521 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:59.520+0000 7f510899d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f510406d200 0x7f510406d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:05:59.521 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:59.520+0000 7f510899d640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5104159d50 con 0x7f5104159970 2024-05-11T08:05:59.521 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:59.520+0000 7f510899d640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51040d04b0 con 0x7f51040d2db0 2024-05-11T08:05:59.521 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:59.520+0000 7f510899d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f51040d0790 con 0x7f510406d200 2024-05-11T08:05:59.521 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:59.520+0000 7f510259c640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f5104159970 0x7f51040d1270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:40618/0 (socket says 172.21.15.167:40618) 2024-05-11T08:05:59.521 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:59.520+0000 7f510259c640 1 -- 172.21.15.167:0/684322724 learned_addr learned my addr 172.21.15.167:0/684322724 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T08:05:59.522 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:59.520+0000 7f50fbfff640 1 --2- 172.21.15.167:0/684322724 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f510406d200 0x7f510406d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:05:59.522 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:59.520+0000 7f5101d9b640 1 -- 172.21.15.167:0/684322724 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3368122727 0 0) 0x7f5104159d50 con 0x7f5104159970 2024-05-11T08:05:59.522 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:59.520+0000 7f5101d9b640 1 -- 172.21.15.167:0/684322724 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f50f0003520 con 0x7f5104159970 2024-05-11T08:05:59.522 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:59.520+0000 7f5101d9b640 1 -- 172.21.15.167:0/684322724 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1844713149 0 0) 0x7f51040d04b0 con 0x7f51040d2db0 2024-05-11T08:05:59.522 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:59.520+0000 7f5101d9b640 1 -- 172.21.15.167:0/684322724 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5104159d50 con 0x7f51040d2db0 2024-05-11T08:05:59.522 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:59.520+0000 7f5101d9b640 1 -- 172.21.15.167:0/684322724 <== mon.0 v1:172.21.15.167:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1553445292 0 0) 0x7f50f0003520 con 0x7f5104159970 2024-05-11T08:05:59.522 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:59.520+0000 7f50fbfff640 1 -- 172.21.15.167:0/684322724 >> v1:172.21.15.204:6789/0 conn(0x7f51040d2db0 legacy=0x7f5104162f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:05:59.522 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:59.520+0000 7f50fbfff640 1 -- 172.21.15.167:0/684322724 >> v1:172.21.15.167:6789/0 conn(0x7f5104159970 legacy=0x7f51040d1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:05:59.522 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:59.520+0000 7f50fbfff640 1 -- 172.21.15.167:0/684322724 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51040d0a70 con 0x7f510406d200 2024-05-11T08:05:59.522 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:59.521+0000 7f50fbfff640 1 --2- 172.21.15.167:0/684322724 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f510406d200 0x7f510406d660 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f50ec009800 tx=0x7f50ec02a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=776147df49e7e7ce server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:05:59.522 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:59.521+0000 7f5101d9b640 1 -- 172.21.15.167:0/684322724 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50ec037070 con 0x7f510406d200 2024-05-11T08:05:59.522 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:59.521+0000 7f5101d9b640 1 -- 172.21.15.167:0/684322724 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50ec02ac50 con 0x7f510406d200 2024-05-11T08:05:59.522 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:59.521+0000 7f5101d9b640 1 -- 172.21.15.167:0/684322724 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50f4001670 con 0x7f510406d200 2024-05-11T08:05:59.523 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:59.522+0000 7f510899d640 1 -- 172.21.15.167:0/684322724 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f510406d200 msgr2=0x7f510406d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:05:59.524 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:59.522+0000 7f510899d640 1 --2- 172.21.15.167:0/684322724 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f510406d200 0x7f510406d660 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f50ec009800 tx=0x7f50ec02a1c0 comp rx=0 tx=0).stop 2024-05-11T08:05:59.524 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:59.522+0000 7f510899d640 1 -- 172.21.15.167:0/684322724 shutdown_connections 2024-05-11T08:05:59.524 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:59.522+0000 7f510899d640 1 --2- 172.21.15.167:0/684322724 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f510406d200 0x7f510406d660 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:59.524 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:59.522+0000 7f510899d640 1 -- 172.21.15.167:0/684322724 >> 172.21.15.167:0/684322724 conn(0x7f51040d51e0 msgr2=0x7f51040d55f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:05:59.524 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:59.522+0000 7f510899d640 1 -- 172.21.15.167:0/684322724 shutdown_connections 2024-05-11T08:05:59.524 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:59.522+0000 7f510899d640 1 -- 172.21.15.167:0/684322724 wait complete. 2024-05-11T08:05:59.524 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:59.523+0000 7f510899d640 1 Processor -- start 2024-05-11T08:05:59.524 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:59.523+0000 7f510899d640 1 -- start start 2024-05-11T08:05:59.526 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:59.524+0000 7f510899d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f510406d200 0x7f510415fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:05:59.526 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:59.524+0000 7f510899d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f51040d2db0 0x7f51040cae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:05:59.527 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:59.524+0000 7f510899d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5104159970 0x7f51040cd300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:05:59.527 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:59.524+0000 7f510899d640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f51040d1980 con 0x7f510406d200 2024-05-11T08:05:59.527 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:59.524+0000 7f510899d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5104163d00 con 0x7f5104159970 2024-05-11T08:05:59.527 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:59.524+0000 7f510899d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f5104163620 con 0x7f51040d2db0 2024-05-11T08:05:59.527 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:59.524+0000 7f510259c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f510406d200 0x7f510415fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:05:59.527 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:59.524+0000 7f510259c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f510406d200 0x7f510415fd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:58954/0 (socket says 172.21.15.167:58954) 2024-05-11T08:05:59.527 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:59.524+0000 7f510259c640 1 -- 172.21.15.167:0/1283760304 learned_addr learned my addr 172.21.15.167:0/1283760304 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:05:59.527 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:59.524+0000 7f50fbfff640 1 --2- 172.21.15.167:0/1283760304 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f51040d2db0 0x7f51040cae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:05:59.527 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:59.524+0000 7f5102d9d640 1 --2- 172.21.15.167:0/1283760304 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5104159970 0x7f51040cd300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:05:59.527 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:59.524+0000 7f510259c640 1 -- 172.21.15.167:0/1283760304 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f51040d2db0 msgr2=0x7f51040cae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:05:59.527 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:59.524+0000 7f510259c640 1 --2- 172.21.15.167:0/1283760304 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f51040d2db0 0x7f51040cae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:59.527 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:59.524+0000 7f510259c640 1 -- 172.21.15.167:0/1283760304 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5104159970 msgr2=0x7f51040cd300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:05:59.527 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:59.524+0000 7f510259c640 1 --2- 172.21.15.167:0/1283760304 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5104159970 0x7f51040cd300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:05:59.527 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:59.524+0000 7f510259c640 1 -- 172.21.15.167:0/1283760304 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51040cda50 con 0x7f510406d200 2024-05-11T08:05:59.528 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:59.524+0000 7f5102d9d640 1 --2- 172.21.15.167:0/1283760304 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5104159970 0x7f51040cd300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T08:05:59.528 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:59.525+0000 7f510259c640 1 --2- 172.21.15.167:0/1283760304 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f510406d200 0x7f510415fd40 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f50f400c700 tx=0x7f50f400cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:05:59.528 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:59.525+0000 7f50faffd640 1 -- 172.21.15.167:0/1283760304 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50f4005ad0 con 0x7f510406d200 2024-05-11T08:05:59.528 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:59.525+0000 7f50faffd640 1 -- 172.21.15.167:0/1283760304 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50f4005c70 con 0x7f510406d200 2024-05-11T08:05:59.528 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:59.525+0000 7f50faffd640 1 -- 172.21.15.167:0/1283760304 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50f4007290 con 0x7f510406d200 2024-05-11T08:05:59.528 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:59.525+0000 7f510899d640 1 -- 172.21.15.167:0/1283760304 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51040cdd30 con 0x7f510406d200 2024-05-11T08:05:59.528 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:59.525+0000 7f510899d640 1 -- 172.21.15.167:0/1283760304 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51040ce2f0 con 0x7f510406d200 2024-05-11T08:05:59.528 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:59.526+0000 7f510899d640 1 -- 172.21.15.167:0/1283760304 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51040763e0 con 0x7f510406d200 2024-05-11T08:05:59.528 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:59.527+0000 7f50faffd640 1 -- 172.21.15.167:0/1283760304 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f50f4007430 con 0x7f510406d200 2024-05-11T08:05:59.529 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:59.527+0000 7f50faffd640 1 --2- 172.21.15.167:0/1283760304 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f50e40762d0 0x7f50e4078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:05:59.529 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:59.527+0000 7f50fbfff640 1 --2- 172.21.15.167:0/1283760304 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f50e40762d0 0x7f50e4078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:05:59.529 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:59.527+0000 7f50faffd640 1 -- 172.21.15.167:0/1283760304 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6500+0+0 (secure 0 0 0) 0x7f50f4097610 con 0x7f510406d200 2024-05-11T08:05:59.529 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:59.528+0000 7f50fbfff640 1 --2- 172.21.15.167:0/1283760304 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f50e40762d0 0x7f50e4078790 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f50ec034000 tx=0x7f50ec02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:05:59.531 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:59.531+0000 7f50faffd640 1 -- 172.21.15.167:0/1283760304 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f51040763e0 con 0x7f510406d200 2024-05-11T08:05:59.755 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:05:59.755+0000 7f510899d640 1 -- 172.21.15.167:0/1283760304 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7f510406cb20 con 0x7f510406d200 2024-05-11T08:06:00.331 INFO:tasks.ceph.mds.b.smithi204.stderr:2024-05-11T08:06:00.330+0000 7fb112f60640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-11T08:06:00.392 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.392+0000 7f50faffd640 1 -- 172.21.15.167:0/1283760304 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v7) ==== 137+0+0 (secure 0 0 0) 0x7f510406cb20 con 0x7f510406d200 2024-05-11T08:06:00.393 INFO:teuthology.orchestra.run.smithi167.stderr:added data pool 4 to fsmap 2024-05-11T08:06:00.396 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.395+0000 7f510899d640 1 -- 172.21.15.167:0/1283760304 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f50e40762d0 msgr2=0x7f50e4078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:00.396 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.395+0000 7f510899d640 1 --2- 172.21.15.167:0/1283760304 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f50e40762d0 0x7f50e4078790 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f50ec034000 tx=0x7f50ec02e000 comp rx=0 tx=0).stop 2024-05-11T08:06:00.396 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.395+0000 7f510899d640 1 -- 172.21.15.167:0/1283760304 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f510406d200 msgr2=0x7f510415fd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:00.396 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.395+0000 7f510899d640 1 --2- 172.21.15.167:0/1283760304 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f510406d200 0x7f510415fd40 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f50f400c700 tx=0x7f50f400cbc0 comp rx=0 tx=0).stop 2024-05-11T08:06:00.396 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.395+0000 7f510899d640 1 -- 172.21.15.167:0/1283760304 shutdown_connections 2024-05-11T08:06:00.396 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.395+0000 7f510899d640 1 --2- 172.21.15.167:0/1283760304 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f50e40762d0 0x7f50e4078790 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:00.396 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.395+0000 7f510899d640 1 --2- 172.21.15.167:0/1283760304 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5104159970 0x7f51040cd300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:00.396 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.395+0000 7f510899d640 1 --2- 172.21.15.167:0/1283760304 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f51040d2db0 0x7f51040cae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:00.396 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.395+0000 7f510899d640 1 --2- 172.21.15.167:0/1283760304 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f510406d200 0x7f510415fd40 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:00.396 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.395+0000 7f510899d640 1 -- 172.21.15.167:0/1283760304 >> 172.21.15.167:0/1283760304 conn(0x7f51040d51e0 msgr2=0x7f5104162c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:00.396 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.395+0000 7f510899d640 1 -- 172.21.15.167:0/1283760304 shutdown_connections 2024-05-11T08:06:00.396 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.395+0000 7f510899d640 1 -- 172.21.15.167:0/1283760304 wait complete. 2024-05-11T08:06:00.409 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-11T08:06:00.515 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.515+0000 7fcee4cfc640 1 Processor -- start 2024-05-11T08:06:00.515 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.515+0000 7fcee4cfc640 1 -- start start 2024-05-11T08:06:00.516 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.516+0000 7fcee4cfc640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fcee015b860 0x7fcee006f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:00.517 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.516+0000 7fcee4cfc640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fcee0073210 0x7fcee00b47d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:00.517 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.516+0000 7fcee4cfc640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcee015bc40 con 0x7fcee00736a0 2024-05-11T08:06:00.517 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.516+0000 7fcee4cfc640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7fcee006e410 con 0x7fcee015b860 2024-05-11T08:06:00.517 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.516+0000 7fcee4cfc640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fcee006e6f0 con 0x7fcee0073210 2024-05-11T08:06:00.517 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.516+0000 7fcede59c640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fcee015b860 0x7fcee006f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:00.517 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.516+0000 7fceddd9b640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7fcee00736a0 0x7fcee0072b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:40622/0 (socket says 172.21.15.167:40622) 2024-05-11T08:06:00.517 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.516+0000 7fceddd9b640 1 -- 172.21.15.167:0/4256710717 learned_addr learned my addr 172.21.15.167:0/4256710717 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T08:06:00.517 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.516+0000 7fceded9d640 1 --2- 172.21.15.167:0/4256710717 >> v2:172.21.15.204:3300/0 conn(0x7fcee0073210 0x7fcee00b47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:00.517 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.516+0000 7fcede59c640 1 -- 172.21.15.167:0/4256710717 >> v2:172.21.15.204:3300/0 conn(0x7fcee0073210 msgr2=0x7fcee00b47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:00.517 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.516+0000 7fcede59c640 1 --2- 172.21.15.167:0/4256710717 >> v2:172.21.15.204:3300/0 conn(0x7fcee0073210 0x7fcee00b47d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:00.517 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.516+0000 7fcede59c640 1 -- 172.21.15.167:0/4256710717 >> v1:172.21.15.167:6789/0 conn(0x7fcee00736a0 legacy=0x7fcee0072b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:00.517 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.516+0000 7fcedd59a640 1 -- 172.21.15.167:0/4256710717 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3878733617 0 0) 0x7fced4001110 con 0x7fcee00736a0 2024-05-11T08:06:00.517 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.516+0000 7fcede59c640 1 -- 172.21.15.167:0/4256710717 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcee006e9d0 con 0x7fcee015b860 2024-05-11T08:06:00.519 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.517+0000 7fcede59c640 1 --2- 172.21.15.167:0/4256710717 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcee015b860 0x7fcee006f4b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fcee00d0880 tx=0x7fced002b560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1d12e469128cb2d server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:00.519 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.517+0000 7fcedd59a640 1 -- 172.21.15.167:0/4256710717 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fced000b040 con 0x7fcee015b860 2024-05-11T08:06:00.519 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.517+0000 7fcedd59a640 1 -- 172.21.15.167:0/4256710717 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fced0030710 con 0x7fcee015b860 2024-05-11T08:06:00.519 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.517+0000 7fcedd59a640 1 -- 172.21.15.167:0/4256710717 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fced0030ae0 con 0x7fcee015b860 2024-05-11T08:06:00.519 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.518+0000 7fcee4cfc640 1 -- 172.21.15.167:0/4256710717 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcee015b860 msgr2=0x7fcee006f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:00.519 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.518+0000 7fcee4cfc640 1 --2- 172.21.15.167:0/4256710717 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcee015b860 0x7fcee006f4b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fcee00d0880 tx=0x7fced002b560 comp rx=0 tx=0).stop 2024-05-11T08:06:00.519 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.518+0000 7fcee4cfc640 1 -- 172.21.15.167:0/4256710717 shutdown_connections 2024-05-11T08:06:00.520 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.518+0000 7fcee4cfc640 1 --2- 172.21.15.167:0/4256710717 >> v2:172.21.15.204:3300/0 conn(0x7fcee0073210 0x7fcee00b47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:00.520 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.518+0000 7fcee4cfc640 1 --2- 172.21.15.167:0/4256710717 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcee015b860 0x7fcee006f4b0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:00.520 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.518+0000 7fcee4cfc640 1 -- 172.21.15.167:0/4256710717 >> 172.21.15.167:0/4256710717 conn(0x7fcee00cd8f0 msgr2=0x7fcee00cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:00.520 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.518+0000 7fcee4cfc640 1 -- 172.21.15.167:0/4256710717 shutdown_connections 2024-05-11T08:06:00.520 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.518+0000 7fcee4cfc640 1 -- 172.21.15.167:0/4256710717 wait complete. 2024-05-11T08:06:00.520 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.519+0000 7fcee4cfc640 1 Processor -- start 2024-05-11T08:06:00.520 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.519+0000 7fcee4cfc640 1 -- start start 2024-05-11T08:06:00.523 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.520+0000 7fcee4cfc640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcee0073210 0x7fcee0156360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:00.523 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.520+0000 7fcee4cfc640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcee00736a0 0x7fcee014e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:00.523 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.520+0000 7fcee4cfc640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcee015b860 0x7fcee01f3280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:00.523 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.520+0000 7fcee4cfc640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fcee0072640 con 0x7fcee015b860 2024-05-11T08:06:00.523 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.520+0000 7fcee4cfc640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fcee006ef90 con 0x7fcee00736a0 2024-05-11T08:06:00.523 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.520+0000 7fcee4cfc640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fcee00727c0 con 0x7fcee0073210 2024-05-11T08:06:00.523 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.520+0000 7fceded9d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcee015b860 0x7fcee01f3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:00.523 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.520+0000 7fceded9d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcee015b860 0x7fcee01f3280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:58964/0 (socket says 172.21.15.167:58964) 2024-05-11T08:06:00.523 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.520+0000 7fceded9d640 1 -- 172.21.15.167:0/3217311384 learned_addr learned my addr 172.21.15.167:0/3217311384 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:00.523 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.520+0000 7fceddd9b640 1 --2- 172.21.15.167:0/3217311384 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcee00736a0 0x7fcee014e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:00.523 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.520+0000 7fcede59c640 1 --2- 172.21.15.167:0/3217311384 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcee0073210 0x7fcee0156360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:00.523 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.520+0000 7fceded9d640 1 -- 172.21.15.167:0/3217311384 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcee0073210 msgr2=0x7fcee0156360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:00.523 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.520+0000 7fceded9d640 1 --2- 172.21.15.167:0/3217311384 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcee0073210 0x7fcee0156360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:00.524 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.520+0000 7fceded9d640 1 -- 172.21.15.167:0/3217311384 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcee00736a0 msgr2=0x7fcee014e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:00.524 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.520+0000 7fceded9d640 1 --2- 172.21.15.167:0/3217311384 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcee00736a0 0x7fcee014e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:00.524 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.520+0000 7fceded9d640 1 -- 172.21.15.167:0/3217311384 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcee0156c40 con 0x7fcee015b860 2024-05-11T08:06:00.524 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.521+0000 7fceded9d640 1 --2- 172.21.15.167:0/3217311384 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcee015b860 0x7fcee01f3280 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fced8007e40 tx=0x7fced800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:00.524 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.521+0000 7fceceffd640 1 -- 172.21.15.167:0/3217311384 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fced8014620 con 0x7fcee015b860 2024-05-11T08:06:00.524 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.521+0000 7fceceffd640 1 -- 172.21.15.167:0/3217311384 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fced8019070 con 0x7fcee015b860 2024-05-11T08:06:00.524 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.521+0000 7fcee4cfc640 1 -- 172.21.15.167:0/3217311384 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcee0156f80 con 0x7fcee015b860 2024-05-11T08:06:00.524 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.521+0000 7fceceffd640 1 -- 172.21.15.167:0/3217311384 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fced8014a20 con 0x7fcee015b860 2024-05-11T08:06:00.524 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.521+0000 7fcee4cfc640 1 -- 172.21.15.167:0/3217311384 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcee0157200 con 0x7fcee015b860 2024-05-11T08:06:00.524 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.522+0000 7fcee4cfc640 1 -- 172.21.15.167:0/3217311384 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcee0069590 con 0x7fcee015b860 2024-05-11T08:06:00.524 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.522+0000 7fceceffd640 1 -- 172.21.15.167:0/3217311384 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fced8014bc0 con 0x7fcee015b860 2024-05-11T08:06:00.524 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.523+0000 7fceceffd640 1 --2- 172.21.15.167:0/3217311384 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fcec00762d0 0x7fcec0078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:00.524 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.523+0000 7fcede59c640 1 --2- 172.21.15.167:0/3217311384 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fcec00762d0 0x7fcec0078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:00.524 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.523+0000 7fceceffd640 1 -- 172.21.15.167:0/3217311384 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fced8098260 con 0x7fcee015b860 2024-05-11T08:06:00.525 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.523+0000 7fcede59c640 1 --2- 172.21.15.167:0/3217311384 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fcec00762d0 0x7fcec0078790 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fced0035000 tx=0x7fced000c040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:00.527 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.527+0000 7fceceffd640 1 -- 172.21.15.167:0/3217311384 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fcee0156f80 con 0x7fcee015b860 2024-05-11T08:06:00.743 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.743+0000 7fcee4cfc640 1 -- 172.21.15.167:0/3217311384 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcee006f490 con 0x7fcee015b860 2024-05-11T08:06:00.743 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.743+0000 7fceceffd640 1 -- 172.21.15.167:0/3217311384 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5183 (secure 0 0 0) 0x7fced8061880 con 0x7fcee015b860 2024-05-11T08:06:00.744 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T08:06:00.744 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6834/506789040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":506789040},{"type":"v1","addr":"172.21.15.204:6834","nonce":506789040}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4287,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1448126000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":1448126000},{"type":"v1","addr":"172.21.15.167:6837","nonce":1448126000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4290,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/766834562","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":766834562},{"type":"v1","addr":"172.21.15.167:6836","nonce":766834562}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-11T08:05:54.711448+0000","modified":"2024-05-11T08:05:59.756961+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4305},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4305":{"gid":4305,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.204:6835/1247774569","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6833","nonce":1247774569},{"type":"v1","addr":"172.21.15.204:6835","nonce":1247774569}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":4305,"qdb_cluster":[4305]},"id":1}]} 2024-05-11T08:06:00.744 INFO:teuthology.orchestra.run.smithi167.stderr:dumped fsmap epoch 7 2024-05-11T08:06:00.746 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.745+0000 7fcee4cfc640 1 -- 172.21.15.167:0/3217311384 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fcec00762d0 msgr2=0x7fcec0078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:00.746 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.745+0000 7fcee4cfc640 1 --2- 172.21.15.167:0/3217311384 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fcec00762d0 0x7fcec0078790 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fced0035000 tx=0x7fced000c040 comp rx=0 tx=0).stop 2024-05-11T08:06:00.746 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.745+0000 7fcee4cfc640 1 -- 172.21.15.167:0/3217311384 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcee015b860 msgr2=0x7fcee01f3280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:00.746 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.745+0000 7fcee4cfc640 1 --2- 172.21.15.167:0/3217311384 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcee015b860 0x7fcee01f3280 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fced8007e40 tx=0x7fced800e4b0 comp rx=0 tx=0).stop 2024-05-11T08:06:00.746 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.745+0000 7fcee4cfc640 1 -- 172.21.15.167:0/3217311384 shutdown_connections 2024-05-11T08:06:00.746 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.745+0000 7fcee4cfc640 1 --2- 172.21.15.167:0/3217311384 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fcec00762d0 0x7fcec0078790 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:00.746 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.745+0000 7fcee4cfc640 1 --2- 172.21.15.167:0/3217311384 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcee015b860 0x7fcee01f3280 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:00.746 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.745+0000 7fcee4cfc640 1 --2- 172.21.15.167:0/3217311384 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcee00736a0 0x7fcee014e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:00.746 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.745+0000 7fcee4cfc640 1 --2- 172.21.15.167:0/3217311384 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcee0073210 0x7fcee0156360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:00.746 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.745+0000 7fcee4cfc640 1 -- 172.21.15.167:0/3217311384 >> 172.21.15.167:0/3217311384 conn(0x7fcee00cd8f0 msgr2=0x7fcee014f900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:00.747 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.745+0000 7fcee4cfc640 1 -- 172.21.15.167:0/3217311384 shutdown_connections 2024-05-11T08:06:00.747 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.745+0000 7fcee4cfc640 1 -- 172.21.15.167:0/3217311384 wait complete. 2024-05-11T08:06:00.756 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-11T08:06:00.856 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.856+0000 7fbe61075640 1 Processor -- start 2024-05-11T08:06:00.857 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.856+0000 7fbe61075640 1 -- start start 2024-05-11T08:06:00.858 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.857+0000 7fbe61075640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fbe5c1598d0 0x7fbe5c0d2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:00.859 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.857+0000 7fbe61075640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fbe5c0cb6c0 0x7fbe5c0d0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:00.859 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.857+0000 7fbe61075640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe5c0d1190 con 0x7fbe5c0d0640 2024-05-11T08:06:00.859 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.857+0000 7fbe61075640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7fbe5c06d500 con 0x7fbe5c0cb6c0 2024-05-11T08:06:00.859 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.857+0000 7fbe61075640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fbe5c06d7e0 con 0x7fbe5c1598d0 2024-05-11T08:06:00.859 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.857+0000 7fbe5b59e640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fbe5c0cb6c0 0x7fbe5c0d0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:00.859 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.857+0000 7fbe5a59c640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7fbe5c0d0640 0x7fbe5c0cafb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:40626/0 (socket says 172.21.15.167:40626) 2024-05-11T08:06:00.859 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.857+0000 7fbe5a59c640 1 -- 172.21.15.167:0/2436082681 learned_addr learned my addr 172.21.15.167:0/2436082681 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T08:06:00.859 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.857+0000 7fbe5b59e640 1 --2- 172.21.15.167:0/2436082681 >> v2:172.21.15.167:3300/0 conn(0x7fbe5c0cb6c0 0x7fbe5c0d0ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:58976/0 (socket says 172.21.15.167:58976) 2024-05-11T08:06:00.859 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.858+0000 7fbe5b59e640 1 -- 172.21.15.167:0/2436082681 >> v2:172.21.15.204:3300/0 conn(0x7fbe5c1598d0 msgr2=0x7fbe5c0d2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:00.859 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.858+0000 7fbe5ad9d640 1 --2- 172.21.15.167:0/2436082681 >> v2:172.21.15.204:3300/0 conn(0x7fbe5c1598d0 0x7fbe5c0d2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:00.859 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.858+0000 7fbe5b59e640 1 --2- 172.21.15.167:0/2436082681 >> v2:172.21.15.204:3300/0 conn(0x7fbe5c1598d0 0x7fbe5c0d2cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:00.859 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.858+0000 7fbe5b59e640 1 -- 172.21.15.167:0/2436082681 >> v1:172.21.15.167:6789/0 conn(0x7fbe5c0d0640 legacy=0x7fbe5c0cafb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:00.859 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.858+0000 7fbe5b59e640 1 -- 172.21.15.167:0/2436082681 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe5c06dac0 con 0x7fbe5c0cb6c0 2024-05-11T08:06:00.862 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.858+0000 7fbe59d9b640 1 -- 172.21.15.167:0/2436082681 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1931395415 0 0) 0x7fbe5c0d1190 con 0x7fbe5c0d0640 2024-05-11T08:06:00.862 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.858+0000 7fbe5b59e640 1 --2- 172.21.15.167:0/2436082681 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe5c0cb6c0 0x7fbe5c0d0ad0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c08d610 tx=0x7fbe5402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3429da807f1e357 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:00.862 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.858+0000 7fbe59d9b640 1 -- 172.21.15.167:0/2436082681 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe54007740 con 0x7fbe5c0cb6c0 2024-05-11T08:06:00.862 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.859+0000 7fbe59d9b640 1 -- 172.21.15.167:0/2436082681 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe540078e0 con 0x7fbe5c0cb6c0 2024-05-11T08:06:00.862 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.859+0000 7fbe59d9b640 1 -- 172.21.15.167:0/2436082681 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe54007cb0 con 0x7fbe5c0cb6c0 2024-05-11T08:06:00.866 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.859+0000 7fbe61075640 1 -- 172.21.15.167:0/2436082681 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe5c0cb6c0 msgr2=0x7fbe5c0d0ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:00.866 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.859+0000 7fbe61075640 1 --2- 172.21.15.167:0/2436082681 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe5c0cb6c0 0x7fbe5c0d0ad0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c08d610 tx=0x7fbe5402a680 comp rx=0 tx=0).stop 2024-05-11T08:06:00.866 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.859+0000 7fbe61075640 1 -- 172.21.15.167:0/2436082681 shutdown_connections 2024-05-11T08:06:00.866 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.859+0000 7fbe61075640 1 --2- 172.21.15.167:0/2436082681 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe5c0cb6c0 0x7fbe5c0d0ad0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:00.866 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.859+0000 7fbe61075640 1 --2- 172.21.15.167:0/2436082681 >> v2:172.21.15.204:3300/0 conn(0x7fbe5c1598d0 0x7fbe5c0d2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:00.866 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.859+0000 7fbe61075640 1 -- 172.21.15.167:0/2436082681 >> 172.21.15.167:0/2436082681 conn(0x7fbe5c0d1ba0 msgr2=0x7fbe5c0d1fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:00.866 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.859+0000 7fbe61075640 1 -- 172.21.15.167:0/2436082681 shutdown_connections 2024-05-11T08:06:00.866 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.859+0000 7fbe61075640 1 -- 172.21.15.167:0/2436082681 wait complete. 2024-05-11T08:06:00.866 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.861+0000 7fbe61075640 1 Processor -- start 2024-05-11T08:06:00.866 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.861+0000 7fbe61075640 1 -- start start 2024-05-11T08:06:00.866 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.861+0000 7fbe61075640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbe5c0cb6c0 0x7fbe5c155ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:00.867 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.861+0000 7fbe61075640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe5c0d0640 0x7fbe5c146ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:00.867 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.861+0000 7fbe61075640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe5c1598d0 0x7fbe5c0b7e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:00.867 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.861+0000 7fbe61075640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fbe5c06d1b0 con 0x7fbe5c0d0640 2024-05-11T08:06:00.867 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.861+0000 7fbe61075640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fbe5c159cb0 con 0x7fbe5c1598d0 2024-05-11T08:06:00.867 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.861+0000 7fbe61075640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fbe5c06dfe0 con 0x7fbe5c0cb6c0 2024-05-11T08:06:00.867 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.861+0000 7fbe5a59c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe5c0d0640 0x7fbe5c146ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:00.867 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.861+0000 7fbe5a59c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe5c0d0640 0x7fbe5c146ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:58992/0 (socket says 172.21.15.167:58992) 2024-05-11T08:06:00.867 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.861+0000 7fbe5a59c640 1 -- 172.21.15.167:0/1057143576 learned_addr learned my addr 172.21.15.167:0/1057143576 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:00.867 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.862+0000 7fbe5ad9d640 1 --2- 172.21.15.167:0/1057143576 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbe5c0cb6c0 0x7fbe5c155ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:00.867 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.862+0000 7fbe5b59e640 1 --2- 172.21.15.167:0/1057143576 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe5c1598d0 0x7fbe5c0b7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:00.867 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.862+0000 7fbe5a59c640 1 -- 172.21.15.167:0/1057143576 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbe5c0cb6c0 msgr2=0x7fbe5c155ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:00.867 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.862+0000 7fbe5a59c640 1 --2- 172.21.15.167:0/1057143576 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbe5c0cb6c0 0x7fbe5c155ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:00.867 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.862+0000 7fbe5a59c640 1 -- 172.21.15.167:0/1057143576 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe5c1598d0 msgr2=0x7fbe5c0b7e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:00.868 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.862+0000 7fbe5a59c640 1 --2- 172.21.15.167:0/1057143576 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe5c1598d0 0x7fbe5c0b7e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:00.868 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.862+0000 7fbe5a59c640 1 -- 172.21.15.167:0/1057143576 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe5c151210 con 0x7fbe5c0d0640 2024-05-11T08:06:00.868 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.862+0000 7fbe5b59e640 1 --2- 172.21.15.167:0/1057143576 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe5c1598d0 0x7fbe5c0b7e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T08:06:00.868 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.862+0000 7fbe5ad9d640 1 --2- 172.21.15.167:0/1057143576 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbe5c0cb6c0 0x7fbe5c155ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T08:06:00.868 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.862+0000 7fbe5a59c640 1 --2- 172.21.15.167:0/1057143576 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe5c0d0640 0x7fbe5c146ad0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fbe5000c490 tx=0x7fbe5000c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:00.868 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.862+0000 7fbe437fe640 1 -- 172.21.15.167:0/1057143576 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe50018070 con 0x7fbe5c0d0640 2024-05-11T08:06:00.868 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.862+0000 7fbe437fe640 1 -- 172.21.15.167:0/1057143576 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe50007500 con 0x7fbe5c0d0640 2024-05-11T08:06:00.868 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.862+0000 7fbe61075640 1 -- 172.21.15.167:0/1057143576 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe5c1563b0 con 0x7fbe5c0d0640 2024-05-11T08:06:00.868 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.862+0000 7fbe437fe640 1 -- 172.21.15.167:0/1057143576 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe50004a20 con 0x7fbe5c0d0640 2024-05-11T08:06:00.868 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.862+0000 7fbe61075640 1 -- 172.21.15.167:0/1057143576 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe5c152340 con 0x7fbe5c0d0640 2024-05-11T08:06:00.868 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.863+0000 7fbe61075640 1 -- 172.21.15.167:0/1057143576 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe5c069590 con 0x7fbe5c0d0640 2024-05-11T08:06:00.868 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.864+0000 7fbe437fe640 1 -- 172.21.15.167:0/1057143576 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fbe50004bc0 con 0x7fbe5c0d0640 2024-05-11T08:06:00.868 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.864+0000 7fbe437fe640 1 --2- 172.21.15.167:0/1057143576 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fbe48076300 0x7fbe480787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:00.868 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.864+0000 7fbe5ad9d640 1 --2- 172.21.15.167:0/1057143576 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fbe48076300 0x7fbe480787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:00.868 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.864+0000 7fbe437fe640 1 -- 172.21.15.167:0/1057143576 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fbe50096b40 con 0x7fbe5c0d0640 2024-05-11T08:06:00.869 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.865+0000 7fbe5ad9d640 1 --2- 172.21.15.167:0/1057143576 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fbe48076300 0x7fbe480787c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fbe4c00b820 tx=0x7fbe4c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:00.869 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:00.869+0000 7fbe437fe640 1 -- 172.21.15.167:0/1057143576 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fbe50060170 con 0x7fbe5c0d0640 2024-05-11T08:06:01.054 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.054+0000 7fbe61075640 1 -- 172.21.15.167:0/1057143576 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbe5c0d0c50 con 0x7fbe5c0d0640 2024-05-11T08:06:01.055 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.055+0000 7fbe437fe640 1 -- 172.21.15.167:0/1057143576 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18562 (secure 0 0 0) 0x7fbe5005fa70 con 0x7fbe5c0d0640 2024-05-11T08:06:01.055 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T08:06:01.055 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":27,"fsid":"2034778d-8ff1-4823-aa44-ded32b518241","created":"2024-05-11T08:05:31.612421+0000","modified":"2024-05-11T08:05:59.756806+0000","last_up_change":"2024-05-11T08:05:42.557382+0000","last_in_change":"2024-05-11T08:05:33.757311+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-11T08:05:43.931263+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-11T08:05:52.573726+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-11T08:05:53.670746+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-11T08:05:57.380272+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"587eaced-5ded-406d-884b-30c1caa8686b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":3863307356},{"type":"v1","addr":"172.21.15.167:6806","nonce":3863307356}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":3863307356},{"type":"v1","addr":"172.21.15.167:6810","nonce":3863307356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":3863307356},{"type":"v1","addr":"172.21.15.167:6815","nonce":3863307356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":3863307356},{"type":"v1","addr":"172.21.15.167:6813","nonce":3863307356}]},"public_addr":"172.21.15.167:6806/3863307356","cluster_addr":"172.21.15.167:6810/3863307356","heartbeat_back_addr":"172.21.15.167:6815/3863307356","heartbeat_front_addr":"172.21.15.167:6813/3863307356","state":["exists","up"]},{"osd":1,"uuid":"37c59104-2dd0-4cf9-85ea-56636f6b02a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":1585024048},{"type":"v1","addr":"172.21.15.167:6817","nonce":1585024048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1585024048},{"type":"v1","addr":"172.21.15.167:6819","nonce":1585024048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1585024048},{"type":"v1","addr":"172.21.15.167:6823","nonce":1585024048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":1585024048},{"type":"v1","addr":"172.21.15.167:6821","nonce":1585024048}]},"public_addr":"172.21.15.167:6817/1585024048","cluster_addr":"172.21.15.167:6819/1585024048","heartbeat_back_addr":"172.21.15.167:6823/1585024048","heartbeat_front_addr":"172.21.15.167:6821/1585024048","state":["exists","up"]},{"osd":2,"uuid":"2bb4f7f5-c698-451b-b4bb-78e716921291","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":3242166900},{"type":"v1","addr":"172.21.15.167:6801","nonce":3242166900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":3242166900},{"type":"v1","addr":"172.21.15.167:6803","nonce":3242166900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6809","nonce":3242166900},{"type":"v1","addr":"172.21.15.167:6811","nonce":3242166900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6805","nonce":3242166900},{"type":"v1","addr":"172.21.15.167:6807","nonce":3242166900}]},"public_addr":"172.21.15.167:6801/3242166900","cluster_addr":"172.21.15.167:6803/3242166900","heartbeat_back_addr":"172.21.15.167:6811/3242166900","heartbeat_front_addr":"172.21.15.167:6807/3242166900","state":["exists","up"]},{"osd":3,"uuid":"9dd26759-a338-49d2-8103-45d068b9c000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":453689293},{"type":"v1","addr":"172.21.15.167:6825","nonce":453689293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":453689293},{"type":"v1","addr":"172.21.15.167:6827","nonce":453689293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":453689293},{"type":"v1","addr":"172.21.15.167:6831","nonce":453689293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":453689293},{"type":"v1","addr":"172.21.15.167:6829","nonce":453689293}]},"public_addr":"172.21.15.167:6825/453689293","cluster_addr":"172.21.15.167:6827/453689293","heartbeat_back_addr":"172.21.15.167:6831/453689293","heartbeat_front_addr":"172.21.15.167:6829/453689293","state":["exists","up"]},{"osd":4,"uuid":"cb27250d-2a80-4c7b-8cef-3a7af05c8a16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1311831500},{"type":"v1","addr":"172.21.15.204:6801","nonce":1311831500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1311831500},{"type":"v1","addr":"172.21.15.204:6803","nonce":1311831500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1311831500},{"type":"v1","addr":"172.21.15.204:6807","nonce":1311831500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1311831500},{"type":"v1","addr":"172.21.15.204:6805","nonce":1311831500}]},"public_addr":"172.21.15.204:6801/1311831500","cluster_addr":"172.21.15.204:6803/1311831500","heartbeat_back_addr":"172.21.15.204:6807/1311831500","heartbeat_front_addr":"172.21.15.204:6805/1311831500","state":["exists","up"]},{"osd":5,"uuid":"92998541-ffec-430b-ab76-8c3cb7c2752d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2921847521},{"type":"v1","addr":"172.21.15.204:6825","nonce":2921847521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2921847521},{"type":"v1","addr":"172.21.15.204:6827","nonce":2921847521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2921847521},{"type":"v1","addr":"172.21.15.204:6831","nonce":2921847521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2921847521},{"type":"v1","addr":"172.21.15.204:6829","nonce":2921847521}]},"public_addr":"172.21.15.204:6825/2921847521","cluster_addr":"172.21.15.204:6827/2921847521","heartbeat_back_addr":"172.21.15.204:6831/2921847521","heartbeat_front_addr":"172.21.15.204:6829/2921847521","state":["exists","up"]},{"osd":6,"uuid":"9543479c-2ec5-42a8-8933-707bfed2e048","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":2889851077},{"type":"v1","addr":"172.21.15.204:6817","nonce":2889851077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":2889851077},{"type":"v1","addr":"172.21.15.204:6819","nonce":2889851077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":2889851077},{"type":"v1","addr":"172.21.15.204:6823","nonce":2889851077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":2889851077},{"type":"v1","addr":"172.21.15.204:6821","nonce":2889851077}]},"public_addr":"172.21.15.204:6817/2889851077","cluster_addr":"172.21.15.204:6819/2889851077","heartbeat_back_addr":"172.21.15.204:6823/2889851077","heartbeat_front_addr":"172.21.15.204:6821/2889851077","state":["exists","up"]},{"osd":7,"uuid":"c45a5647-8358-409e-af91-c05dda65f0b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":1949534184},{"type":"v1","addr":"172.21.15.204:6809","nonce":1949534184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":1949534184},{"type":"v1","addr":"172.21.15.204:6811","nonce":1949534184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1949534184},{"type":"v1","addr":"172.21.15.204:6815","nonce":1949534184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1949534184},{"type":"v1","addr":"172.21.15.204:6813","nonce":1949534184}]},"public_addr":"172.21.15.204:6809/1949534184","cluster_addr":"172.21.15.204:6811/1949534184","heartbeat_back_addr":"172.21.15.204:6815/1949534184","heartbeat_front_addr":"172.21.15.204:6813/1949534184","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.952253+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.940689+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.860682+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.929350+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.901627+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.957451+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.987249+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.951197+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-11T08:06:01.057 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.057+0000 7fbe61075640 1 -- 172.21.15.167:0/1057143576 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fbe48076300 msgr2=0x7fbe480787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:01.057 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.057+0000 7fbe61075640 1 --2- 172.21.15.167:0/1057143576 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fbe48076300 0x7fbe480787c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fbe4c00b820 tx=0x7fbe4c008040 comp rx=0 tx=0).stop 2024-05-11T08:06:01.057 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.057+0000 7fbe61075640 1 -- 172.21.15.167:0/1057143576 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe5c0d0640 msgr2=0x7fbe5c146ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:01.057 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.057+0000 7fbe61075640 1 --2- 172.21.15.167:0/1057143576 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe5c0d0640 0x7fbe5c146ad0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fbe5000c490 tx=0x7fbe5000c950 comp rx=0 tx=0).stop 2024-05-11T08:06:01.058 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.057+0000 7fbe61075640 1 -- 172.21.15.167:0/1057143576 shutdown_connections 2024-05-11T08:06:01.058 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.057+0000 7fbe61075640 1 --2- 172.21.15.167:0/1057143576 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fbe48076300 0x7fbe480787c0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:01.058 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.057+0000 7fbe61075640 1 --2- 172.21.15.167:0/1057143576 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe5c1598d0 0x7fbe5c0b7e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:01.058 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.057+0000 7fbe61075640 1 --2- 172.21.15.167:0/1057143576 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe5c0d0640 0x7fbe5c146ad0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:01.058 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.057+0000 7fbe61075640 1 --2- 172.21.15.167:0/1057143576 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbe5c0cb6c0 0x7fbe5c155ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:01.058 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.057+0000 7fbe61075640 1 -- 172.21.15.167:0/1057143576 >> 172.21.15.167:0/1057143576 conn(0x7fbe5c0d1ba0 msgr2=0x7fbe5c160b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:01.058 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.057+0000 7fbe61075640 1 -- 172.21.15.167:0/1057143576 shutdown_connections 2024-05-11T08:06:01.058 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.057+0000 7fbe61075640 1 -- 172.21.15.167:0/1057143576 wait complete. 2024-05-11T08:06:01.068 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-11T08:06:01.168 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.168+0000 7f5f2f389640 1 Processor -- start 2024-05-11T08:06:01.168 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.168+0000 7f5f2f389640 1 -- start start 2024-05-11T08:06:01.169 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.169+0000 7f5f2f389640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5f280b5e90 0x7f5f2815f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:01.169 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.169+0000 7f5f2f389640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f5f2815fc80 0x7f5f280cd610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:01.169 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.169+0000 7f5f2f389640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f280d2410 con 0x7f5f2815f360 2024-05-11T08:06:01.169 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.169+0000 7f5f2f389640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f5f280d26f0 con 0x7f5f2815fc80 2024-05-11T08:06:01.170 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.169+0000 7f5f2f389640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f5f280d29d0 con 0x7f5f280b5e90 2024-05-11T08:06:01.170 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.169+0000 7f5f2d926640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f5f2815fc80 0x7f5f280cd610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:01.170 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.169+0000 7f5f2d125640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f5f2815f360 0x7f5f280cfdc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:40642/0 (socket says 172.21.15.167:40642) 2024-05-11T08:06:01.170 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.169+0000 7f5f2d125640 1 -- 172.21.15.167:0/2530888668 learned_addr learned my addr 172.21.15.167:0/2530888668 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T08:06:01.170 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.169+0000 7f5f2d926640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f5f2815fc80 0x7f5f280cd610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:59004/0 (socket says 172.21.15.167:59004) 2024-05-11T08:06:01.170 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.169+0000 7f5f2c924640 1 --2- 172.21.15.167:0/2530888668 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5f280b5e90 0x7f5f2815f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:01.170 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.169+0000 7f5f2d926640 1 -- 172.21.15.167:0/2530888668 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5f280b5e90 msgr2=0x7f5f2815f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:01.170 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.169+0000 7f5f2d926640 1 --2- 172.21.15.167:0/2530888668 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5f280b5e90 0x7f5f2815f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:01.170 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.169+0000 7f5f2d926640 1 -- 172.21.15.167:0/2530888668 >> v1:172.21.15.167:6789/0 conn(0x7f5f2815f360 legacy=0x7f5f280cfdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:01.170 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.169+0000 7f5f1ffff640 1 -- 172.21.15.167:0/2530888668 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3738787829 0 0) 0x7f5f280d2410 con 0x7f5f2815f360 2024-05-11T08:06:01.170 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.169+0000 7f5f2d926640 1 -- 172.21.15.167:0/2530888668 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f280b7e20 con 0x7f5f2815fc80 2024-05-11T08:06:01.172 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.170+0000 7f5f2d926640 1 --2- 172.21.15.167:0/2530888668 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5f2815fc80 0x7f5f280cd610 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5f20009d90 tx=0x7f5f2002a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b52b0d496652ec9 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:01.172 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.170+0000 7f5f1ffff640 1 -- 172.21.15.167:0/2530888668 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f20031600 con 0x7f5f2815fc80 2024-05-11T08:06:01.172 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.170+0000 7f5f1ffff640 1 -- 172.21.15.167:0/2530888668 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f200317a0 con 0x7f5f2815fc80 2024-05-11T08:06:01.172 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.170+0000 7f5f1ffff640 1 -- 172.21.15.167:0/2530888668 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f20031b70 con 0x7f5f2815fc80 2024-05-11T08:06:01.172 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.170+0000 7f5f2f389640 1 -- 172.21.15.167:0/2530888668 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5f2815fc80 msgr2=0x7f5f280cd610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:01.172 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.170+0000 7f5f2f389640 1 --2- 172.21.15.167:0/2530888668 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5f2815fc80 0x7f5f280cd610 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5f20009d90 tx=0x7f5f2002a5d0 comp rx=0 tx=0).stop 2024-05-11T08:06:01.172 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.171+0000 7f5f2f389640 1 -- 172.21.15.167:0/2530888668 shutdown_connections 2024-05-11T08:06:01.172 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.171+0000 7f5f2f389640 1 --2- 172.21.15.167:0/2530888668 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5f2815fc80 0x7f5f280cd610 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:01.172 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.171+0000 7f5f2f389640 1 --2- 172.21.15.167:0/2530888668 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5f280b5e90 0x7f5f2815f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:01.172 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.171+0000 7f5f2f389640 1 -- 172.21.15.167:0/2530888668 >> 172.21.15.167:0/2530888668 conn(0x7f5f28072730 msgr2=0x7f5f280d2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:01.172 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.171+0000 7f5f2f389640 1 -- 172.21.15.167:0/2530888668 shutdown_connections 2024-05-11T08:06:01.172 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.171+0000 7f5f2f389640 1 -- 172.21.15.167:0/2530888668 wait complete. 2024-05-11T08:06:01.173 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.172+0000 7f5f2f389640 1 Processor -- start 2024-05-11T08:06:01.173 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.172+0000 7f5f2f389640 1 -- start start 2024-05-11T08:06:01.175 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.172+0000 7f5f2f389640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5f280b5e90 0x7f5f280b2f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:01.175 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.173+0000 7f5f2f389640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5f2815f360 0x7f5f28150300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:01.175 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.173+0000 7f5f2f389640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5f2815fc80 0x7f5f280d13d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:01.175 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.173+0000 7f5f2f389640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5f280cdcd0 con 0x7f5f2815fc80 2024-05-11T08:06:01.175 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.173+0000 7f5f2f389640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5f280cdb50 con 0x7f5f2815f360 2024-05-11T08:06:01.175 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.173+0000 7f5f2f389640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f5f280cde50 con 0x7f5f280b5e90 2024-05-11T08:06:01.175 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.173+0000 7f5f2d926640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5f2815fc80 0x7f5f280d13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:01.175 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.173+0000 7f5f2d926640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5f2815fc80 0x7f5f280d13d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:59012/0 (socket says 172.21.15.167:59012) 2024-05-11T08:06:01.175 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.173+0000 7f5f2d926640 1 -- 172.21.15.167:0/2388517082 learned_addr learned my addr 172.21.15.167:0/2388517082 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:01.175 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.173+0000 7f5f2d125640 1 --2- 172.21.15.167:0/2388517082 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5f280b5e90 0x7f5f280b2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:01.175 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.173+0000 7f5f2c924640 1 --2- 172.21.15.167:0/2388517082 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5f2815f360 0x7f5f28150300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:01.176 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.173+0000 7f5f2d926640 1 -- 172.21.15.167:0/2388517082 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5f280b5e90 msgr2=0x7f5f280b2f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:01.176 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.173+0000 7f5f2d926640 1 --2- 172.21.15.167:0/2388517082 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5f280b5e90 0x7f5f280b2f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:01.176 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.173+0000 7f5f2d926640 1 -- 172.21.15.167:0/2388517082 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5f2815f360 msgr2=0x7f5f28150300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:01.176 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.173+0000 7f5f2d926640 1 --2- 172.21.15.167:0/2388517082 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5f2815f360 0x7f5f28150300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:01.176 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.173+0000 7f5f2d926640 1 -- 172.21.15.167:0/2388517082 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f28070cb0 con 0x7f5f2815fc80 2024-05-11T08:06:01.176 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.173+0000 7f5f2d926640 1 --2- 172.21.15.167:0/2388517082 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5f2815fc80 0x7f5f280d13d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5f2002abc0 tx=0x7f5f20004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:01.176 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.173+0000 7f5f1dffb640 1 -- 172.21.15.167:0/2388517082 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f20040070 con 0x7f5f2815fc80 2024-05-11T08:06:01.176 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.173+0000 7f5f1dffb640 1 -- 172.21.15.167:0/2388517082 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f20030b60 con 0x7f5f2815fc80 2024-05-11T08:06:01.176 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.173+0000 7f5f2f389640 1 -- 172.21.15.167:0/2388517082 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f28070f90 con 0x7f5f2815fc80 2024-05-11T08:06:01.176 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.173+0000 7f5f1dffb640 1 -- 172.21.15.167:0/2388517082 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f20009450 con 0x7f5f2815fc80 2024-05-11T08:06:01.176 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.174+0000 7f5f2f389640 1 -- 172.21.15.167:0/2388517082 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f280af2d0 con 0x7f5f2815fc80 2024-05-11T08:06:01.176 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.174+0000 7f5f2f389640 1 -- 172.21.15.167:0/2388517082 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f28069590 con 0x7f5f2815fc80 2024-05-11T08:06:01.176 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.175+0000 7f5f1dffb640 1 -- 172.21.15.167:0/2388517082 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f5f20039040 con 0x7f5f2815fc80 2024-05-11T08:06:01.177 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.175+0000 7f5f1dffb640 1 --2- 172.21.15.167:0/2388517082 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f5f0c0762d0 0x7f5f0c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:01.177 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.176+0000 7f5f2d125640 1 --2- 172.21.15.167:0/2388517082 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f5f0c0762d0 0x7f5f0c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:01.177 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.176+0000 7f5f1dffb640 1 -- 172.21.15.167:0/2388517082 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f5f200b8e10 con 0x7f5f2815fc80 2024-05-11T08:06:01.177 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.176+0000 7f5f2d125640 1 --2- 172.21.15.167:0/2388517082 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f5f0c0762d0 0x7f5f0c078790 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5f24000f60 tx=0x7f5f24007cf0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:01.179 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.180+0000 7f5f1dffb640 1 -- 172.21.15.167:0/2388517082 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f5f20081de0 con 0x7f5f2815fc80 2024-05-11T08:06:01.361 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.361+0000 7f5f2f389640 1 -- 172.21.15.167:0/2388517082 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5f28161880 con 0x7f5f2815fc80 2024-05-11T08:06:01.362 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.362+0000 7f5f1dffb640 1 -- 172.21.15.167:0/2388517082 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18562 (secure 0 0 0) 0x7f5f200860e0 con 0x7f5f2815fc80 2024-05-11T08:06:01.362 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T08:06:01.362 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":27,"fsid":"2034778d-8ff1-4823-aa44-ded32b518241","created":"2024-05-11T08:05:31.612421+0000","modified":"2024-05-11T08:05:59.756806+0000","last_up_change":"2024-05-11T08:05:42.557382+0000","last_in_change":"2024-05-11T08:05:33.757311+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-11T08:05:43.931263+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-11T08:05:52.573726+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-11T08:05:53.670746+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-11T08:05:57.380272+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"587eaced-5ded-406d-884b-30c1caa8686b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":3863307356},{"type":"v1","addr":"172.21.15.167:6806","nonce":3863307356}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":3863307356},{"type":"v1","addr":"172.21.15.167:6810","nonce":3863307356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":3863307356},{"type":"v1","addr":"172.21.15.167:6815","nonce":3863307356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":3863307356},{"type":"v1","addr":"172.21.15.167:6813","nonce":3863307356}]},"public_addr":"172.21.15.167:6806/3863307356","cluster_addr":"172.21.15.167:6810/3863307356","heartbeat_back_addr":"172.21.15.167:6815/3863307356","heartbeat_front_addr":"172.21.15.167:6813/3863307356","state":["exists","up"]},{"osd":1,"uuid":"37c59104-2dd0-4cf9-85ea-56636f6b02a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":1585024048},{"type":"v1","addr":"172.21.15.167:6817","nonce":1585024048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1585024048},{"type":"v1","addr":"172.21.15.167:6819","nonce":1585024048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1585024048},{"type":"v1","addr":"172.21.15.167:6823","nonce":1585024048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":1585024048},{"type":"v1","addr":"172.21.15.167:6821","nonce":1585024048}]},"public_addr":"172.21.15.167:6817/1585024048","cluster_addr":"172.21.15.167:6819/1585024048","heartbeat_back_addr":"172.21.15.167:6823/1585024048","heartbeat_front_addr":"172.21.15.167:6821/1585024048","state":["exists","up"]},{"osd":2,"uuid":"2bb4f7f5-c698-451b-b4bb-78e716921291","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":3242166900},{"type":"v1","addr":"172.21.15.167:6801","nonce":3242166900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":3242166900},{"type":"v1","addr":"172.21.15.167:6803","nonce":3242166900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6809","nonce":3242166900},{"type":"v1","addr":"172.21.15.167:6811","nonce":3242166900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6805","nonce":3242166900},{"type":"v1","addr":"172.21.15.167:6807","nonce":3242166900}]},"public_addr":"172.21.15.167:6801/3242166900","cluster_addr":"172.21.15.167:6803/3242166900","heartbeat_back_addr":"172.21.15.167:6811/3242166900","heartbeat_front_addr":"172.21.15.167:6807/3242166900","state":["exists","up"]},{"osd":3,"uuid":"9dd26759-a338-49d2-8103-45d068b9c000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":453689293},{"type":"v1","addr":"172.21.15.167:6825","nonce":453689293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":453689293},{"type":"v1","addr":"172.21.15.167:6827","nonce":453689293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":453689293},{"type":"v1","addr":"172.21.15.167:6831","nonce":453689293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":453689293},{"type":"v1","addr":"172.21.15.167:6829","nonce":453689293}]},"public_addr":"172.21.15.167:6825/453689293","cluster_addr":"172.21.15.167:6827/453689293","heartbeat_back_addr":"172.21.15.167:6831/453689293","heartbeat_front_addr":"172.21.15.167:6829/453689293","state":["exists","up"]},{"osd":4,"uuid":"cb27250d-2a80-4c7b-8cef-3a7af05c8a16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1311831500},{"type":"v1","addr":"172.21.15.204:6801","nonce":1311831500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1311831500},{"type":"v1","addr":"172.21.15.204:6803","nonce":1311831500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1311831500},{"type":"v1","addr":"172.21.15.204:6807","nonce":1311831500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1311831500},{"type":"v1","addr":"172.21.15.204:6805","nonce":1311831500}]},"public_addr":"172.21.15.204:6801/1311831500","cluster_addr":"172.21.15.204:6803/1311831500","heartbeat_back_addr":"172.21.15.204:6807/1311831500","heartbeat_front_addr":"172.21.15.204:6805/1311831500","state":["exists","up"]},{"osd":5,"uuid":"92998541-ffec-430b-ab76-8c3cb7c2752d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2921847521},{"type":"v1","addr":"172.21.15.204:6825","nonce":2921847521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2921847521},{"type":"v1","addr":"172.21.15.204:6827","nonce":2921847521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2921847521},{"type":"v1","addr":"172.21.15.204:6831","nonce":2921847521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2921847521},{"type":"v1","addr":"172.21.15.204:6829","nonce":2921847521}]},"public_addr":"172.21.15.204:6825/2921847521","cluster_addr":"172.21.15.204:6827/2921847521","heartbeat_back_addr":"172.21.15.204:6831/2921847521","heartbeat_front_addr":"172.21.15.204:6829/2921847521","state":["exists","up"]},{"osd":6,"uuid":"9543479c-2ec5-42a8-8933-707bfed2e048","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":2889851077},{"type":"v1","addr":"172.21.15.204:6817","nonce":2889851077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":2889851077},{"type":"v1","addr":"172.21.15.204:6819","nonce":2889851077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":2889851077},{"type":"v1","addr":"172.21.15.204:6823","nonce":2889851077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":2889851077},{"type":"v1","addr":"172.21.15.204:6821","nonce":2889851077}]},"public_addr":"172.21.15.204:6817/2889851077","cluster_addr":"172.21.15.204:6819/2889851077","heartbeat_back_addr":"172.21.15.204:6823/2889851077","heartbeat_front_addr":"172.21.15.204:6821/2889851077","state":["exists","up"]},{"osd":7,"uuid":"c45a5647-8358-409e-af91-c05dda65f0b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":1949534184},{"type":"v1","addr":"172.21.15.204:6809","nonce":1949534184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":1949534184},{"type":"v1","addr":"172.21.15.204:6811","nonce":1949534184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1949534184},{"type":"v1","addr":"172.21.15.204:6815","nonce":1949534184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1949534184},{"type":"v1","addr":"172.21.15.204:6813","nonce":1949534184}]},"public_addr":"172.21.15.204:6809/1949534184","cluster_addr":"172.21.15.204:6811/1949534184","heartbeat_back_addr":"172.21.15.204:6815/1949534184","heartbeat_front_addr":"172.21.15.204:6813/1949534184","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.952253+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.940689+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.860682+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.929350+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.901627+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.957451+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.987249+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.951197+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-11T08:06:01.364 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.364+0000 7f5f2f389640 1 -- 172.21.15.167:0/2388517082 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f5f0c0762d0 msgr2=0x7f5f0c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:01.364 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.364+0000 7f5f2f389640 1 --2- 172.21.15.167:0/2388517082 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f5f0c0762d0 0x7f5f0c078790 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5f24000f60 tx=0x7f5f24007cf0 comp rx=0 tx=0).stop 2024-05-11T08:06:01.365 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.364+0000 7f5f2f389640 1 -- 172.21.15.167:0/2388517082 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5f2815fc80 msgr2=0x7f5f280d13d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:01.365 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.364+0000 7f5f2f389640 1 --2- 172.21.15.167:0/2388517082 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5f2815fc80 0x7f5f280d13d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5f2002abc0 tx=0x7f5f20004a40 comp rx=0 tx=0).stop 2024-05-11T08:06:01.365 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.364+0000 7f5f2f389640 1 -- 172.21.15.167:0/2388517082 shutdown_connections 2024-05-11T08:06:01.365 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.364+0000 7f5f2f389640 1 --2- 172.21.15.167:0/2388517082 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f5f0c0762d0 0x7f5f0c078790 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:01.365 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.364+0000 7f5f2f389640 1 --2- 172.21.15.167:0/2388517082 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5f2815fc80 0x7f5f280d13d0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:01.365 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.364+0000 7f5f2f389640 1 --2- 172.21.15.167:0/2388517082 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5f2815f360 0x7f5f28150300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:01.365 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.364+0000 7f5f2f389640 1 --2- 172.21.15.167:0/2388517082 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5f280b5e90 0x7f5f280b2f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:01.365 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.364+0000 7f5f2f389640 1 -- 172.21.15.167:0/2388517082 >> 172.21.15.167:0/2388517082 conn(0x7f5f28072730 msgr2=0x7f5f28161170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:01.365 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.364+0000 7f5f2f389640 1 -- 172.21.15.167:0/2388517082 shutdown_connections 2024-05-11T08:06:01.365 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.364+0000 7f5f2f389640 1 -- 172.21.15.167:0/2388517082 wait complete. 2024-05-11T08:06:01.389 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-11T08:06:01.489 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.489+0000 7f82cd8e3640 1 Processor -- start 2024-05-11T08:06:01.489 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.489+0000 7f82cd8e3640 1 -- start start 2024-05-11T08:06:01.491 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.490+0000 7f82cd8e3640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f82c80d2cf0 0x7f82c80cafb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:01.491 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.490+0000 7f82cd8e3640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f82c80cb4f0 0x7f82c806dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:01.491 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.490+0000 7f82cd8e3640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82c80d1190 con 0x7f82c81598d0 2024-05-11T08:06:01.491 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.490+0000 7f82cd8e3640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f82c80d3150 con 0x7f82c80cb4f0 2024-05-11T08:06:01.491 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.490+0000 7f82cd8e3640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f82c80cb940 con 0x7f82c80d2cf0 2024-05-11T08:06:01.491 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.490+0000 7f82c77fe640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f82c80cb4f0 0x7f82c806dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:01.491 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.490+0000 7f82c77fe640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f82c80cb4f0 0x7f82c806dfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:59016/0 (socket says 172.21.15.167:59016) 2024-05-11T08:06:01.491 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.490+0000 7f82c77fe640 1 -- 172.21.15.167:0/3599223546 learned_addr learned my addr 172.21.15.167:0/3599223546 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:01.491 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.490+0000 7f82c67fc640 1 --2- 172.21.15.167:0/3599223546 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f82c80d2cf0 0x7f82c80cafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:01.492 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.490+0000 7f82c77fe640 1 -- 172.21.15.167:0/3599223546 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f82c80d2cf0 msgr2=0x7f82c80cafb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:01.492 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.490+0000 7f82c77fe640 1 --2- 172.21.15.167:0/3599223546 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f82c80d2cf0 0x7f82c80cafb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:01.492 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.490+0000 7f82c77fe640 1 -- 172.21.15.167:0/3599223546 >> v1:172.21.15.204:6789/0 conn(0x7f82c81598d0 legacy=0x7f82c80d0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:01.492 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.490+0000 7f82c77fe640 1 -- 172.21.15.167:0/3599223546 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82c806d500 con 0x7f82c80cb4f0 2024-05-11T08:06:01.492 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.491+0000 7f82c77fe640 1 --2- 172.21.15.167:0/3599223546 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f82c80cb4f0 0x7f82c806dfe0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f82c0009e50 tx=0x7f82c002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=176d60f2be3ab8ed server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:01.492 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.491+0000 7f82c5ffb640 1 -- 172.21.15.167:0/3599223546 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c0036070 con 0x7f82c80cb4f0 2024-05-11T08:06:01.492 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.491+0000 7f82c5ffb640 1 -- 172.21.15.167:0/3599223546 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82c0031600 con 0x7f82c80cb4f0 2024-05-11T08:06:01.492 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.491+0000 7f82c5ffb640 1 -- 172.21.15.167:0/3599223546 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c00319d0 con 0x7f82c80cb4f0 2024-05-11T08:06:01.493 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.492+0000 7f82cd8e3640 1 -- 172.21.15.167:0/3599223546 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f82c80cb4f0 msgr2=0x7f82c806dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:01.493 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.492+0000 7f82cd8e3640 1 --2- 172.21.15.167:0/3599223546 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f82c80cb4f0 0x7f82c806dfe0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f82c0009e50 tx=0x7f82c002a680 comp rx=0 tx=0).stop 2024-05-11T08:06:01.493 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.492+0000 7f82cd8e3640 1 -- 172.21.15.167:0/3599223546 shutdown_connections 2024-05-11T08:06:01.493 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.492+0000 7f82cd8e3640 1 --2- 172.21.15.167:0/3599223546 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f82c80cb4f0 0x7f82c806dfe0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:01.493 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.492+0000 7f82cd8e3640 1 --2- 172.21.15.167:0/3599223546 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f82c80d2cf0 0x7f82c80cafb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:01.493 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.492+0000 7f82cd8e3640 1 -- 172.21.15.167:0/3599223546 >> 172.21.15.167:0/3599223546 conn(0x7f82c80d1ba0 msgr2=0x7f82c80d1fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:01.493 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.492+0000 7f82cd8e3640 1 -- 172.21.15.167:0/3599223546 shutdown_connections 2024-05-11T08:06:01.494 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.492+0000 7f82cd8e3640 1 -- 172.21.15.167:0/3599223546 wait complete. 2024-05-11T08:06:01.494 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.493+0000 7f82cd8e3640 1 Processor -- start 2024-05-11T08:06:01.494 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.493+0000 7f82cd8e3640 1 -- start start 2024-05-11T08:06:01.496 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.494+0000 7f82cd8e3640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f82c80cb4f0 0x7f82c80b9210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:01.496 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.494+0000 7f82cd8e3640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82c80d2cf0 0x7f82c81f9120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:01.496 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.494+0000 7f82cd8e3640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f82c81598d0 0x7f82c8157fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:01.496 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.494+0000 7f82cd8e3640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f82c81607c0 con 0x7f82c81598d0 2024-05-11T08:06:01.496 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.494+0000 7f82cd8e3640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f82c806dac0 con 0x7f82c80d2cf0 2024-05-11T08:06:01.496 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.494+0000 7f82cd8e3640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f82c8159cb0 con 0x7f82c80cb4f0 2024-05-11T08:06:01.496 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.494+0000 7f82c77fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f82c81598d0 0x7f82c8157fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:01.496 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.494+0000 7f82c77fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f82c81598d0 0x7f82c8157fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:59018/0 (socket says 172.21.15.167:59018) 2024-05-11T08:06:01.496 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.494+0000 7f82c77fe640 1 -- 172.21.15.167:0/1847832438 learned_addr learned my addr 172.21.15.167:0/1847832438 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:01.496 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.494+0000 7f82c67fc640 1 --2- 172.21.15.167:0/1847832438 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82c80d2cf0 0x7f82c81f9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:01.496 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.494+0000 7f82c6ffd640 1 --2- 172.21.15.167:0/1847832438 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f82c80cb4f0 0x7f82c80b9210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:01.496 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.494+0000 7f82c77fe640 1 -- 172.21.15.167:0/1847832438 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f82c80cb4f0 msgr2=0x7f82c80b9210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:01.497 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.494+0000 7f82c77fe640 1 --2- 172.21.15.167:0/1847832438 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f82c80cb4f0 0x7f82c80b9210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:01.497 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.494+0000 7f82c77fe640 1 -- 172.21.15.167:0/1847832438 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82c80d2cf0 msgr2=0x7f82c81f9120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:01.497 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.494+0000 7f82c77fe640 1 --2- 172.21.15.167:0/1847832438 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82c80d2cf0 0x7f82c81f9120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:01.497 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.494+0000 7f82c77fe640 1 -- 172.21.15.167:0/1847832438 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82c80b9750 con 0x7f82c81598d0 2024-05-11T08:06:01.497 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.494+0000 7f82c6ffd640 1 --2- 172.21.15.167:0/1847832438 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f82c80cb4f0 0x7f82c80b9210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T08:06:01.497 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.494+0000 7f82c77fe640 1 --2- 172.21.15.167:0/1847832438 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f82c81598d0 0x7f82c8157fe0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f82c0009e50 tx=0x7f82c0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:01.497 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.494+0000 7f82affff640 1 -- 172.21.15.167:0/1847832438 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c0036050 con 0x7f82c81598d0 2024-05-11T08:06:01.497 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.494+0000 7f82affff640 1 -- 172.21.15.167:0/1847832438 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82c0042070 con 0x7f82c81598d0 2024-05-11T08:06:01.497 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.495+0000 7f82cd8e3640 1 -- 172.21.15.167:0/1847832438 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82c81f9660 con 0x7f82c81598d0 2024-05-11T08:06:01.497 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.495+0000 7f82affff640 1 -- 172.21.15.167:0/1847832438 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c0009510 con 0x7f82c81598d0 2024-05-11T08:06:01.497 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.495+0000 7f82cd8e3640 1 -- 172.21.15.167:0/1847832438 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82c80b8450 con 0x7f82c81598d0 2024-05-11T08:06:01.497 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.495+0000 7f82cd8e3640 1 -- 172.21.15.167:0/1847832438 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82c8069590 con 0x7f82c81598d0 2024-05-11T08:06:01.497 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.496+0000 7f82affff640 1 -- 172.21.15.167:0/1847832438 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f82c0007bd0 con 0x7f82c81598d0 2024-05-11T08:06:01.498 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.496+0000 7f82affff640 1 --2- 172.21.15.167:0/1847832438 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f82b40762d0 0x7f82b4078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:01.498 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.497+0000 7f82affff640 1 -- 172.21.15.167:0/1847832438 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f82c00b87c0 con 0x7f82c81598d0 2024-05-11T08:06:01.498 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.497+0000 7f82c6ffd640 1 --2- 172.21.15.167:0/1847832438 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f82b40762d0 0x7f82b4078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:01.498 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.497+0000 7f82c6ffd640 1 --2- 172.21.15.167:0/1847832438 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f82b40762d0 0x7f82b4078790 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f82b8006fd0 tx=0x7f82b8008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:01.501 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.501+0000 7f82affff640 1 -- 172.21.15.167:0/1847832438 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f82c0081de0 con 0x7f82c81598d0 2024-05-11T08:06:01.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.721+0000 7f82cd8e3640 1 -- 172.21.15.167:0/1847832438 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f82c80cbfb0 con 0x7f82c81598d0 2024-05-11T08:06:01.722 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.722+0000 7f82affff640 1 -- 172.21.15.167:0/1847832438 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5183 (secure 0 0 0) 0x7f82c00816e0 con 0x7f82c81598d0 2024-05-11T08:06:01.722 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T08:06:01.722 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6834/506789040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":506789040},{"type":"v1","addr":"172.21.15.204:6834","nonce":506789040}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4287,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1448126000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":1448126000},{"type":"v1","addr":"172.21.15.167:6837","nonce":1448126000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4290,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/766834562","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":766834562},{"type":"v1","addr":"172.21.15.167:6836","nonce":766834562}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-11T08:05:54.711448+0000","modified":"2024-05-11T08:05:59.756961+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4305},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4305":{"gid":4305,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.204:6835/1247774569","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6833","nonce":1247774569},{"type":"v1","addr":"172.21.15.204:6835","nonce":1247774569}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":4305,"qdb_cluster":[4305]},"id":1}]} 2024-05-11T08:06:01.722 INFO:teuthology.orchestra.run.smithi167.stderr:dumped fsmap epoch 7 2024-05-11T08:06:01.724 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.724+0000 7f82cd8e3640 1 -- 172.21.15.167:0/1847832438 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f82b40762d0 msgr2=0x7f82b4078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:01.724 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.724+0000 7f82cd8e3640 1 --2- 172.21.15.167:0/1847832438 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f82b40762d0 0x7f82b4078790 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f82b8006fd0 tx=0x7f82b8008040 comp rx=0 tx=0).stop 2024-05-11T08:06:01.724 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.724+0000 7f82cd8e3640 1 -- 172.21.15.167:0/1847832438 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f82c81598d0 msgr2=0x7f82c8157fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:01.724 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.724+0000 7f82cd8e3640 1 --2- 172.21.15.167:0/1847832438 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f82c81598d0 0x7f82c8157fe0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f82c0009e50 tx=0x7f82c0004a40 comp rx=0 tx=0).stop 2024-05-11T08:06:01.725 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.724+0000 7f82cd8e3640 1 -- 172.21.15.167:0/1847832438 shutdown_connections 2024-05-11T08:06:01.725 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.724+0000 7f82cd8e3640 1 --2- 172.21.15.167:0/1847832438 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f82b40762d0 0x7f82b4078790 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:01.725 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.724+0000 7f82cd8e3640 1 --2- 172.21.15.167:0/1847832438 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f82c81598d0 0x7f82c8157fe0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:01.725 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.724+0000 7f82cd8e3640 1 --2- 172.21.15.167:0/1847832438 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82c80d2cf0 0x7f82c81f9120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:01.725 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.724+0000 7f82cd8e3640 1 --2- 172.21.15.167:0/1847832438 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f82c80cb4f0 0x7f82c80b9210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:01.725 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.724+0000 7f82cd8e3640 1 -- 172.21.15.167:0/1847832438 >> 172.21.15.167:0/1847832438 conn(0x7f82c80d1ba0 msgr2=0x7f82c80cf4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:01.725 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.724+0000 7f82cd8e3640 1 -- 172.21.15.167:0/1847832438 shutdown_connections 2024-05-11T08:06:01.725 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.724+0000 7f82cd8e3640 1 -- 172.21.15.167:0/1847832438 wait complete. 2024-05-11T08:06:01.735 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-11T08:06:01.835 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.835+0000 7ff526912640 1 Processor -- start 2024-05-11T08:06:01.836 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.835+0000 7ff526912640 1 -- start start 2024-05-11T08:06:01.837 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.836+0000 7ff526912640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff5201598d0 0x7ff5200d2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:01.837 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.836+0000 7ff526912640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7ff5200d0640 0x7ff5200d0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:01.837 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.836+0000 7ff526912640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5200d1190 con 0x7ff5200cafb0 2024-05-11T08:06:01.837 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.836+0000 7ff526912640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7ff52006d500 con 0x7ff5200d0640 2024-05-11T08:06:01.837 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.836+0000 7ff526912640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7ff52006d7e0 con 0x7ff5201598d0 2024-05-11T08:06:01.837 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.836+0000 7ff51ffff640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff5201598d0 0x7ff5200d2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:01.837 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.836+0000 7ff517fff640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7ff5200d0640 0x7ff5200d0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:01.837 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.837+0000 7ff51ffff640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff5201598d0 0x7ff5200d2cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.167:37618/0 (socket says 172.21.15.167:37618) 2024-05-11T08:06:01.837 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.837+0000 7ff51ffff640 1 -- 172.21.15.167:0/957063190 learned_addr learned my addr 172.21.15.167:0/957063190 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:01.837 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.837+0000 7ff517fff640 1 --2- 172.21.15.167:0/957063190 >> v2:172.21.15.204:3300/0 conn(0x7ff5200d0640 0x7ff5200d0aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.167:38546/0 (socket says 172.21.15.167:38546) 2024-05-11T08:06:01.837 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.837+0000 7ff51ffff640 1 -- 172.21.15.167:0/957063190 >> v2:172.21.15.204:3300/0 conn(0x7ff5200d0640 msgr2=0x7ff5200d0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:01.838 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.837+0000 7ff51ffff640 1 --2- 172.21.15.167:0/957063190 >> v2:172.21.15.204:3300/0 conn(0x7ff5200d0640 0x7ff5200d0aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:01.838 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.837+0000 7ff51ffff640 1 -- 172.21.15.167:0/957063190 >> v1:172.21.15.204:6789/0 conn(0x7ff5200cafb0 legacy=0x7ff52006dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:01.838 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.837+0000 7ff51ffff640 1 -- 172.21.15.167:0/957063190 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff52006dac0 con 0x7ff5201598d0 2024-05-11T08:06:01.839 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.838+0000 7ff51ffff640 1 --2- 172.21.15.167:0/957063190 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff5201598d0 0x7ff5200d2cf0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7ff510009870 tx=0x7ff51002a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec4bed9aa88c82ad server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:01.839 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.838+0000 7ff51f7fe640 1 -- 172.21.15.167:0/957063190 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff510037070 con 0x7ff5201598d0 2024-05-11T08:06:01.839 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.838+0000 7ff51f7fe640 1 -- 172.21.15.167:0/957063190 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff510032440 con 0x7ff5201598d0 2024-05-11T08:06:01.839 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.838+0000 7ff51f7fe640 1 -- 172.21.15.167:0/957063190 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff510032810 con 0x7ff5201598d0 2024-05-11T08:06:01.839 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.838+0000 7ff526912640 1 -- 172.21.15.167:0/957063190 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff5201598d0 msgr2=0x7ff5200d2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:01.839 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.838+0000 7ff526912640 1 --2- 172.21.15.167:0/957063190 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff5201598d0 0x7ff5200d2cf0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7ff510009870 tx=0x7ff51002a270 comp rx=0 tx=0).stop 2024-05-11T08:06:01.839 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.838+0000 7ff526912640 1 -- 172.21.15.167:0/957063190 shutdown_connections 2024-05-11T08:06:01.839 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.838+0000 7ff526912640 1 --2- 172.21.15.167:0/957063190 >> v2:172.21.15.204:3300/0 conn(0x7ff5200d0640 0x7ff5200d0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:01.839 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.838+0000 7ff526912640 1 --2- 172.21.15.167:0/957063190 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff5201598d0 0x7ff5200d2cf0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:01.839 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.838+0000 7ff526912640 1 -- 172.21.15.167:0/957063190 >> 172.21.15.167:0/957063190 conn(0x7ff5200d1ba0 msgr2=0x7ff5200d1fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:01.840 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.838+0000 7ff526912640 1 -- 172.21.15.167:0/957063190 shutdown_connections 2024-05-11T08:06:01.840 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.838+0000 7ff526912640 1 -- 172.21.15.167:0/957063190 wait complete. 2024-05-11T08:06:01.842 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.839+0000 7ff526912640 1 Processor -- start 2024-05-11T08:06:01.842 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.840+0000 7ff526912640 1 -- start start 2024-05-11T08:06:01.842 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.840+0000 7ff526912640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff5200cafb0 0x7ff5201e9d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:01.842 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.840+0000 7ff526912640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff5200d0640 0x7ff52014d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:01.842 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.840+0000 7ff526912640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff5201598d0 0x7ff5200ce180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:01.842 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.840+0000 7ff526912640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff52006d1b0 con 0x7ff5200d0640 2024-05-11T08:06:01.842 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.840+0000 7ff526912640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff520159cb0 con 0x7ff5200cafb0 2024-05-11T08:06:01.843 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.840+0000 7ff526912640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7ff5200cb400 con 0x7ff5201598d0 2024-05-11T08:06:01.843 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.840+0000 7ff517fff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff5200d0640 0x7ff52014d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:01.843 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.840+0000 7ff517fff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff5200d0640 0x7ff52014d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:59034/0 (socket says 172.21.15.167:59034) 2024-05-11T08:06:01.843 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.840+0000 7ff517fff640 1 -- 172.21.15.167:0/993172095 learned_addr learned my addr 172.21.15.167:0/993172095 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:01.843 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.841+0000 7ff51ffff640 1 --2- 172.21.15.167:0/993172095 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff5200cafb0 0x7ff5201e9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:01.843 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.841+0000 7ff524eaf640 1 --2- 172.21.15.167:0/993172095 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff5201598d0 0x7ff5200ce180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:01.843 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.841+0000 7ff517fff640 1 -- 172.21.15.167:0/993172095 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff5201598d0 msgr2=0x7ff5200ce180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:01.843 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.841+0000 7ff517fff640 1 --2- 172.21.15.167:0/993172095 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff5201598d0 0x7ff5200ce180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:01.843 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.841+0000 7ff517fff640 1 -- 172.21.15.167:0/993172095 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff5200cafb0 msgr2=0x7ff5201e9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:01.843 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.841+0000 7ff517fff640 1 --2- 172.21.15.167:0/993172095 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff5200cafb0 0x7ff5201e9d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:01.843 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.841+0000 7ff517fff640 1 -- 172.21.15.167:0/993172095 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5201ea400 con 0x7ff5200d0640 2024-05-11T08:06:01.843 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.841+0000 7ff51ffff640 1 --2- 172.21.15.167:0/993172095 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff5200cafb0 0x7ff5201e9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T08:06:01.843 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.841+0000 7ff524eaf640 1 --2- 172.21.15.167:0/993172095 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff5201598d0 0x7ff5200ce180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T08:06:01.843 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.841+0000 7ff517fff640 1 --2- 172.21.15.167:0/993172095 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff5200d0640 0x7ff52014d550 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff50800e9b0 tx=0x7ff50800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:01.844 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.841+0000 7ff51d7fa640 1 -- 172.21.15.167:0/993172095 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff508017070 con 0x7ff5200d0640 2024-05-11T08:06:01.844 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.841+0000 7ff51d7fa640 1 -- 172.21.15.167:0/993172095 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff50800c9a0 con 0x7ff5200d0640 2024-05-11T08:06:01.844 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.841+0000 7ff526912640 1 -- 172.21.15.167:0/993172095 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5201ea6e0 con 0x7ff5200d0640 2024-05-11T08:06:01.844 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.841+0000 7ff51d7fa640 1 -- 172.21.15.167:0/993172095 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff50800cd90 con 0x7ff5200d0640 2024-05-11T08:06:01.844 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.841+0000 7ff526912640 1 -- 172.21.15.167:0/993172095 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5201eaca0 con 0x7ff5200d0640 2024-05-11T08:06:01.844 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.842+0000 7ff526912640 1 -- 172.21.15.167:0/993172095 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff520069590 con 0x7ff5200d0640 2024-05-11T08:06:01.845 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.843+0000 7ff51d7fa640 1 -- 172.21.15.167:0/993172095 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7ff50801a070 con 0x7ff5200d0640 2024-05-11T08:06:01.845 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.843+0000 7ff51d7fa640 1 --2- 172.21.15.167:0/993172095 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7ff500076300 0x7ff5000787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:01.845 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.843+0000 7ff51ffff640 1 --2- 172.21.15.167:0/993172095 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7ff500076300 0x7ff5000787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:01.845 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.843+0000 7ff51d7fa640 1 -- 172.21.15.167:0/993172095 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7ff508097cf0 con 0x7ff5200d0640 2024-05-11T08:06:01.845 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.844+0000 7ff51ffff640 1 --2- 172.21.15.167:0/993172095 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7ff500076300 0x7ff5000787c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff510034000 tx=0x7ff51002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:01.848 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:01.848+0000 7ff51d7fa640 1 -- 172.21.15.167:0/993172095 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7ff50809c070 con 0x7ff5200d0640 2024-05-11T08:06:02.032 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.032+0000 7ff526912640 1 -- 172.21.15.167:0/993172095 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff5200cf5e0 con 0x7ff5200d0640 2024-05-11T08:06:02.033 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.033+0000 7ff51d7fa640 1 -- 172.21.15.167:0/993172095 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18562 (secure 0 0 0) 0x7ff508061310 con 0x7ff5200d0640 2024-05-11T08:06:02.034 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T08:06:02.034 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":27,"fsid":"2034778d-8ff1-4823-aa44-ded32b518241","created":"2024-05-11T08:05:31.612421+0000","modified":"2024-05-11T08:05:59.756806+0000","last_up_change":"2024-05-11T08:05:42.557382+0000","last_in_change":"2024-05-11T08:05:33.757311+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-11T08:05:43.931263+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-11T08:05:52.573726+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-11T08:05:53.670746+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-11T08:05:57.380272+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"587eaced-5ded-406d-884b-30c1caa8686b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":3863307356},{"type":"v1","addr":"172.21.15.167:6806","nonce":3863307356}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":3863307356},{"type":"v1","addr":"172.21.15.167:6810","nonce":3863307356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":3863307356},{"type":"v1","addr":"172.21.15.167:6815","nonce":3863307356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":3863307356},{"type":"v1","addr":"172.21.15.167:6813","nonce":3863307356}]},"public_addr":"172.21.15.167:6806/3863307356","cluster_addr":"172.21.15.167:6810/3863307356","heartbeat_back_addr":"172.21.15.167:6815/3863307356","heartbeat_front_addr":"172.21.15.167:6813/3863307356","state":["exists","up"]},{"osd":1,"uuid":"37c59104-2dd0-4cf9-85ea-56636f6b02a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":1585024048},{"type":"v1","addr":"172.21.15.167:6817","nonce":1585024048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1585024048},{"type":"v1","addr":"172.21.15.167:6819","nonce":1585024048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1585024048},{"type":"v1","addr":"172.21.15.167:6823","nonce":1585024048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":1585024048},{"type":"v1","addr":"172.21.15.167:6821","nonce":1585024048}]},"public_addr":"172.21.15.167:6817/1585024048","cluster_addr":"172.21.15.167:6819/1585024048","heartbeat_back_addr":"172.21.15.167:6823/1585024048","heartbeat_front_addr":"172.21.15.167:6821/1585024048","state":["exists","up"]},{"osd":2,"uuid":"2bb4f7f5-c698-451b-b4bb-78e716921291","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":3242166900},{"type":"v1","addr":"172.21.15.167:6801","nonce":3242166900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":3242166900},{"type":"v1","addr":"172.21.15.167:6803","nonce":3242166900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6809","nonce":3242166900},{"type":"v1","addr":"172.21.15.167:6811","nonce":3242166900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6805","nonce":3242166900},{"type":"v1","addr":"172.21.15.167:6807","nonce":3242166900}]},"public_addr":"172.21.15.167:6801/3242166900","cluster_addr":"172.21.15.167:6803/3242166900","heartbeat_back_addr":"172.21.15.167:6811/3242166900","heartbeat_front_addr":"172.21.15.167:6807/3242166900","state":["exists","up"]},{"osd":3,"uuid":"9dd26759-a338-49d2-8103-45d068b9c000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":453689293},{"type":"v1","addr":"172.21.15.167:6825","nonce":453689293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":453689293},{"type":"v1","addr":"172.21.15.167:6827","nonce":453689293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":453689293},{"type":"v1","addr":"172.21.15.167:6831","nonce":453689293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":453689293},{"type":"v1","addr":"172.21.15.167:6829","nonce":453689293}]},"public_addr":"172.21.15.167:6825/453689293","cluster_addr":"172.21.15.167:6827/453689293","heartbeat_back_addr":"172.21.15.167:6831/453689293","heartbeat_front_addr":"172.21.15.167:6829/453689293","state":["exists","up"]},{"osd":4,"uuid":"cb27250d-2a80-4c7b-8cef-3a7af05c8a16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1311831500},{"type":"v1","addr":"172.21.15.204:6801","nonce":1311831500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1311831500},{"type":"v1","addr":"172.21.15.204:6803","nonce":1311831500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1311831500},{"type":"v1","addr":"172.21.15.204:6807","nonce":1311831500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1311831500},{"type":"v1","addr":"172.21.15.204:6805","nonce":1311831500}]},"public_addr":"172.21.15.204:6801/1311831500","cluster_addr":"172.21.15.204:6803/1311831500","heartbeat_back_addr":"172.21.15.204:6807/1311831500","heartbeat_front_addr":"172.21.15.204:6805/1311831500","state":["exists","up"]},{"osd":5,"uuid":"92998541-ffec-430b-ab76-8c3cb7c2752d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2921847521},{"type":"v1","addr":"172.21.15.204:6825","nonce":2921847521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2921847521},{"type":"v1","addr":"172.21.15.204:6827","nonce":2921847521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2921847521},{"type":"v1","addr":"172.21.15.204:6831","nonce":2921847521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2921847521},{"type":"v1","addr":"172.21.15.204:6829","nonce":2921847521}]},"public_addr":"172.21.15.204:6825/2921847521","cluster_addr":"172.21.15.204:6827/2921847521","heartbeat_back_addr":"172.21.15.204:6831/2921847521","heartbeat_front_addr":"172.21.15.204:6829/2921847521","state":["exists","up"]},{"osd":6,"uuid":"9543479c-2ec5-42a8-8933-707bfed2e048","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":2889851077},{"type":"v1","addr":"172.21.15.204:6817","nonce":2889851077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":2889851077},{"type":"v1","addr":"172.21.15.204:6819","nonce":2889851077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":2889851077},{"type":"v1","addr":"172.21.15.204:6823","nonce":2889851077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":2889851077},{"type":"v1","addr":"172.21.15.204:6821","nonce":2889851077}]},"public_addr":"172.21.15.204:6817/2889851077","cluster_addr":"172.21.15.204:6819/2889851077","heartbeat_back_addr":"172.21.15.204:6823/2889851077","heartbeat_front_addr":"172.21.15.204:6821/2889851077","state":["exists","up"]},{"osd":7,"uuid":"c45a5647-8358-409e-af91-c05dda65f0b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":1949534184},{"type":"v1","addr":"172.21.15.204:6809","nonce":1949534184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":1949534184},{"type":"v1","addr":"172.21.15.204:6811","nonce":1949534184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1949534184},{"type":"v1","addr":"172.21.15.204:6815","nonce":1949534184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1949534184},{"type":"v1","addr":"172.21.15.204:6813","nonce":1949534184}]},"public_addr":"172.21.15.204:6809/1949534184","cluster_addr":"172.21.15.204:6811/1949534184","heartbeat_back_addr":"172.21.15.204:6815/1949534184","heartbeat_front_addr":"172.21.15.204:6813/1949534184","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.952253+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.940689+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.860682+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.929350+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.901627+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.957451+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.987249+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.951197+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-11T08:06:02.036 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.035+0000 7ff526912640 1 -- 172.21.15.167:0/993172095 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7ff500076300 msgr2=0x7ff5000787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:02.036 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.035+0000 7ff526912640 1 --2- 172.21.15.167:0/993172095 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7ff500076300 0x7ff5000787c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff510034000 tx=0x7ff51002e000 comp rx=0 tx=0).stop 2024-05-11T08:06:02.036 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.035+0000 7ff526912640 1 -- 172.21.15.167:0/993172095 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff5200d0640 msgr2=0x7ff52014d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:02.036 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.035+0000 7ff526912640 1 --2- 172.21.15.167:0/993172095 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff5200d0640 0x7ff52014d550 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff50800e9b0 tx=0x7ff50800ee70 comp rx=0 tx=0).stop 2024-05-11T08:06:02.036 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.035+0000 7ff526912640 1 -- 172.21.15.167:0/993172095 shutdown_connections 2024-05-11T08:06:02.036 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.035+0000 7ff526912640 1 --2- 172.21.15.167:0/993172095 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7ff500076300 0x7ff5000787c0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:02.036 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.035+0000 7ff526912640 1 --2- 172.21.15.167:0/993172095 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff5201598d0 0x7ff5200ce180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:02.036 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.035+0000 7ff526912640 1 --2- 172.21.15.167:0/993172095 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff5200d0640 0x7ff52014d550 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:02.036 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.035+0000 7ff526912640 1 --2- 172.21.15.167:0/993172095 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff5200cafb0 0x7ff5201e9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:02.036 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.035+0000 7ff526912640 1 -- 172.21.15.167:0/993172095 >> 172.21.15.167:0/993172095 conn(0x7ff5200d1ba0 msgr2=0x7ff5200d1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:02.037 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.036+0000 7ff526912640 1 -- 172.21.15.167:0/993172095 shutdown_connections 2024-05-11T08:06:02.037 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.036+0000 7ff526912640 1 -- 172.21.15.167:0/993172095 wait complete. 2024-05-11T08:06:02.450 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-11T08:05:54.711448+0000', 'modified': '2024-05-11T08:05:59.756961+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4305}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4305': {'gid': 4305, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.204:6835/1247774569', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.204:6833', 'nonce': 1247774569}, {'type': 'v1', 'addr': '172.21.15.204:6835', 'nonce': 1247774569}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 4305, 'qdb_cluster': [4305]} 2024-05-11T08:06:02.450 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-11T08:06:02.450 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-05-11T08:06:02.553 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.553+0000 7f6c6fb89640 1 Processor -- start 2024-05-11T08:06:02.553 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.553+0000 7f6c6fb89640 1 -- start start 2024-05-11T08:06:02.555 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.554+0000 7f6c6fb89640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f6c6815f360 0x7f6c680cfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:02.555 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.554+0000 7f6c6fb89640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f6c680b5e90 0x7f6c6815f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:02.555 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.554+0000 7f6c6fb89640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6c680d2410 con 0x7f6c6815fc80 2024-05-11T08:06:02.555 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.554+0000 7f6c6fb89640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f6c680d26f0 con 0x7f6c680b5e90 2024-05-11T08:06:02.555 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.554+0000 7f6c6fb89640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f6c680d29d0 con 0x7f6c6815f360 2024-05-11T08:06:02.556 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.554+0000 7f6c6d124640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f6c680b5e90 0x7f6c6815f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:02.556 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.554+0000 7f6c6d124640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f6c680b5e90 0x7f6c6815f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:59048/0 (socket says 172.21.15.167:59048) 2024-05-11T08:06:02.556 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.554+0000 7f6c6d124640 1 -- 172.21.15.167:0/1037350289 learned_addr learned my addr 172.21.15.167:0/1037350289 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:02.556 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.554+0000 7f6c6d925640 1 --2- 172.21.15.167:0/1037350289 >> v2:172.21.15.204:3300/0 conn(0x7f6c6815f360 0x7f6c680cfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:02.556 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.554+0000 7f6c6d124640 1 -- 172.21.15.167:0/1037350289 >> v2:172.21.15.204:3300/0 conn(0x7f6c6815f360 msgr2=0x7f6c680cfdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:02.556 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.554+0000 7f6c6d124640 1 --2- 172.21.15.167:0/1037350289 >> v2:172.21.15.204:3300/0 conn(0x7f6c6815f360 0x7f6c680cfdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:02.556 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.554+0000 7f6c6d124640 1 -- 172.21.15.167:0/1037350289 >> v1:172.21.15.204:6789/0 conn(0x7f6c6815fc80 legacy=0x7f6c680cd610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:02.556 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.554+0000 7f6c6d124640 1 -- 172.21.15.167:0/1037350289 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c680b7e20 con 0x7f6c680b5e90 2024-05-11T08:06:02.556 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.555+0000 7f6c6d124640 1 --2- 172.21.15.167:0/1037350289 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6c680b5e90 0x7f6c6815f740 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f6c680d09a0 tx=0x7f6c5802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8ea1c6cc03bcff8 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:02.556 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.555+0000 7f6c6c923640 1 -- 172.21.15.167:0/1037350289 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c58037070 con 0x7f6c680b5e90 2024-05-11T08:06:02.556 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.555+0000 7f6c6c923640 1 -- 172.21.15.167:0/1037350289 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c58032440 con 0x7f6c680b5e90 2024-05-11T08:06:02.556 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.555+0000 7f6c6c923640 1 -- 172.21.15.167:0/1037350289 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c58032810 con 0x7f6c680b5e90 2024-05-11T08:06:02.560 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.556+0000 7f6c6fb89640 1 -- 172.21.15.167:0/1037350289 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6c680b5e90 msgr2=0x7f6c6815f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:02.560 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.556+0000 7f6c6fb89640 1 --2- 172.21.15.167:0/1037350289 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6c680b5e90 0x7f6c6815f740 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f6c680d09a0 tx=0x7f6c5802a140 comp rx=0 tx=0).stop 2024-05-11T08:06:02.560 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.556+0000 7f6c6fb89640 1 -- 172.21.15.167:0/1037350289 shutdown_connections 2024-05-11T08:06:02.560 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.556+0000 7f6c6fb89640 1 --2- 172.21.15.167:0/1037350289 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6c680b5e90 0x7f6c6815f740 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:02.560 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.556+0000 7f6c6fb89640 1 --2- 172.21.15.167:0/1037350289 >> v2:172.21.15.204:3300/0 conn(0x7f6c6815f360 0x7f6c680cfdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:02.560 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.556+0000 7f6c6fb89640 1 -- 172.21.15.167:0/1037350289 >> 172.21.15.167:0/1037350289 conn(0x7f6c68072730 msgr2=0x7f6c680d2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:02.560 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.556+0000 7f6c6fb89640 1 -- 172.21.15.167:0/1037350289 shutdown_connections 2024-05-11T08:06:02.560 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.556+0000 7f6c6fb89640 1 -- 172.21.15.167:0/1037350289 wait complete. 2024-05-11T08:06:02.560 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.557+0000 7f6c6fb89640 1 Processor -- start 2024-05-11T08:06:02.560 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.557+0000 7f6c6fb89640 1 -- start start 2024-05-11T08:06:02.560 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.558+0000 7f6c6fb89640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6c680b5e90 0x7f6c6814f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:02.560 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.558+0000 7f6c6fb89640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6c6815f360 0x7f6c6815be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:02.561 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.558+0000 7f6c6fb89640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6c6815fc80 0x7f6c680d0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:02.561 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.558+0000 7f6c6fb89640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6c680cdd20 con 0x7f6c6815fc80 2024-05-11T08:06:02.561 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.558+0000 7f6c6fb89640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6c680b7ad0 con 0x7f6c6815f360 2024-05-11T08:06:02.561 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.558+0000 7f6c6fb89640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f6c680cdea0 con 0x7f6c680b5e90 2024-05-11T08:06:02.561 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.558+0000 7f6c6e126640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6c6815fc80 0x7f6c680d0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:02.561 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.558+0000 7f6c6e126640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6c6815fc80 0x7f6c680d0ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:59062/0 (socket says 172.21.15.167:59062) 2024-05-11T08:06:02.561 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.558+0000 7f6c6e126640 1 -- 172.21.15.167:0/1654596045 learned_addr learned my addr 172.21.15.167:0/1654596045 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:02.561 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.558+0000 7f6c6d124640 1 --2- 172.21.15.167:0/1654596045 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6c6815f360 0x7f6c6815be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:02.561 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.558+0000 7f6c6d925640 1 --2- 172.21.15.167:0/1654596045 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6c680b5e90 0x7f6c6814f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:02.561 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.558+0000 7f6c6e126640 1 -- 172.21.15.167:0/1654596045 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6c680b5e90 msgr2=0x7f6c6814f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:02.561 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.558+0000 7f6c6e126640 1 --2- 172.21.15.167:0/1654596045 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6c680b5e90 0x7f6c6814f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:02.561 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.558+0000 7f6c6e126640 1 -- 172.21.15.167:0/1654596045 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6c6815f360 msgr2=0x7f6c6815be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:02.561 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.558+0000 7f6c6e126640 1 --2- 172.21.15.167:0/1654596045 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6c6815f360 0x7f6c6815be20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:02.562 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.558+0000 7f6c6e126640 1 -- 172.21.15.167:0/1654596045 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c6806e2c0 con 0x7f6c6815fc80 2024-05-11T08:06:02.562 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.558+0000 7f6c6d124640 1 --2- 172.21.15.167:0/1654596045 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6c6815f360 0x7f6c6815be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T08:06:02.562 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.558+0000 7f6c6d925640 1 --2- 172.21.15.167:0/1654596045 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6c680b5e90 0x7f6c6814f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T08:06:02.562 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.558+0000 7f6c6e126640 1 --2- 172.21.15.167:0/1654596045 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6c6815fc80 0x7f6c680d0ec0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f6c6000b490 tx=0x7f6c6000b950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:02.562 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.559+0000 7f6c5e7fc640 1 -- 172.21.15.167:0/1654596045 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c60015020 con 0x7f6c6815fc80 2024-05-11T08:06:02.562 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.559+0000 7f6c5e7fc640 1 -- 172.21.15.167:0/1654596045 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c60011850 con 0x7f6c6815fc80 2024-05-11T08:06:02.562 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.559+0000 7f6c6fb89640 1 -- 172.21.15.167:0/1654596045 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c6806e5a0 con 0x7f6c6815fc80 2024-05-11T08:06:02.562 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.559+0000 7f6c5e7fc640 1 -- 172.21.15.167:0/1654596045 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c60011c40 con 0x7f6c6815fc80 2024-05-11T08:06:02.562 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.559+0000 7f6c6fb89640 1 -- 172.21.15.167:0/1654596045 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c6806eb60 con 0x7f6c6815fc80 2024-05-11T08:06:02.569 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.560+0000 7f6c5e7fc640 1 -- 172.21.15.167:0/1654596045 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f6c60018020 con 0x7f6c6815fc80 2024-05-11T08:06:02.569 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.561+0000 7f6c5e7fc640 1 --2- 172.21.15.167:0/1654596045 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f6c4c0762d0 0x7f6c4c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:02.569 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.561+0000 7f6c5e7fc640 1 -- 172.21.15.167:0/1654596045 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f6c60097810 con 0x7f6c6815fc80 2024-05-11T08:06:02.569 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.561+0000 7f6c6d925640 1 --2- 172.21.15.167:0/1654596045 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f6c4c0762d0 0x7f6c4c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:02.569 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.561+0000 7f6c6d925640 1 --2- 172.21.15.167:0/1654596045 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f6c4c0762d0 0x7f6c4c078790 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6c6400e4a0 tx=0x7f6c64007650 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:02.569 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.562+0000 7f6c7297b740 1 Processor -- start 2024-05-11T08:06:02.569 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.562+0000 7f6c7297b740 1 -- start start 2024-05-11T08:06:02.569 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.562+0000 7f6c7297b740 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55991f481ff0 0x55991f484800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:02.570 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.562+0000 7f6c7297b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55991f4823d0 0x55991f484d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:02.570 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.562+0000 7f6c7297b740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55991f454c50 0x55991f458460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:02.570 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.562+0000 7f6c7297b740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55991f3be8d0 con 0x55991f454c50 2024-05-11T08:06:02.570 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.562+0000 7f6c7297b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55991f377c10 con 0x55991f4823d0 2024-05-11T08:06:02.570 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.562+0000 7f6c7297b740 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x55991f3ba890 con 0x55991f481ff0 2024-05-11T08:06:02.570 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.562+0000 7f6c6e126640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55991f454c50 0x55991f458460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:02.570 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.562+0000 7f6c6e126640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55991f454c50 0x55991f458460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:59070/0 (socket says 172.21.15.167:59070) 2024-05-11T08:06:02.570 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.562+0000 7f6c6e126640 1 -- 172.21.15.167:0/3634097115 learned_addr learned my addr 172.21.15.167:0/3634097115 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:02.570 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.563+0000 7f6c6d925640 1 --2- 172.21.15.167:0/3634097115 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55991f481ff0 0x55991f484800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:02.570 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.563+0000 7f6c6d124640 1 --2- 172.21.15.167:0/3634097115 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55991f4823d0 0x55991f484d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:02.570 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.563+0000 7f6c6e126640 1 -- 172.21.15.167:0/3634097115 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55991f481ff0 msgr2=0x55991f484800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:02.570 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.563+0000 7f6c6e126640 1 --2- 172.21.15.167:0/3634097115 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55991f481ff0 0x55991f484800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:02.570 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.563+0000 7f6c6e126640 1 -- 172.21.15.167:0/3634097115 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55991f4823d0 msgr2=0x55991f484d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:02.571 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.563+0000 7f6c6e126640 1 --2- 172.21.15.167:0/3634097115 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55991f4823d0 0x55991f484d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:02.571 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.563+0000 7f6c6e126640 1 -- 172.21.15.167:0/3634097115 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55991f37f5d0 con 0x55991f454c50 2024-05-11T08:06:02.571 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.563+0000 7f6c6e126640 1 --2- 172.21.15.167:0/3634097115 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55991f454c50 0x55991f458460 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f6c6005cd20 tx=0x7f6c6005ff60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ebc4e2e9e72c2e5 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:02.571 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.563+0000 7f6c4b7fe640 1 -- 172.21.15.167:0/3634097115 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c60060730 con 0x55991f454c50 2024-05-11T08:06:02.571 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.563+0000 7f6c4b7fe640 1 -- 172.21.15.167:0/3634097115 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c60064ae0 con 0x55991f454c50 2024-05-11T08:06:02.571 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.563+0000 7f6c4b7fe640 1 -- 172.21.15.167:0/3634097115 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c60069500 con 0x55991f454c50 2024-05-11T08:06:02.571 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.564+0000 7f6c7297b740 1 -- 172.21.15.167:0/3634097115 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55991f454c50 msgr2=0x55991f458460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:02.571 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.564+0000 7f6c7297b740 1 --2- 172.21.15.167:0/3634097115 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55991f454c50 0x55991f458460 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f6c6005cd20 tx=0x7f6c6005ff60 comp rx=0 tx=0).stop 2024-05-11T08:06:02.571 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.564+0000 7f6c7297b740 1 -- 172.21.15.167:0/3634097115 shutdown_connections 2024-05-11T08:06:02.571 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.564+0000 7f6c7297b740 1 --2- 172.21.15.167:0/3634097115 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55991f454c50 0x55991f458460 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:02.571 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.564+0000 7f6c7297b740 1 --2- 172.21.15.167:0/3634097115 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55991f4823d0 0x55991f484d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:02.571 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.564+0000 7f6c7297b740 1 --2- 172.21.15.167:0/3634097115 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55991f481ff0 0x55991f484800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:02.571 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.564+0000 7f6c7297b740 1 -- 172.21.15.167:0/3634097115 >> 172.21.15.167:0/3634097115 conn(0x55991f456ec0 msgr2=0x55991f49f800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:02.571 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.564+0000 7f6c7297b740 1 -- 172.21.15.167:0/3634097115 shutdown_connections 2024-05-11T08:06:02.572 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.564+0000 7f6c7297b740 1 -- 172.21.15.167:0/3634097115 wait complete. 2024-05-11T08:06:02.572 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.564+0000 7f6c7297b740 1 -- start start 2024-05-11T08:06:02.572 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.564+0000 7f6c7297b740 1 Processor -- start 2024-05-11T08:06:02.572 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.565+0000 7f6c7297b740 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55991f481ff0 0x55991f47abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:02.572 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.565+0000 7f6c7297b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55991f4823d0 0x55991f4a6630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:02.572 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.565+0000 7f6c7297b740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55991f4a7520 0x55991f4a79a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:02.572 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.565+0000 7f6c7297b740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55991f45a070 con 0x55991f4a7520 2024-05-11T08:06:02.572 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.565+0000 7f6c7297b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55991f37b760 con 0x55991f4823d0 2024-05-11T08:06:02.572 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.565+0000 7f6c7297b740 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x55991f45a350 con 0x55991f481ff0 2024-05-11T08:06:02.572 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.565+0000 7f6c7297b740 10 client.0 _ll_register_callbacks cb 0x55991f3b4cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T08:06:02.572 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.565+0000 7f6c6e126640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55991f4a7520 0x55991f4a79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:02.572 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.565+0000 7f6c6e126640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55991f4a7520 0x55991f4a79a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:59072/0 (socket says 172.21.15.167:59072) 2024-05-11T08:06:02.572 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.565+0000 7f6c6e126640 1 -- 172.21.15.167:0/4105049212 learned_addr learned my addr 172.21.15.167:0/4105049212 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:02.572 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.565+0000 7f6c6d124640 1 --2- 172.21.15.167:0/4105049212 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55991f4823d0 0x55991f4a6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:02.573 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.565+0000 7f6c6d925640 1 --2- 172.21.15.167:0/4105049212 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55991f481ff0 0x55991f47abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:02.573 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.566+0000 7f6c6e126640 1 -- 172.21.15.167:0/4105049212 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55991f481ff0 msgr2=0x55991f47abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:02.573 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.566+0000 7f6c6e126640 1 --2- 172.21.15.167:0/4105049212 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55991f481ff0 0x55991f47abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:02.573 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.566+0000 7f6c6e126640 1 -- 172.21.15.167:0/4105049212 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55991f4823d0 msgr2=0x55991f4a6630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:02.573 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.566+0000 7f6c6e126640 1 --2- 172.21.15.167:0/4105049212 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55991f4823d0 0x55991f4a6630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:02.573 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.566+0000 7f6c6e126640 1 -- 172.21.15.167:0/4105049212 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55991f0d02d0 con 0x55991f4a7520 2024-05-11T08:06:02.573 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.566+0000 7f6c6d925640 1 --2- 172.21.15.167:0/4105049212 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55991f481ff0 0x55991f47abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T08:06:02.573 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.566+0000 7f6c6e126640 1 --2- 172.21.15.167:0/4105049212 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55991f4a7520 0x55991f4a79a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f6c6005c760 tx=0x7f6c6000e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a2080f20f63752e server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:02.573 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.566+0000 7f6c48ff9640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-05-11T08:06:02.573 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.566+0000 7f6c48ff9640 1 -- 172.21.15.167:0/4105049212 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c6008eba0 con 0x55991f4a7520 2024-05-11T08:06:02.573 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.566+0000 7f6c7297b740 1 -- 172.21.15.167:0/4105049212 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c60064ae0 con 0x55991f4a7520 2024-05-11T08:06:02.573 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.566+0000 7f6c48ff9640 1 -- 172.21.15.167:0/4105049212 <== mon.0 v2:172.21.15.167:3300/0 2 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f6c6008ed40 con 0x55991f4a7520 2024-05-11T08:06:02.573 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.566+0000 7f6c48ff9640 1 -- 172.21.15.167:0/4105049212 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) ==== 24+0+0 (secure 0 0 0) 0x7f6c6005f990 con 0x55991f4a7520 2024-05-11T08:06:02.573 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.566+0000 7f6c7297b740 10 client.4428 fetch_fsmap learned FSMap version 7 2024-05-11T08:06:02.573 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.566+0000 7f6c7297b740 1 -- 172.21.15.167:0/4105049212 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) -- 0x55991f19a6b0 con 0x55991f4a7520 2024-05-11T08:06:02.573 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.566+0000 7f6c48ff9640 1 -- 172.21.15.167:0/4105049212 <== mon.0 v2:172.21.15.167:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c60064ae0 con 0x55991f4a7520 2024-05-11T08:06:02.574 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.566+0000 7f6c48ff9640 1 -- 172.21.15.167:0/4105049212 <== mon.0 v2:172.21.15.167:3300/0 5 ==== fsmap(e 7) ==== 3502+0+0 (secure 0 0 0) 0x7f6c6005fb30 con 0x55991f4a7520 2024-05-11T08:06:02.574 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.566+0000 7f6c7297b740 10 client.4428 fetch_fsmap finished waiting for FSMap version 7 2024-05-11T08:06:02.574 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.566+0000 7f6c7297b740 10 client.4428 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-11T08:06:02.574 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.566+0000 7f6c7297b740 20 client.4428 populate_metadata read hostname 'smithi167' 2024-05-11T08:06:02.574 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.566+0000 7f6c7297b740 1 --2- 172.21.15.167:0/4105049212 >> [v2:172.21.15.204:6833/1247774569,v1:172.21.15.204:6835/1247774569] conn(0x55991f47b790 0x55991f47bb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:02.574 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.566+0000 7f6c7297b740 4 client.4428 mds_command: new command op to 4305 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-11T08:06:02.574 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.566+0000 7f6c7297b740 1 -- 172.21.15.167:0/4105049212 --> [v2:172.21.15.204:6833/1247774569,v1:172.21.15.204:6835/1247774569] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55991f342760 con 0x55991f47b790 2024-05-11T08:06:02.574 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.567+0000 7f6c6d925640 1 --2- 172.21.15.167:0/4105049212 >> [v2:172.21.15.204:6833/1247774569,v1:172.21.15.204:6835/1247774569] conn(0x55991f47b790 0x55991f47bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:02.574 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.567+0000 7f6c48ff9640 10 client.4428 ms_handle_connect on v2:172.21.15.204:6833/1247774569 2024-05-11T08:06:02.574 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.567+0000 7f6c6d925640 1 --2- 172.21.15.167:0/4105049212 >> [v2:172.21.15.204:6833/1247774569,v1:172.21.15.204:6835/1247774569] conn(0x55991f47b790 0x55991f47bb70 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e54224931e0e9f server_cookie=f03d5bbdd7b2877c in_seq=0 out_seq=0 2024-05-11T08:06:02.574 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.568+0000 7f6c48ff9640 1 -- 172.21.15.167:0/4105049212 <== mds.0 v2:172.21.15.204:6833/1247774569 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55991f342760 con 0x55991f47b790 2024-05-11T08:06:02.575 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.568+0000 7f6c48ff9640 10 client.4428 handle_command_reply: tid=0 2024-05-11T08:06:02.575 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.568+0000 7f6c7297b740 1 client.4428 shutdown 2024-05-11T08:06:02.575 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.568+0000 7f6c7297b740 1 -- 172.21.15.167:0/4105049212 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55991f4a7520 msgr2=0x55991f4a79a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:02.575 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.568+0000 7f6c7297b740 1 --2- 172.21.15.167:0/4105049212 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55991f4a7520 0x55991f4a79a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f6c6005c760 tx=0x7f6c6000e000 comp rx=0 tx=0).stop 2024-05-11T08:06:02.575 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.568+0000 7f6c7297b740 1 -- 172.21.15.167:0/4105049212 shutdown_connections 2024-05-11T08:06:02.575 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.568+0000 7f6c7297b740 1 --2- 172.21.15.167:0/4105049212 >> [v2:172.21.15.204:6833/1247774569,v1:172.21.15.204:6835/1247774569] conn(0x55991f47b790 0x55991f47bb70 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:02.575 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.568+0000 7f6c7297b740 1 --2- 172.21.15.167:0/4105049212 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55991f4a7520 0x55991f4a79a0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:02.575 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.568+0000 7f6c7297b740 1 --2- 172.21.15.167:0/4105049212 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55991f4823d0 0x55991f4a6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:02.575 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.568+0000 7f6c48ff9640 0 client.4428 ms_handle_reset on v2:172.21.15.204:6833/1247774569 2024-05-11T08:06:02.575 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.568+0000 7f6c7297b740 1 --2- 172.21.15.167:0/4105049212 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55991f481ff0 0x55991f47abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:02.575 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.568+0000 7f6c7297b740 1 -- 172.21.15.167:0/4105049212 >> 172.21.15.167:0/4105049212 conn(0x55991f454c50 msgr2=0x55991f47c860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:02.575 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.568+0000 7f6c7297b740 1 -- 172.21.15.167:0/4105049212 shutdown_connections 2024-05-11T08:06:02.575 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.568+0000 7f6c7297b740 1 -- 172.21.15.167:0/4105049212 wait complete. 2024-05-11T08:06:02.575 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.568+0000 7f6c7297b740 20 client.4428 trim_cache size 0 max 16384 2024-05-11T08:06:02.587 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.587+0000 7f6c7297b740 1 Processor -- start 2024-05-11T08:06:02.587 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.587+0000 7f6c7297b740 1 -- start start 2024-05-11T08:06:02.589 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.587+0000 7f6c7297b740 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55991f47b790 0x55991f450130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:02.589 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.587+0000 7f6c7297b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55991f481ff0 0x55991f49d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:02.589 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.587+0000 7f6c7297b740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55991f4823d0 0x55991f453dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:02.589 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.587+0000 7f6c7297b740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6c68201ee0 con 0x55991f4823d0 2024-05-11T08:06:02.589 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.587+0000 7f6c7297b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6c68201390 con 0x55991f481ff0 2024-05-11T08:06:02.590 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.587+0000 7f6c7297b740 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f6c68201890 con 0x55991f47b790 2024-05-11T08:06:02.590 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.587+0000 7f6c6e126640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55991f4823d0 0x55991f453dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:02.590 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.588+0000 7f6c6e126640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55991f4823d0 0x55991f453dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:59074/0 (socket says 172.21.15.167:59074) 2024-05-11T08:06:02.590 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.588+0000 7f6c6e126640 1 -- 172.21.15.167:0/46721841 learned_addr learned my addr 172.21.15.167:0/46721841 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:02.590 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.588+0000 7f6c6e126640 1 -- 172.21.15.167:0/46721841 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55991f47b790 msgr2=0x55991f450130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:02.590 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.588+0000 7f6c6e126640 1 --2- 172.21.15.167:0/46721841 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55991f47b790 0x55991f450130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:02.590 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.588+0000 7f6c6e126640 1 -- 172.21.15.167:0/46721841 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55991f481ff0 msgr2=0x55991f49d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:02.590 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.588+0000 7f6c6e126640 1 --2- 172.21.15.167:0/46721841 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55991f481ff0 0x55991f49d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:02.590 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.588+0000 7f6c6e126640 1 -- 172.21.15.167:0/46721841 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55991f47af60 con 0x55991f4823d0 2024-05-11T08:06:02.590 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.588+0000 7f6c6e126640 1 --2- 172.21.15.167:0/46721841 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55991f4823d0 0x55991f453dd0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f6c6005d9a0 tx=0x7f6c60090070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5535509c69aac0bd server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:02.590 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.588+0000 7f6c4affd640 1 -- 172.21.15.167:0/46721841 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c60067030 con 0x55991f4823d0 2024-05-11T08:06:02.590 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.588+0000 7f6c4affd640 1 -- 172.21.15.167:0/46721841 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c6005fb30 con 0x55991f4823d0 2024-05-11T08:06:02.590 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.588+0000 7f6c4affd640 1 -- 172.21.15.167:0/46721841 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c6008ede0 con 0x55991f4823d0 2024-05-11T08:06:02.591 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.589+0000 7f6c7297b740 1 -- 172.21.15.167:0/46721841 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55991f4823d0 msgr2=0x55991f453dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:02.591 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.589+0000 7f6c7297b740 1 --2- 172.21.15.167:0/46721841 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55991f4823d0 0x55991f453dd0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f6c6005d9a0 tx=0x7f6c60090070 comp rx=0 tx=0).stop 2024-05-11T08:06:02.591 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.589+0000 7f6c7297b740 1 -- 172.21.15.167:0/46721841 shutdown_connections 2024-05-11T08:06:02.591 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.589+0000 7f6c7297b740 1 --2- 172.21.15.167:0/46721841 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55991f4823d0 0x55991f453dd0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:02.591 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.589+0000 7f6c7297b740 1 --2- 172.21.15.167:0/46721841 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55991f481ff0 0x55991f49d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:02.591 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.589+0000 7f6c7297b740 1 --2- 172.21.15.167:0/46721841 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55991f47b790 0x55991f450130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:02.591 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.589+0000 7f6c7297b740 1 -- 172.21.15.167:0/46721841 >> 172.21.15.167:0/46721841 conn(0x55991f454c50 msgr2=0x55991f4a7920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:02.591 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.589+0000 7f6c7297b740 1 -- 172.21.15.167:0/46721841 shutdown_connections 2024-05-11T08:06:02.591 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.589+0000 7f6c7297b740 1 -- 172.21.15.167:0/46721841 wait complete. 2024-05-11T08:06:02.591 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.589+0000 7f6c7297b740 1 -- start start 2024-05-11T08:06:02.591 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.589+0000 7f6c7297b740 1 Processor -- start 2024-05-11T08:06:02.595 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.590+0000 7f6c7297b740 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55991f47b790 0x55991f51cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:02.595 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.590+0000 7f6c7297b740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55991f481ff0 0x55991f488040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:02.595 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.590+0000 7f6c7297b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55991f4823d0 0x55991f488580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:02.595 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.590+0000 7f6c7297b740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55991f45a070 con 0x55991f481ff0 2024-05-11T08:06:02.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.590+0000 7f6c7297b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55991f485600 con 0x55991f4823d0 2024-05-11T08:06:02.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.590+0000 7f6c7297b740 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x55991f37b760 con 0x55991f47b790 2024-05-11T08:06:02.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.590+0000 7f6c7297b740 10 client.0 _ll_register_callbacks cb 0x7f6c60060080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T08:06:02.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.590+0000 7f6c6d124640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55991f481ff0 0x55991f488040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:02.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.590+0000 7f6c6d124640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55991f481ff0 0x55991f488040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:59086/0 (socket says 172.21.15.167:59086) 2024-05-11T08:06:02.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.590+0000 7f6c6d124640 1 -- 172.21.15.167:0/2456454158 learned_addr learned my addr 172.21.15.167:0/2456454158 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:02.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.590+0000 7f6c6d925640 1 --2- 172.21.15.167:0/2456454158 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55991f47b790 0x55991f51cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:02.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.590+0000 7f6c6e126640 1 --2- 172.21.15.167:0/2456454158 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55991f4823d0 0x55991f488580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:02.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.590+0000 7f6c6d124640 1 -- 172.21.15.167:0/2456454158 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55991f47b790 msgr2=0x55991f51cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:02.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.590+0000 7f6c6d124640 1 --2- 172.21.15.167:0/2456454158 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55991f47b790 0x55991f51cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:02.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.590+0000 7f6c6d124640 1 -- 172.21.15.167:0/2456454158 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55991f4823d0 msgr2=0x55991f488580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:02.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.590+0000 7f6c6d124640 1 --2- 172.21.15.167:0/2456454158 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55991f4823d0 0x55991f488580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:02.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.590+0000 7f6c6d124640 1 -- 172.21.15.167:0/2456454158 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55991f484b30 con 0x55991f481ff0 2024-05-11T08:06:02.597 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.591+0000 7f6c6d124640 1 --2- 172.21.15.167:0/2456454158 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55991f481ff0 0x55991f488040 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f6c58009960 tx=0x7f6c580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba4f4514eb46835e server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:02.597 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.591+0000 7f6c48ff9640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-05-11T08:06:02.597 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.591+0000 7f6c48ff9640 1 -- 172.21.15.167:0/2456454158 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c58037040 con 0x55991f481ff0 2024-05-11T08:06:02.597 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.591+0000 7f6c48ff9640 1 -- 172.21.15.167:0/2456454158 <== mon.0 v2:172.21.15.167:3300/0 2 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f6c58047de0 con 0x55991f481ff0 2024-05-11T08:06:02.597 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.591+0000 7f6c7297b740 1 -- 172.21.15.167:0/2456454158 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c6005fb30 con 0x55991f481ff0 2024-05-11T08:06:02.597 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.591+0000 7f6c48ff9640 1 -- 172.21.15.167:0/2456454158 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) ==== 24+0+0 (secure 0 0 0) 0x7f6c58032d70 con 0x55991f481ff0 2024-05-11T08:06:02.597 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.591+0000 7f6c7297b740 10 client.4434 fetch_fsmap learned FSMap version 7 2024-05-11T08:06:02.597 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.591+0000 7f6c7297b740 1 -- 172.21.15.167:0/2456454158 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) -- 0x55991f4a2ea0 con 0x55991f481ff0 2024-05-11T08:06:02.597 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.591+0000 7f6c48ff9640 1 -- 172.21.15.167:0/2456454158 <== mon.0 v2:172.21.15.167:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c6005fb30 con 0x55991f481ff0 2024-05-11T08:06:02.597 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.591+0000 7f6c48ff9640 1 -- 172.21.15.167:0/2456454158 <== mon.0 v2:172.21.15.167:3300/0 5 ==== fsmap(e 7) ==== 3502+0+0 (secure 0 0 0) 0x7f6c5804e050 con 0x55991f481ff0 2024-05-11T08:06:02.597 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.591+0000 7f6c7297b740 10 client.4434 fetch_fsmap finished waiting for FSMap version 7 2024-05-11T08:06:02.597 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.591+0000 7f6c7297b740 10 client.4434 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-11T08:06:02.597 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.591+0000 7f6c7297b740 20 client.4434 populate_metadata read hostname 'smithi167' 2024-05-11T08:06:02.597 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.591+0000 7f6c7297b740 1 --2- 172.21.15.167:0/2456454158 >> [v2:172.21.15.204:6833/1247774569,v1:172.21.15.204:6835/1247774569] conn(0x55991f4a7520 0x55991f4c3530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:02.598 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.591+0000 7f6c7297b740 4 client.4434 mds_command: new command op to 4305 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-11T08:06:02.598 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.591+0000 7f6c7297b740 1 -- 172.21.15.167:0/2456454158 --> [v2:172.21.15.204:6833/1247774569,v1:172.21.15.204:6835/1247774569] -- command(tid 0: {"prefix": "status"}) -- 0x55991f3cc4e0 con 0x55991f4a7520 2024-05-11T08:06:02.598 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.591+0000 7f6c6d925640 1 --2- 172.21.15.167:0/2456454158 >> [v2:172.21.15.204:6833/1247774569,v1:172.21.15.204:6835/1247774569] conn(0x55991f4a7520 0x55991f4c3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:02.598 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.592+0000 7f6c48ff9640 10 client.4434 ms_handle_connect on v2:172.21.15.204:6833/1247774569 2024-05-11T08:06:02.598 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.592+0000 7f6c6d925640 1 --2- 172.21.15.167:0/2456454158 >> [v2:172.21.15.204:6833/1247774569,v1:172.21.15.204:6835/1247774569] conn(0x55991f4a7520 0x55991f4c3530 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3e627ed46f8d8e0 server_cookie=bbf163a8d9c917d3 in_seq=0 out_seq=0 2024-05-11T08:06:02.598 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.592+0000 7f6c48ff9640 1 -- 172.21.15.167:0/2456454158 <== mds.0 v2:172.21.15.204:6833/1247774569 1 ==== command_reply(tid 0: 0 ) ==== 8+0+330 (crc 0 0 0) 0x55991f3cc4e0 con 0x55991f4a7520 2024-05-11T08:06:02.598 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.592+0000 7f6c48ff9640 10 client.4434 handle_command_reply: tid=0 2024-05-11T08:06:02.598 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.592+0000 7f6c7297b740 1 client.4434 shutdown 2024-05-11T08:06:02.598 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.592+0000 7f6c7297b740 1 -- 172.21.15.167:0/2456454158 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55991f481ff0 msgr2=0x55991f488040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:02.598 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.592+0000 7f6c7297b740 1 --2- 172.21.15.167:0/2456454158 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55991f481ff0 0x55991f488040 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f6c58009960 tx=0x7f6c580046c0 comp rx=0 tx=0).stop 2024-05-11T08:06:02.598 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.592+0000 7f6c7297b740 1 -- 172.21.15.167:0/2456454158 shutdown_connections 2024-05-11T08:06:02.598 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.592+0000 7f6c7297b740 1 --2- 172.21.15.167:0/2456454158 >> [v2:172.21.15.204:6833/1247774569,v1:172.21.15.204:6835/1247774569] conn(0x55991f4a7520 0x55991f4c3530 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:02.598 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.592+0000 7f6c7297b740 1 --2- 172.21.15.167:0/2456454158 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55991f4823d0 0x55991f488580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:02.599 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.592+0000 7f6c7297b740 1 --2- 172.21.15.167:0/2456454158 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55991f481ff0 0x55991f488040 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:02.599 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.592+0000 7f6c48ff9640 0 client.4434 ms_handle_reset on v2:172.21.15.204:6833/1247774569 2024-05-11T08:06:02.599 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.592+0000 7f6c7297b740 1 --2- 172.21.15.167:0/2456454158 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55991f47b790 0x55991f51cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:02.599 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.592+0000 7f6c7297b740 1 -- 172.21.15.167:0/2456454158 >> 172.21.15.167:0/2456454158 conn(0x55991f454c50 msgr2=0x55991f479bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:02.599 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.592+0000 7f6c7297b740 1 -- 172.21.15.167:0/2456454158 shutdown_connections 2024-05-11T08:06:02.599 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.592+0000 7f6c7297b740 1 -- 172.21.15.167:0/2456454158 wait complete. 2024-05-11T08:06:02.599 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.592+0000 7f6c7297b740 20 client.4434 trim_cache size 0 max 16384 2024-05-11T08:06:02.599 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-05-11T08:06:02.599 INFO:teuthology.orchestra.run.smithi167.stdout: "cluster_fsid": "2034778d-8ff1-4823-aa44-ded32b518241", 2024-05-11T08:06:02.599 INFO:teuthology.orchestra.run.smithi167.stdout: "whoami": 0, 2024-05-11T08:06:02.600 INFO:teuthology.orchestra.run.smithi167.stdout: "id": 4305, 2024-05-11T08:06:02.600 INFO:teuthology.orchestra.run.smithi167.stdout: "want_state": "up:active", 2024-05-11T08:06:02.600 INFO:teuthology.orchestra.run.smithi167.stdout: "state": "up:active", 2024-05-11T08:06:02.600 INFO:teuthology.orchestra.run.smithi167.stdout: "fs_name": "cephfs", 2024-05-11T08:06:02.600 INFO:teuthology.orchestra.run.smithi167.stdout: "rank_uptime": 7.2422602510000003, 2024-05-11T08:06:02.600 INFO:teuthology.orchestra.run.smithi167.stdout: "mdsmap_epoch": 7, 2024-05-11T08:06:02.600 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_epoch": 27, 2024-05-11T08:06:02.600 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_epoch_barrier": 27, 2024-05-11T08:06:02.600 INFO:teuthology.orchestra.run.smithi167.stdout: "uptime": 13.257095831999999 2024-05-11T08:06:02.600 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-05-11T08:06:02.601 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.593+0000 7f6c6fb89640 1 -- 172.21.15.167:0/1654596045 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f6c4c0762d0 msgr2=0x7f6c4c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:02.601 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.593+0000 7f6c6fb89640 1 --2- 172.21.15.167:0/1654596045 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f6c4c0762d0 0x7f6c4c078790 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6c6400e4a0 tx=0x7f6c64007650 comp rx=0 tx=0).stop 2024-05-11T08:06:02.601 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.593+0000 7f6c6fb89640 1 -- 172.21.15.167:0/1654596045 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6c6815fc80 msgr2=0x7f6c680d0ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:02.601 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.593+0000 7f6c6fb89640 1 --2- 172.21.15.167:0/1654596045 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6c6815fc80 0x7f6c680d0ec0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f6c6000b490 tx=0x7f6c6000b950 comp rx=0 tx=0).stop 2024-05-11T08:06:02.601 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.593+0000 7f6c6fb89640 1 -- 172.21.15.167:0/1654596045 shutdown_connections 2024-05-11T08:06:02.601 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.593+0000 7f6c6fb89640 1 --2- 172.21.15.167:0/1654596045 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f6c4c0762d0 0x7f6c4c078790 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:02.601 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.593+0000 7f6c6fb89640 1 --2- 172.21.15.167:0/1654596045 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6c6815fc80 0x7f6c680d0ec0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:02.602 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.593+0000 7f6c6fb89640 1 --2- 172.21.15.167:0/1654596045 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6c6815f360 0x7f6c6815be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:02.602 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.593+0000 7f6c6fb89640 1 --2- 172.21.15.167:0/1654596045 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6c680b5e90 0x7f6c6814f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:02.602 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.593+0000 7f6c6fb89640 1 -- 172.21.15.167:0/1654596045 >> 172.21.15.167:0/1654596045 conn(0x7f6c68072730 msgr2=0x7f6c680ceb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:02.602 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.593+0000 7f6c6fb89640 1 -- 172.21.15.167:0/1654596045 shutdown_connections 2024-05-11T08:06:02.602 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.593+0000 7f6c6fb89640 1 -- 172.21.15.167:0/1654596045 wait complete. 2024-05-11T08:06:02.603 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-11T08:06:02.604 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-05-11T08:06:02.604 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-11T08:06:02.704 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.704+0000 7f95cf607640 1 Processor -- start 2024-05-11T08:06:02.704 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.704+0000 7f95cf607640 1 -- start start 2024-05-11T08:06:02.708 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.705+0000 7f95cf607640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f95c80b5e90 0x7f95c815f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:02.708 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.705+0000 7f95cf607640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95c80d2410 con 0x7f95c815fc80 2024-05-11T08:06:02.708 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.705+0000 7f95cf607640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95c80d26f0 con 0x7f95c815f360 2024-05-11T08:06:02.708 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.705+0000 7f95cf607640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f95c80d29d0 con 0x7f95c80b5e90 2024-05-11T08:06:02.708 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.705+0000 7f95cdba4640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f95c815fc80 0x7f95c80cd610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:40646/0 (socket says 172.21.15.167:40646) 2024-05-11T08:06:02.708 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.705+0000 7f95cdba4640 1 -- 172.21.15.167:0/4107721234 learned_addr learned my addr 172.21.15.167:0/4107721234 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T08:06:02.708 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.705+0000 7f95ccba2640 1 --2- 172.21.15.167:0/4107721234 >> v2:172.21.15.204:3300/0 conn(0x7f95c80b5e90 0x7f95c815f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:02.708 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.705+0000 7f95bffff640 1 -- 172.21.15.167:0/4107721234 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1239110659 0 0) 0x7f95c80d2410 con 0x7f95c815fc80 2024-05-11T08:06:02.708 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.705+0000 7f95bffff640 1 -- 172.21.15.167:0/4107721234 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f95b4003520 con 0x7f95c815fc80 2024-05-11T08:06:02.708 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.705+0000 7f95bffff640 1 -- 172.21.15.167:0/4107721234 <== mon.0 v1:172.21.15.167:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3118527667 0 0) 0x7f95b4003520 con 0x7f95c815fc80 2024-05-11T08:06:02.708 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.705+0000 7f95bffff640 1 -- 172.21.15.167:0/4107721234 >> v2:172.21.15.204:3300/0 conn(0x7f95c80b5e90 msgr2=0x7f95c815f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:02.708 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.705+0000 7f95bffff640 1 --2- 172.21.15.167:0/4107721234 >> v2:172.21.15.204:3300/0 conn(0x7f95c80b5e90 0x7f95c815f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:02.709 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.705+0000 7f95bffff640 1 -- 172.21.15.167:0/4107721234 >> v1:172.21.15.204:6789/0 conn(0x7f95c815f360 legacy=0x7f95c80cfdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:02.709 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.705+0000 7f95bffff640 1 -- 172.21.15.167:0/4107721234 --> v1:172.21.15.167:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95c80b7e20 con 0x7f95c815fc80 2024-05-11T08:06:02.709 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.705+0000 7f95bffff640 1 -- 172.21.15.167:0/4107721234 <== mon.0 v1:172.21.15.167:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3602277846 0 0) 0x7f95c00016f0 con 0x7f95c815fc80 2024-05-11T08:06:02.709 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.705+0000 7f95bffff640 1 -- 172.21.15.167:0/4107721234 >> v1:172.21.15.167:6789/0 conn(0x7f95c815fc80 legacy=0x7f95c80cd610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:02.709 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.706+0000 7f95bffff640 1 --2- 172.21.15.167:0/4107721234 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f95b4003a50 0x7f95b4003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:02.709 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.706+0000 7f95bffff640 1 --2- 172.21.15.167:0/4107721234 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f95b4004800 0x7f95b4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:02.709 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.706+0000 7f95bffff640 1 --2- 172.21.15.167:0/4107721234 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f95b40052e0 0x7f95b400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:02.709 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.706+0000 7f95bffff640 1 -- 172.21.15.167:0/4107721234 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f95c80d29d0 con 0x7f95b4004800 2024-05-11T08:06:02.709 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.706+0000 7f95bffff640 1 -- 172.21.15.167:0/4107721234 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f95b400c1b0 con 0x7f95b4003a50 2024-05-11T08:06:02.709 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.706+0000 7f95bffff640 1 -- 172.21.15.167:0/4107721234 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f95b400c3b0 con 0x7f95b40052e0 2024-05-11T08:06:02.709 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.706+0000 7f95ccba2640 1 --2- 172.21.15.167:0/4107721234 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f95b4004800 0x7f95b4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:02.709 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.706+0000 7f95cd3a3640 1 --2- 172.21.15.167:0/4107721234 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f95b4003a50 0x7f95b4003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:02.709 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.706+0000 7f95cdba4640 1 --2- 172.21.15.167:0/4107721234 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f95b40052e0 0x7f95b400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:02.709 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.706+0000 7f95ccba2640 1 -- 172.21.15.167:0/4107721234 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f95b40052e0 msgr2=0x7f95b400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:02.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.706+0000 7f95ccba2640 1 --2- 172.21.15.167:0/4107721234 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f95b40052e0 0x7f95b400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:02.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.706+0000 7f95ccba2640 1 -- 172.21.15.167:0/4107721234 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f95b4003a50 msgr2=0x7f95b4003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:02.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.706+0000 7f95ccba2640 1 --2- 172.21.15.167:0/4107721234 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f95b4003a50 0x7f95b4003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:02.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.706+0000 7f95ccba2640 1 -- 172.21.15.167:0/4107721234 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95b4003520 con 0x7f95b4004800 2024-05-11T08:06:02.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.706+0000 7f95cdba4640 1 -- 172.21.15.167:0/4107721234 reap_dead start 2024-05-11T08:06:02.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.707+0000 7f95ccba2640 1 --2- 172.21.15.167:0/4107721234 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f95b4004800 0x7f95b4004c80 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f95b8009a90 tx=0x7f95b4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f5b1326cb213226 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:02.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.707+0000 7f95bffff640 1 -- 172.21.15.167:0/4107721234 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95b8036070 con 0x7f95b4004800 2024-05-11T08:06:02.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.707+0000 7f95bffff640 1 -- 172.21.15.167:0/4107721234 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95b8031400 con 0x7f95b4004800 2024-05-11T08:06:02.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.707+0000 7f95bffff640 1 -- 172.21.15.167:0/4107721234 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95b8009690 con 0x7f95b4004800 2024-05-11T08:06:02.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.707+0000 7f95cf607640 1 -- 172.21.15.167:0/4107721234 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f95b4004800 msgr2=0x7f95b4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:02.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.707+0000 7f95cf607640 1 --2- 172.21.15.167:0/4107721234 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f95b4004800 0x7f95b4004c80 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f95b8009a90 tx=0x7f95b4005830 comp rx=0 tx=0).stop 2024-05-11T08:06:02.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.707+0000 7f95cf607640 1 -- 172.21.15.167:0/4107721234 shutdown_connections 2024-05-11T08:06:02.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.707+0000 7f95cf607640 1 --2- 172.21.15.167:0/4107721234 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f95b4004800 0x7f95b4004c80 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:02.711 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.707+0000 7f95cf607640 1 -- 172.21.15.167:0/4107721234 >> 172.21.15.167:0/4107721234 conn(0x7f95c8072730 msgr2=0x7f95c80d2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:02.711 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.707+0000 7f95cf607640 1 -- 172.21.15.167:0/4107721234 shutdown_connections 2024-05-11T08:06:02.711 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.707+0000 7f95cf607640 1 -- 172.21.15.167:0/4107721234 wait complete. 2024-05-11T08:06:02.713 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.708+0000 7f95cf607640 1 Processor -- start 2024-05-11T08:06:02.713 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.709+0000 7f95cf607640 1 -- start start 2024-05-11T08:06:02.714 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.709+0000 7f95cf607640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f95b4004800 0x7f95c80ce820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:02.714 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.709+0000 7f95cf607640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f95c8153790 0x7f95c81568d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:02.714 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.709+0000 7f95cf607640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f95c8156e10 0x7f95c80712e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:02.714 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.709+0000 7f95cf607640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f95c815f8c0 con 0x7f95c8156e10 2024-05-11T08:06:02.714 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.709+0000 7f95cf607640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f95c815f740 con 0x7f95b4004800 2024-05-11T08:06:02.714 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.709+0000 7f95cf607640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f95c815fa40 con 0x7f95c8153790 2024-05-11T08:06:02.714 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.709+0000 7f95cdba4640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f95c8156e10 0x7f95c80712e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:02.714 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.709+0000 7f95cdba4640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f95c8156e10 0x7f95c80712e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:59102/0 (socket says 172.21.15.167:59102) 2024-05-11T08:06:02.714 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.709+0000 7f95cdba4640 1 -- 172.21.15.167:0/3085258856 learned_addr learned my addr 172.21.15.167:0/3085258856 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:02.714 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.709+0000 7f95ccba2640 1 --2- 172.21.15.167:0/3085258856 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f95c8153790 0x7f95c81568d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:02.714 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.709+0000 7f95cd3a3640 1 --2- 172.21.15.167:0/3085258856 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f95b4004800 0x7f95c80ce820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:02.714 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.710+0000 7f95cdba4640 1 -- 172.21.15.167:0/3085258856 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f95c8153790 msgr2=0x7f95c81568d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:02.714 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.710+0000 7f95cdba4640 1 --2- 172.21.15.167:0/3085258856 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f95c8153790 0x7f95c81568d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:02.714 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.710+0000 7f95cdba4640 1 -- 172.21.15.167:0/3085258856 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f95b4004800 msgr2=0x7f95c80ce820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:02.715 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.710+0000 7f95cdba4640 1 --2- 172.21.15.167:0/3085258856 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f95b4004800 0x7f95c80ce820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:02.715 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.710+0000 7f95cdba4640 1 -- 172.21.15.167:0/3085258856 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95c8157330 con 0x7f95c8156e10 2024-05-11T08:06:02.715 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.710+0000 7f95cd3a3640 1 --2- 172.21.15.167:0/3085258856 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f95b4004800 0x7f95c80ce820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T08:06:02.715 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.710+0000 7f95cdba4640 1 --2- 172.21.15.167:0/3085258856 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f95c8156e10 0x7f95c80712e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f95c0000c00 tx=0x7f95c0007430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:02.715 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.710+0000 7f95bdffb640 1 -- 172.21.15.167:0/3085258856 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95c000f620 con 0x7f95c8156e10 2024-05-11T08:06:02.715 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.710+0000 7f95bdffb640 1 -- 172.21.15.167:0/3085258856 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95c0018070 con 0x7f95c8156e10 2024-05-11T08:06:02.715 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.710+0000 7f95cf607640 1 -- 172.21.15.167:0/3085258856 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95c8157610 con 0x7f95c8156e10 2024-05-11T08:06:02.715 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.710+0000 7f95bdffb640 1 -- 172.21.15.167:0/3085258856 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95c000fa40 con 0x7f95c8156e10 2024-05-11T08:06:02.715 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.710+0000 7f95cf607640 1 -- 172.21.15.167:0/3085258856 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95c8157bd0 con 0x7f95c8156e10 2024-05-11T08:06:02.715 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.711+0000 7f95cf607640 1 -- 172.21.15.167:0/3085258856 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95c8069590 con 0x7f95c8156e10 2024-05-11T08:06:02.715 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.712+0000 7f95bdffb640 1 -- 172.21.15.167:0/3085258856 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f95c000fbe0 con 0x7f95c8156e10 2024-05-11T08:06:02.715 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.712+0000 7f95bdffb640 1 --2- 172.21.15.167:0/3085258856 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f95ac0762d0 0x7f95ac078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:02.715 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.712+0000 7f95bdffb640 1 -- 172.21.15.167:0/3085258856 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f95c0097630 con 0x7f95c8156e10 2024-05-11T08:06:02.715 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.712+0000 7f95cd3a3640 1 --2- 172.21.15.167:0/3085258856 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f95ac0762d0 0x7f95ac078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:02.716 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.713+0000 7f95cd3a3640 1 --2- 172.21.15.167:0/3085258856 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f95ac0762d0 0x7f95ac078790 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f95c4000f80 tx=0x7f95c4009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:02.716 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.716+0000 7f95bdffb640 1 -- 172.21.15.167:0/3085258856 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f95c0060ce0 con 0x7f95c8156e10 2024-05-11T08:06:02.934 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.934+0000 7f95cf607640 1 -- 172.21.15.167:0/3085258856 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f95c80c95d0 con 0x7f95c8156e10 2024-05-11T08:06:02.934 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.934+0000 7f95bdffb640 1 -- 172.21.15.167:0/3085258856 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) ==== 53+0+88 (secure 0 0 0) 0x7f95c00605e0 con 0x7f95c8156e10 2024-05-11T08:06:02.935 INFO:teuthology.orchestra.run.smithi167.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-05-11T08:06:02.937 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.936+0000 7f95cf607640 1 -- 172.21.15.167:0/3085258856 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f95ac0762d0 msgr2=0x7f95ac078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:02.937 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.936+0000 7f95cf607640 1 --2- 172.21.15.167:0/3085258856 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f95ac0762d0 0x7f95ac078790 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f95c4000f80 tx=0x7f95c4009040 comp rx=0 tx=0).stop 2024-05-11T08:06:02.937 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.936+0000 7f95cf607640 1 -- 172.21.15.167:0/3085258856 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f95c8156e10 msgr2=0x7f95c80712e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:02.937 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.936+0000 7f95cf607640 1 --2- 172.21.15.167:0/3085258856 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f95c8156e10 0x7f95c80712e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f95c0000c00 tx=0x7f95c0007430 comp rx=0 tx=0).stop 2024-05-11T08:06:02.937 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.936+0000 7f95cf607640 1 -- 172.21.15.167:0/3085258856 shutdown_connections 2024-05-11T08:06:02.937 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.936+0000 7f95cf607640 1 --2- 172.21.15.167:0/3085258856 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f95ac0762d0 0x7f95ac078790 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:02.937 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.936+0000 7f95cf607640 1 --2- 172.21.15.167:0/3085258856 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f95c8156e10 0x7f95c80712e0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:02.937 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.936+0000 7f95cf607640 1 --2- 172.21.15.167:0/3085258856 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f95c8153790 0x7f95c81568d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:02.937 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.936+0000 7f95cf607640 1 --2- 172.21.15.167:0/3085258856 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f95b4004800 0x7f95c80ce820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:02.937 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.936+0000 7f95cf607640 1 -- 172.21.15.167:0/3085258856 >> 172.21.15.167:0/3085258856 conn(0x7f95c8072730 msgr2=0x7f95c80cf600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:02.937 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.936+0000 7f95cf607640 1 -- 172.21.15.167:0/3085258856 shutdown_connections 2024-05-11T08:06:02.938 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:02.936+0000 7f95cf607640 1 -- 172.21.15.167:0/3085258856 wait complete. 2024-05-11T08:06:02.947 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-11T08:06:02.947 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-11T08:06:02.948 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi167.front.sepia.ceph.com 2024-05-11T08:06:02.948 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-05-11T08:06:02.948 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-05-11T08:06:02.948 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-11T08:06:02.948 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-11T08:06:02.948 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-11T08:06:02.948 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-05-11T08:06:02.948 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T08:06:02.948 DEBUG:teuthology.orchestra.run.smithi167:> ip addr 2024-05-11T08:06:02.964 INFO:teuthology.orchestra.run.smithi167.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-11T08:06:02.964 INFO:teuthology.orchestra.run.smithi167.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-11T08:06:02.965 INFO:teuthology.orchestra.run.smithi167.stdout: inet 127.0.0.1/8 scope host lo 2024-05-11T08:06:02.965 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2024-05-11T08:06:02.965 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 ::1/128 scope host 2024-05-11T08:06:02.965 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2024-05-11T08:06:02.965 INFO:teuthology.orchestra.run.smithi167.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-11T08:06:02.965 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:d9:35:e6 brd ff:ff:ff:ff:ff:ff 2024-05-11T08:06:02.965 INFO:teuthology.orchestra.run.smithi167.stdout: altname enp7s0f0 2024-05-11T08:06:02.965 INFO:teuthology.orchestra.run.smithi167.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-11T08:06:02.965 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:d9:35:e7 brd ff:ff:ff:ff:ff:ff 2024-05-11T08:06:02.965 INFO:teuthology.orchestra.run.smithi167.stdout: altname enp7s0f1 2024-05-11T08:06:02.965 INFO:teuthology.orchestra.run.smithi167.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-11T08:06:02.965 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:88:73:08 brd ff:ff:ff:ff:ff:ff 2024-05-11T08:06:02.965 INFO:teuthology.orchestra.run.smithi167.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-11T08:06:02.965 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:88:73:09 brd ff:ff:ff:ff:ff:ff 2024-05-11T08:06:02.965 INFO:teuthology.orchestra.run.smithi167.stdout: inet 172.21.15.167/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-05-11T08:06:02.966 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft 42720sec preferred_lft 42720sec 2024-05-11T08:06:02.966 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::ec4:7aff:fe88:7309/64 scope link 2024-05-11T08:06:02.966 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2024-05-11T08:06:02.966 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-11T08:06:02.966 DEBUG:teuthology.orchestra.run.smithi167:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-11T08:06:02.966 DEBUG:teuthology.orchestra.run.smithi167:> set -e 2024-05-11T08:06:02.967 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip link add name ceph-brx type bridge 2024-05-11T08:06:02.967 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip addr flush dev ceph-brx 2024-05-11T08:06:02.967 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip link set ceph-brx up 2024-05-11T08:06:02.967 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-11T08:06:02.967 DEBUG:teuthology.orchestra.run.smithi167:> ') 2024-05-11T08:06:03.054 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:03 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-11T08:06:03.149 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T08:06:03.156 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T08:06:03.156 DEBUG:teuthology.orchestra.run.smithi167:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-11T08:06:03.181 INFO:teuthology.orchestra.run.smithi167.stdout:1 2024-05-11T08:06:03.183 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T08:06:03.183 DEBUG:teuthology.orchestra.run.smithi167:> route 2024-05-11T08:06:03.249 INFO:teuthology.orchestra.run.smithi167.stdout:Kernel IP routing table 2024-05-11T08:06:03.249 INFO:teuthology.orchestra.run.smithi167.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-11T08:06:03.249 INFO:teuthology.orchestra.run.smithi167.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-11T08:06:03.249 INFO:teuthology.orchestra.run.smithi167.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-11T08:06:03.249 INFO:teuthology.orchestra.run.smithi167.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-11T08:06:03.250 DEBUG:teuthology.orchestra.run.smithi167:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-11T08:06:03.250 DEBUG:teuthology.orchestra.run.smithi167:> set -e 2024-05-11T08:06:03.250 DEBUG:teuthology.orchestra.run.smithi167:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-11T08:06:03.250 DEBUG:teuthology.orchestra.run.smithi167:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-11T08:06:03.250 DEBUG:teuthology.orchestra.run.smithi167:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-11T08:06:03.250 DEBUG:teuthology.orchestra.run.smithi167:> ') 2024-05-11T08:06:03.339 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:03 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-11T08:06:03.550 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T08:06:03.556 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T08:06:03.557 DEBUG:teuthology.orchestra.run.smithi167:> ip netns list 2024-05-11T08:06:03.573 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T08:06:03.573 DEBUG:teuthology.orchestra.run.smithi167:> ip netns list-id 2024-05-11T08:06:03.629 DEBUG:teuthology.orchestra.run.smithi167:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-11T08:06:03.630 DEBUG:teuthology.orchestra.run.smithi167:> set -e 2024-05-11T08:06:03.630 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-11T08:06:03.630 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-05-11T08:06:03.630 DEBUG:teuthology.orchestra.run.smithi167:> ') 2024-05-11T08:06:03.718 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:03 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-11T08:06:03.741 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T08:06:03.748 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-05-11T08:06:03.748 DEBUG:teuthology.orchestra.run.smithi167:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-11T08:06:03.748 DEBUG:teuthology.orchestra.run.smithi167:> set -e 2024-05-11T08:06:03.748 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-05-11T08:06:03.748 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-11T08:06:03.748 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-05-11T08:06:03.748 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-05-11T08:06:03.748 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-05-11T08:06:03.749 DEBUG:teuthology.orchestra.run.smithi167:> ') 2024-05-11T08:06:03.798 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:03 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-11T08:06:03.867 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T08:06:03.873 DEBUG:teuthology.orchestra.run.smithi167:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-11T08:06:03.873 DEBUG:teuthology.orchestra.run.smithi167:> set -e 2024-05-11T08:06:03.873 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip link set brx.0 up 2024-05-11T08:06:03.873 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip link set dev brx.0 master ceph-brx 2024-05-11T08:06:03.873 DEBUG:teuthology.orchestra.run.smithi167:> ') 2024-05-11T08:06:03.923 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:03 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-11T08:06:03.945 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T08:06:03.951 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-05-11T08:06:03.952 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-11T08:06:03.952 DEBUG:teuthology.orchestra.run.smithi167:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-05-11T08:06:03.967 INFO:teuthology.orchestra.run.smithi167.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-05-11T08:06:03.967 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-11T08:06:03.967 DEBUG:teuthology.orchestra.run.smithi167:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-05-11T08:06:04.023 DEBUG:teuthology.orchestra.run.smithi167:> sudo modprobe fuse 2024-05-11T08:06:04.089 DEBUG:teuthology.orchestra.run.smithi167:> cat /proc/self/mounts | awk '{print $2}' 2024-05-11T08:06:04.146 INFO:teuthology.orchestra.run.smithi167.stdout:/proc 2024-05-11T08:06:04.146 INFO:teuthology.orchestra.run.smithi167.stdout:/sys 2024-05-11T08:06:04.147 INFO:teuthology.orchestra.run.smithi167.stdout:/dev 2024-05-11T08:06:04.147 INFO:teuthology.orchestra.run.smithi167.stdout:/sys/kernel/security 2024-05-11T08:06:04.147 INFO:teuthology.orchestra.run.smithi167.stdout:/dev/shm 2024-05-11T08:06:04.147 INFO:teuthology.orchestra.run.smithi167.stdout:/dev/pts 2024-05-11T08:06:04.147 INFO:teuthology.orchestra.run.smithi167.stdout:/run 2024-05-11T08:06:04.147 INFO:teuthology.orchestra.run.smithi167.stdout:/sys/fs/cgroup 2024-05-11T08:06:04.147 INFO:teuthology.orchestra.run.smithi167.stdout:/sys/fs/pstore 2024-05-11T08:06:04.147 INFO:teuthology.orchestra.run.smithi167.stdout:/sys/fs/bpf 2024-05-11T08:06:04.147 INFO:teuthology.orchestra.run.smithi167.stdout:/ 2024-05-11T08:06:04.147 INFO:teuthology.orchestra.run.smithi167.stdout:/sys/fs/selinux 2024-05-11T08:06:04.147 INFO:teuthology.orchestra.run.smithi167.stdout:/proc/sys/fs/binfmt_misc 2024-05-11T08:06:04.147 INFO:teuthology.orchestra.run.smithi167.stdout:/dev/hugepages 2024-05-11T08:06:04.147 INFO:teuthology.orchestra.run.smithi167.stdout:/dev/mqueue 2024-05-11T08:06:04.148 INFO:teuthology.orchestra.run.smithi167.stdout:/sys/kernel/debug 2024-05-11T08:06:04.148 INFO:teuthology.orchestra.run.smithi167.stdout:/sys/kernel/tracing 2024-05-11T08:06:04.148 INFO:teuthology.orchestra.run.smithi167.stdout:/sys/fs/fuse/connections 2024-05-11T08:06:04.148 INFO:teuthology.orchestra.run.smithi167.stdout:/sys/kernel/config 2024-05-11T08:06:04.148 INFO:teuthology.orchestra.run.smithi167.stdout:/run/credentials/systemd-sysctl.service 2024-05-11T08:06:04.148 INFO:teuthology.orchestra.run.smithi167.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-11T08:06:04.148 INFO:teuthology.orchestra.run.smithi167.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-11T08:06:04.148 INFO:teuthology.orchestra.run.smithi167.stdout:/var/lib/nfs/rpc_pipefs 2024-05-11T08:06:04.148 INFO:teuthology.orchestra.run.smithi167.stdout:/run/user/1000 2024-05-11T08:06:04.148 INFO:teuthology.orchestra.run.smithi167.stdout:/var/lib/ceph 2024-05-11T08:06:04.148 INFO:teuthology.orchestra.run.smithi167.stdout:/var/lib/ceph/osd/ceph-0 2024-05-11T08:06:04.148 INFO:teuthology.orchestra.run.smithi167.stdout:/var/lib/ceph/osd/ceph-1 2024-05-11T08:06:04.148 INFO:teuthology.orchestra.run.smithi167.stdout:/var/lib/ceph/osd/ceph-2 2024-05-11T08:06:04.148 INFO:teuthology.orchestra.run.smithi167.stdout:/var/lib/ceph/osd/ceph-3 2024-05-11T08:06:04.148 INFO:teuthology.orchestra.run.smithi167.stdout:/run/netns 2024-05-11T08:06:04.149 INFO:teuthology.orchestra.run.smithi167.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-11T08:06:04.149 INFO:teuthology.orchestra.run.smithi167.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-11T08:06:04.149 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T08:06:04.149 DEBUG:teuthology.orchestra.run.smithi167:> ls /sys/fs/fuse/connections 2024-05-11T08:06:04.202 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-11T08:06:04.202 DEBUG:teuthology.orchestra.run.smithi167:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-05-11T08:06:04.245 DEBUG:teuthology.orchestra.run.smithi167:> sudo modprobe fuse 2024-05-11T08:06:04.269 DEBUG:teuthology.orchestra.run.smithi167:> cat /proc/self/mounts | awk '{print $2}' 2024-05-11T08:06:04.325 INFO:teuthology.orchestra.run.smithi167.stdout:/proc 2024-05-11T08:06:04.325 INFO:teuthology.orchestra.run.smithi167.stdout:/sys 2024-05-11T08:06:04.325 INFO:teuthology.orchestra.run.smithi167.stdout:/dev 2024-05-11T08:06:04.325 INFO:teuthology.orchestra.run.smithi167.stdout:/sys/kernel/security 2024-05-11T08:06:04.325 INFO:teuthology.orchestra.run.smithi167.stdout:/dev/shm 2024-05-11T08:06:04.325 INFO:teuthology.orchestra.run.smithi167.stdout:/dev/pts 2024-05-11T08:06:04.325 INFO:teuthology.orchestra.run.smithi167.stdout:/run 2024-05-11T08:06:04.325 INFO:teuthology.orchestra.run.smithi167.stdout:/sys/fs/cgroup 2024-05-11T08:06:04.325 INFO:teuthology.orchestra.run.smithi167.stdout:/sys/fs/pstore 2024-05-11T08:06:04.326 INFO:teuthology.orchestra.run.smithi167.stdout:/sys/fs/bpf 2024-05-11T08:06:04.326 INFO:teuthology.orchestra.run.smithi167.stdout:/ 2024-05-11T08:06:04.326 INFO:teuthology.orchestra.run.smithi167.stdout:/sys/fs/selinux 2024-05-11T08:06:04.326 INFO:teuthology.orchestra.run.smithi167.stdout:/proc/sys/fs/binfmt_misc 2024-05-11T08:06:04.326 INFO:teuthology.orchestra.run.smithi167.stdout:/dev/hugepages 2024-05-11T08:06:04.326 INFO:teuthology.orchestra.run.smithi167.stdout:/dev/mqueue 2024-05-11T08:06:04.326 INFO:teuthology.orchestra.run.smithi167.stdout:/sys/kernel/debug 2024-05-11T08:06:04.326 INFO:teuthology.orchestra.run.smithi167.stdout:/sys/kernel/tracing 2024-05-11T08:06:04.326 INFO:teuthology.orchestra.run.smithi167.stdout:/sys/fs/fuse/connections 2024-05-11T08:06:04.326 INFO:teuthology.orchestra.run.smithi167.stdout:/sys/kernel/config 2024-05-11T08:06:04.326 INFO:teuthology.orchestra.run.smithi167.stdout:/run/credentials/systemd-sysctl.service 2024-05-11T08:06:04.326 INFO:teuthology.orchestra.run.smithi167.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-11T08:06:04.326 INFO:teuthology.orchestra.run.smithi167.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-11T08:06:04.327 INFO:teuthology.orchestra.run.smithi167.stdout:/var/lib/nfs/rpc_pipefs 2024-05-11T08:06:04.327 INFO:teuthology.orchestra.run.smithi167.stdout:/run/user/1000 2024-05-11T08:06:04.327 INFO:teuthology.orchestra.run.smithi167.stdout:/var/lib/ceph 2024-05-11T08:06:04.327 INFO:teuthology.orchestra.run.smithi167.stdout:/var/lib/ceph/osd/ceph-0 2024-05-11T08:06:04.327 INFO:teuthology.orchestra.run.smithi167.stdout:/var/lib/ceph/osd/ceph-1 2024-05-11T08:06:04.327 INFO:teuthology.orchestra.run.smithi167.stdout:/var/lib/ceph/osd/ceph-2 2024-05-11T08:06:04.327 INFO:teuthology.orchestra.run.smithi167.stdout:/var/lib/ceph/osd/ceph-3 2024-05-11T08:06:04.327 INFO:teuthology.orchestra.run.smithi167.stdout:/run/netns 2024-05-11T08:06:04.327 INFO:teuthology.orchestra.run.smithi167.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-11T08:06:04.327 INFO:teuthology.orchestra.run.smithi167.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-11T08:06:04.327 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T08:06:04.327 DEBUG:teuthology.orchestra.run.smithi167:> ls /sys/fs/fuse/connections 2024-05-11T08:06:04.333 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi167.stderr:2024-05-11T08:06:04.333+0000 7f270872d480 -1 init, newargv = 0x5593cc5d15c0 newargc=15 2024-05-11T08:06:04.333 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi167.stderr:ceph-fuse[42259]: starting ceph client 2024-05-11T08:06:04.339 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi167.stderr:ceph-fuse[42259]: starting fuse 2024-05-11T08:06:04.348 INFO:teuthology.orchestra.run.smithi167.stdout:41 2024-05-11T08:06:04.349 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2024-05-11T08:06:04.349 DEBUG:teuthology.orchestra.run.smithi167:> sudo stdin-killer -- python3 -c ' 2024-05-11T08:06:04.349 DEBUG:teuthology.orchestra.run.smithi167:> import glob 2024-05-11T08:06:04.349 DEBUG:teuthology.orchestra.run.smithi167:> import re 2024-05-11T08:06:04.349 DEBUG:teuthology.orchestra.run.smithi167:> import os 2024-05-11T08:06:04.349 DEBUG:teuthology.orchestra.run.smithi167:> import subprocess 2024-05-11T08:06:04.349 DEBUG:teuthology.orchestra.run.smithi167:> 2024-05-11T08:06:04.349 DEBUG:teuthology.orchestra.run.smithi167:> def _find_admin_socket(client_name): 2024-05-11T08:06:04.349 DEBUG:teuthology.orchestra.run.smithi167:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-05-11T08:06:04.349 DEBUG:teuthology.orchestra.run.smithi167:> files = glob.glob(asok_path) 2024-05-11T08:06:04.349 DEBUG:teuthology.orchestra.run.smithi167:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-05-11T08:06:04.350 DEBUG:teuthology.orchestra.run.smithi167:> 2024-05-11T08:06:04.350 DEBUG:teuthology.orchestra.run.smithi167:> # Given a non-glob path, it better be there 2024-05-11T08:06:04.350 DEBUG:teuthology.orchestra.run.smithi167:> if "*" not in asok_path: 2024-05-11T08:06:04.350 DEBUG:teuthology.orchestra.run.smithi167:> assert(len(files) == 1) 2024-05-11T08:06:04.350 DEBUG:teuthology.orchestra.run.smithi167:> return files[0] 2024-05-11T08:06:04.350 DEBUG:teuthology.orchestra.run.smithi167:> 2024-05-11T08:06:04.350 DEBUG:teuthology.orchestra.run.smithi167:> for f in files: 2024-05-11T08:06:04.350 DEBUG:teuthology.orchestra.run.smithi167:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-11T08:06:04.350 DEBUG:teuthology.orchestra.run.smithi167:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-11T08:06:04.350 DEBUG:teuthology.orchestra.run.smithi167:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-11T08:06:04.350 DEBUG:teuthology.orchestra.run.smithi167:> contents = proc_f.read() 2024-05-11T08:06:04.350 DEBUG:teuthology.orchestra.run.smithi167:> if mountpoint in contents: 2024-05-11T08:06:04.350 DEBUG:teuthology.orchestra.run.smithi167:> return f 2024-05-11T08:06:04.350 DEBUG:teuthology.orchestra.run.smithi167:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-11T08:06:04.350 DEBUG:teuthology.orchestra.run.smithi167:> 2024-05-11T08:06:04.351 DEBUG:teuthology.orchestra.run.smithi167:> print(_find_admin_socket("client.admin")) 2024-05-11T08:06:04.351 DEBUG:teuthology.orchestra.run.smithi167:> ' 2024-05-11T08:06:04.467 INFO:teuthology.orchestra.run.smithi167.stdout:/var/run/ceph/ceph-client.admin.42259.asok 2024-05-11T08:06:04.470 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T08:06:04.478 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.42259.asok 2024-05-11T08:06:04.478 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T08:06:04.478 DEBUG:teuthology.orchestra.run.smithi167:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.42259.asok status 2024-05-11T08:06:04.612 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-05-11T08:06:04.612 INFO:teuthology.orchestra.run.smithi167.stdout: "metadata": { 2024-05-11T08:06:04.613 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_sha1": "308506802b0bdea4df7050cfc1fcbdc04fea6a6e", 2024-05-11T08:06:04.613 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_version": "ceph version 19.0.0-3601-g30850680 (308506802b0bdea4df7050cfc1fcbdc04fea6a6e) squid (dev)", 2024-05-11T08:06:04.613 INFO:teuthology.orchestra.run.smithi167.stdout: "entity_id": "admin", 2024-05-11T08:06:04.613 INFO:teuthology.orchestra.run.smithi167.stdout: "hostname": "smithi167", 2024-05-11T08:06:04.613 INFO:teuthology.orchestra.run.smithi167.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-05-11T08:06:04.613 INFO:teuthology.orchestra.run.smithi167.stdout: "pid": "42259", 2024-05-11T08:06:04.613 INFO:teuthology.orchestra.run.smithi167.stdout: "root": "/" 2024-05-11T08:06:04.613 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T08:06:04.613 INFO:teuthology.orchestra.run.smithi167.stdout: "dentry_count": 0, 2024-05-11T08:06:04.613 INFO:teuthology.orchestra.run.smithi167.stdout: "dentry_pinned_count": 0, 2024-05-11T08:06:04.613 INFO:teuthology.orchestra.run.smithi167.stdout: "id": 4443, 2024-05-11T08:06:04.613 INFO:teuthology.orchestra.run.smithi167.stdout: "inst": { 2024-05-11T08:06:04.613 INFO:teuthology.orchestra.run.smithi167.stdout: "name": { 2024-05-11T08:06:04.613 INFO:teuthology.orchestra.run.smithi167.stdout: "type": "client", 2024-05-11T08:06:04.614 INFO:teuthology.orchestra.run.smithi167.stdout: "num": 4443 2024-05-11T08:06:04.614 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T08:06:04.614 INFO:teuthology.orchestra.run.smithi167.stdout: "addr": { 2024-05-11T08:06:04.614 INFO:teuthology.orchestra.run.smithi167.stdout: "type": "v1", 2024-05-11T08:06:04.614 INFO:teuthology.orchestra.run.smithi167.stdout: "addr": "192.168.0.1:0", 2024-05-11T08:06:04.614 INFO:teuthology.orchestra.run.smithi167.stdout: "nonce": 1702641499 2024-05-11T08:06:04.614 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T08:06:04.614 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T08:06:04.614 INFO:teuthology.orchestra.run.smithi167.stdout: "addr": { 2024-05-11T08:06:04.614 INFO:teuthology.orchestra.run.smithi167.stdout: "type": "v1", 2024-05-11T08:06:04.614 INFO:teuthology.orchestra.run.smithi167.stdout: "addr": "192.168.0.1:0", 2024-05-11T08:06:04.614 INFO:teuthology.orchestra.run.smithi167.stdout: "nonce": 1702641499 2024-05-11T08:06:04.614 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T08:06:04.614 INFO:teuthology.orchestra.run.smithi167.stdout: "inst_str": "client.4443 192.168.0.1:0/1702641499", 2024-05-11T08:06:04.615 INFO:teuthology.orchestra.run.smithi167.stdout: "addr_str": "192.168.0.1:0/1702641499", 2024-05-11T08:06:04.615 INFO:teuthology.orchestra.run.smithi167.stdout: "inode_count": 1, 2024-05-11T08:06:04.615 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_epoch": 7, 2024-05-11T08:06:04.615 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_epoch": 27, 2024-05-11T08:06:04.615 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_epoch_barrier": 0, 2024-05-11T08:06:04.615 INFO:teuthology.orchestra.run.smithi167.stdout: "blocklisted": false, 2024-05-11T08:06:04.615 INFO:teuthology.orchestra.run.smithi167.stdout: "fs_name": "cephfs" 2024-05-11T08:06:04.615 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-05-11T08:06:04.621 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T08:06:04.621 DEBUG:teuthology.orchestra.run.smithi167:> stat --file-system '--printf=%T 2024-05-11T08:06:04.621 DEBUG:teuthology.orchestra.run.smithi167:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-05-11T08:06:04.637 INFO:teuthology.orchestra.run.smithi167.stdout:fuseblk 2024-05-11T08:06:04.637 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-05-11T08:06:04.638 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T08:06:04.638 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-05-11T08:06:04.704 DEBUG:teuthology.orchestra.run.smithi167:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-05-11T08:06:04.795 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:04 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-11T08:06:04.815 INFO:teuthology.orchestra.run.smithi167.stdout:# file: . 2024-05-11T08:06:04.815 INFO:teuthology.orchestra.run.smithi167.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-05-11T08:06:04.815 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T08:06:04.816 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T08:06:04.821 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-05-11T08:06:04.821 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T08:06:04.837 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-11T08:06:04.977 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:04.977+0000 7f9bf0bda640 1 Processor -- start 2024-05-11T08:06:04.977 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:04.977+0000 7f9bf0bda640 1 -- start start 2024-05-11T08:06:04.979 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:04.978+0000 7f9bf0bda640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f9bec0b5e90 0x7f9bec15f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:04.979 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:04.978+0000 7f9bf0bda640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f9bec15fc80 0x7f9bec0cd610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:04.979 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:04.978+0000 7f9bf0bda640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9bec0d2410 con 0x7f9bec15f360 2024-05-11T08:06:04.979 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:04.978+0000 7f9bf0bda640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f9bec0d26f0 con 0x7f9bec0b5e90 2024-05-11T08:06:04.979 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:04.978+0000 7f9bf0bda640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f9bec0d29d0 con 0x7f9bec15fc80 2024-05-11T08:06:04.979 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:04.978+0000 7f9be3fff640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f9bec0b5e90 0x7f9bec15f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:04.979 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:04.978+0000 7f9be3fff640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f9bec0b5e90 0x7f9bec15f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:45646/0 (socket says 172.21.15.167:45646) 2024-05-11T08:06:04.979 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:04.978+0000 7f9bea59c640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f9bec15f360 0x7f9bec0cfdc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:36326/0 (socket says 172.21.15.167:36326) 2024-05-11T08:06:04.979 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:04.978+0000 7f9be3fff640 1 -- 172.21.15.167:0/326543098 learned_addr learned my addr 172.21.15.167:0/326543098 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:04.980 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:04.978+0000 7f9bead9d640 1 --2- 172.21.15.167:0/326543098 >> v2:172.21.15.204:3300/0 conn(0x7f9bec15fc80 0x7f9bec0cd610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:04.980 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:04.978+0000 7f9be3fff640 1 -- 172.21.15.167:0/326543098 >> v2:172.21.15.204:3300/0 conn(0x7f9bec15fc80 msgr2=0x7f9bec0cd610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:04.980 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:04.978+0000 7f9be3fff640 1 --2- 172.21.15.167:0/326543098 >> v2:172.21.15.204:3300/0 conn(0x7f9bec15fc80 0x7f9bec0cd610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:04.980 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:04.978+0000 7f9be3fff640 1 -- 172.21.15.167:0/326543098 >> v1:172.21.15.167:6789/0 conn(0x7f9bec15f360 legacy=0x7f9bec0cfdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:04.980 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:04.978+0000 7f9be3fff640 1 -- 172.21.15.167:0/326543098 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bec0b7e20 con 0x7f9bec0b5e90 2024-05-11T08:06:04.980 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:04.978+0000 7f9be9d9b640 1 -- 172.21.15.167:0/326543098 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4249240964 0 0) 0x7f9bec0d2410 con 0x7f9bec15f360 2024-05-11T08:06:04.980 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:04.978+0000 7f9bead9d640 1 --2- 172.21.15.167:0/326543098 >> v2:172.21.15.204:3300/0 conn(0x7f9bec15fc80 0x7f9bec0cd610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T08:06:04.980 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:04.979+0000 7f9be3fff640 1 --2- 172.21.15.167:0/326543098 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9bec0b5e90 0x7f9bec15f740 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9bec162fe0 tx=0x7f9bd402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9038f72ed6f9b24 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:04.980 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:04.979+0000 7f9be9d9b640 1 -- 172.21.15.167:0/326543098 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bd4037070 con 0x7f9bec0b5e90 2024-05-11T08:06:04.980 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:04.979+0000 7f9be9d9b640 1 -- 172.21.15.167:0/326543098 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9bd4032440 con 0x7f9bec0b5e90 2024-05-11T08:06:04.980 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:04.979+0000 7f9be9d9b640 1 -- 172.21.15.167:0/326543098 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bd4032810 con 0x7f9bec0b5e90 2024-05-11T08:06:04.983 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:04.980+0000 7f9bf0bda640 1 -- 172.21.15.167:0/326543098 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9bec0b5e90 msgr2=0x7f9bec15f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:04.983 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:04.980+0000 7f9bf0bda640 1 --2- 172.21.15.167:0/326543098 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9bec0b5e90 0x7f9bec15f740 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9bec162fe0 tx=0x7f9bd402a1e0 comp rx=0 tx=0).stop 2024-05-11T08:06:04.983 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:04.980+0000 7f9bf0bda640 1 -- 172.21.15.167:0/326543098 shutdown_connections 2024-05-11T08:06:04.983 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:04.980+0000 7f9bf0bda640 1 --2- 172.21.15.167:0/326543098 >> v2:172.21.15.204:3300/0 conn(0x7f9bec15fc80 0x7f9bec0cd610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:04.983 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:04.980+0000 7f9bf0bda640 1 --2- 172.21.15.167:0/326543098 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9bec0b5e90 0x7f9bec15f740 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:04.983 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:04.980+0000 7f9bf0bda640 1 -- 172.21.15.167:0/326543098 >> 172.21.15.167:0/326543098 conn(0x7f9bec072730 msgr2=0x7f9bec0d2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:04.983 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:04.980+0000 7f9bf0bda640 1 -- 172.21.15.167:0/326543098 shutdown_connections 2024-05-11T08:06:04.983 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:04.980+0000 7f9bf0bda640 1 -- 172.21.15.167:0/326543098 wait complete. 2024-05-11T08:06:04.984 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:04.981+0000 7f9bf0bda640 1 Processor -- start 2024-05-11T08:06:04.984 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:04.981+0000 7f9bf0bda640 1 -- start start 2024-05-11T08:06:04.984 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:04.982+0000 7f9bf0bda640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9bec0b5e90 0x7f9bec06f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:04.984 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:04.982+0000 7f9bf0bda640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9bec15f360 0x7f9bec06ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:04.984 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:04.982+0000 7f9bf0bda640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9bec15fc80 0x7f9bec15d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:04.984 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:04.982+0000 7f9bf0bda640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9bec0cdcd0 con 0x7f9bec15fc80 2024-05-11T08:06:04.984 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:04.982+0000 7f9bf0bda640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f9bec0cdb50 con 0x7f9bec15f360 2024-05-11T08:06:04.984 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:04.982+0000 7f9bf0bda640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f9bec0cde50 con 0x7f9bec0b5e90 2024-05-11T08:06:04.984 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:04.982+0000 7f9bead9d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9bec15fc80 0x7f9bec15d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:04.984 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:04.982+0000 7f9bead9d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9bec15fc80 0x7f9bec15d1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:45660/0 (socket says 172.21.15.167:45660) 2024-05-11T08:06:04.984 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:04.982+0000 7f9bead9d640 1 -- 172.21.15.167:0/469871773 learned_addr learned my addr 172.21.15.167:0/469871773 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:04.984 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:04.982+0000 7f9be3fff640 1 --2- 172.21.15.167:0/469871773 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9bec15f360 0x7f9bec06ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:04.984 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:04.982+0000 7f9bea59c640 1 --2- 172.21.15.167:0/469871773 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9bec0b5e90 0x7f9bec06f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:04.984 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:04.982+0000 7f9bead9d640 1 -- 172.21.15.167:0/469871773 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9bec0b5e90 msgr2=0x7f9bec06f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:04.984 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:04.982+0000 7f9bead9d640 1 --2- 172.21.15.167:0/469871773 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9bec0b5e90 0x7f9bec06f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:04.985 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:04.982+0000 7f9bead9d640 1 -- 172.21.15.167:0/469871773 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9bec15f360 msgr2=0x7f9bec06ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:04.985 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:04.982+0000 7f9bead9d640 1 --2- 172.21.15.167:0/469871773 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9bec15f360 0x7f9bec06ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:04.985 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:04.982+0000 7f9bead9d640 1 -- 172.21.15.167:0/469871773 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bec06e740 con 0x7f9bec15fc80 2024-05-11T08:06:04.985 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:04.982+0000 7f9bead9d640 1 --2- 172.21.15.167:0/469871773 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9bec15fc80 0x7f9bec15d1c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f9be400cef0 tx=0x7f9be400a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:04.985 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:04.983+0000 7f9be2ffd640 1 -- 172.21.15.167:0/469871773 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9be401a070 con 0x7f9bec15fc80 2024-05-11T08:06:04.985 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:04.983+0000 7f9be2ffd640 1 -- 172.21.15.167:0/469871773 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9be400d860 con 0x7f9bec15fc80 2024-05-11T08:06:04.985 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:04.983+0000 7f9bf0bda640 1 -- 172.21.15.167:0/469871773 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bec06ea20 con 0x7f9bec15fc80 2024-05-11T08:06:04.985 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:04.983+0000 7f9be2ffd640 1 -- 172.21.15.167:0/469871773 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9be400dc50 con 0x7f9bec15fc80 2024-05-11T08:06:04.985 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:04.983+0000 7f9bf0bda640 1 -- 172.21.15.167:0/469871773 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bec06efe0 con 0x7f9bec15fc80 2024-05-11T08:06:04.986 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:04.984+0000 7f9bf0bda640 1 -- 172.21.15.167:0/469871773 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9bec069590 con 0x7f9bec15fc80 2024-05-11T08:06:04.986 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:04.984+0000 7f9be2ffd640 1 -- 172.21.15.167:0/469871773 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f9be401d070 con 0x7f9bec15fc80 2024-05-11T08:06:04.986 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:04.985+0000 7f9be2ffd640 1 --2- 172.21.15.167:0/469871773 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f9bcc0762d0 0x7f9bcc078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:04.986 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:04.985+0000 7f9be2ffd640 1 -- 172.21.15.167:0/469871773 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f9be40980f0 con 0x7f9bec15fc80 2024-05-11T08:06:04.986 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:04.985+0000 7f9bea59c640 1 --2- 172.21.15.167:0/469871773 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f9bcc0762d0 0x7f9bcc078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:04.986 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:04.985+0000 7f9bea59c640 1 --2- 172.21.15.167:0/469871773 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f9bcc0762d0 0x7f9bcc078790 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f9bdc007fd0 tx=0x7f9bdc007de0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:04.989 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:04.989+0000 7f9be2ffd640 1 -- 172.21.15.167:0/469871773 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f9be4061180 con 0x7f9bec15fc80 2024-05-11T08:06:05.182 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:05.182+0000 7f9bf0bda640 1 -- 172.21.15.167:0/469871773 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f9bec15bb50 con 0x7f9bec15fc80 2024-05-11T08:06:05.182 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:05.182+0000 7f9be2ffd640 1 -- 172.21.15.167:0/469871773 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) ==== 80+0+0 (secure 0 0 0) 0x7f9be4065480 con 0x7f9bec15fc80 2024-05-11T08:06:05.182 INFO:teuthology.orchestra.run.smithi167.stderr:listed 0 entries 2024-05-11T08:06:05.185 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:05.184+0000 7f9bf0bda640 1 -- 172.21.15.167:0/469871773 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f9bcc0762d0 msgr2=0x7f9bcc078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:05.185 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:05.184+0000 7f9bf0bda640 1 --2- 172.21.15.167:0/469871773 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f9bcc0762d0 0x7f9bcc078790 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f9bdc007fd0 tx=0x7f9bdc007de0 comp rx=0 tx=0).stop 2024-05-11T08:06:05.185 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:05.184+0000 7f9bf0bda640 1 -- 172.21.15.167:0/469871773 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9bec15fc80 msgr2=0x7f9bec15d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:05.185 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:05.184+0000 7f9bf0bda640 1 --2- 172.21.15.167:0/469871773 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9bec15fc80 0x7f9bec15d1c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f9be400cef0 tx=0x7f9be400a710 comp rx=0 tx=0).stop 2024-05-11T08:06:05.185 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:05.184+0000 7f9bf0bda640 1 -- 172.21.15.167:0/469871773 shutdown_connections 2024-05-11T08:06:05.185 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:05.184+0000 7f9bf0bda640 1 --2- 172.21.15.167:0/469871773 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f9bcc0762d0 0x7f9bcc078790 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:05.185 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:05.184+0000 7f9bf0bda640 1 --2- 172.21.15.167:0/469871773 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9bec15fc80 0x7f9bec15d1c0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:05.185 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:05.184+0000 7f9bf0bda640 1 --2- 172.21.15.167:0/469871773 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9bec15f360 0x7f9bec06ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:05.185 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:05.184+0000 7f9bf0bda640 1 --2- 172.21.15.167:0/469871773 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9bec0b5e90 0x7f9bec06f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:05.185 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:05.184+0000 7f9bf0bda640 1 -- 172.21.15.167:0/469871773 >> 172.21.15.167:0/469871773 conn(0x7f9bec072730 msgr2=0x7f9bec15bd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:05.185 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:05.184+0000 7f9bf0bda640 1 -- 172.21.15.167:0/469871773 shutdown_connections 2024-05-11T08:06:05.185 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:05.184+0000 7f9bf0bda640 1 -- 172.21.15.167:0/469871773 wait complete. 2024-05-11T08:06:05.195 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-05-11T08:06:05.195 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T08:06:05.211 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-11T08:06:05.351 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:05.351+0000 7f987821c640 1 Processor -- start 2024-05-11T08:06:05.352 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:05.352+0000 7f987821c640 1 -- start start 2024-05-11T08:06:05.353 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:05.352+0000 7f987821c640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f987015f220 0x7f98700d1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:05.353 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:05.352+0000 7f987821c640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f98700b74c0 0x7f98700b7900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:05.353 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:05.352+0000 7f987821c640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98700d1a50 con 0x7f98700cd580 2024-05-11T08:06:05.353 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:05.352+0000 7f987821c640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f987015f600 con 0x7f98700b74c0 2024-05-11T08:06:05.353 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:05.352+0000 7f987821c640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f98700cf450 con 0x7f987015f220 2024-05-11T08:06:05.353 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:05.352+0000 7f98757b7640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f98700b74c0 0x7f98700b7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:05.353 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:05.352+0000 7f98767b9640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f98700cd580 0x7f98700cf8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:36342/0 (socket says 172.21.15.167:36342) 2024-05-11T08:06:05.353 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:05.352+0000 7f98767b9640 1 -- 172.21.15.167:0/885292684 learned_addr learned my addr 172.21.15.167:0/885292684 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T08:06:05.353 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:05.353+0000 7f9875fb8640 1 --2- 172.21.15.167:0/885292684 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f987015f220 0x7f98700d1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:05.353 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:05.353+0000 7f98757b7640 1 -- 172.21.15.167:0/885292684 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f987015f220 msgr2=0x7f98700d1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:05.354 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:05.353+0000 7f98757b7640 1 --2- 172.21.15.167:0/885292684 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f987015f220 0x7f98700d1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:05.354 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:05.353+0000 7f98757b7640 1 -- 172.21.15.167:0/885292684 >> v1:172.21.15.167:6789/0 conn(0x7f98700cd580 legacy=0x7f98700cf8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:05.354 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:05.353+0000 7f98757b7640 1 -- 172.21.15.167:0/885292684 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98700d27c0 con 0x7f98700b74c0 2024-05-11T08:06:05.354 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:05.353+0000 7f9874fb6640 1 -- 172.21.15.167:0/885292684 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2490587554 0 0) 0x7f9868001490 con 0x7f98700cd580 2024-05-11T08:06:05.355 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:05.353+0000 7f98757b7640 1 --2- 172.21.15.167:0/885292684 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f98700b74c0 0x7f98700b7900 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f98600097a0 tx=0x7f986002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bf1d661b49e37c6 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:05.355 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:05.354+0000 7f9874fb6640 1 -- 172.21.15.167:0/885292684 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9860037070 con 0x7f98700b74c0 2024-05-11T08:06:05.355 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:05.354+0000 7f9874fb6640 1 -- 172.21.15.167:0/885292684 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9860032440 con 0x7f98700b74c0 2024-05-11T08:06:05.355 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:05.354+0000 7f9874fb6640 1 -- 172.21.15.167:0/885292684 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9860032810 con 0x7f98700b74c0 2024-05-11T08:06:05.356 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:05.354+0000 7f987821c640 1 -- 172.21.15.167:0/885292684 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f98700b74c0 msgr2=0x7f98700b7900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:05.356 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:05.354+0000 7f987821c640 1 --2- 172.21.15.167:0/885292684 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f98700b74c0 0x7f98700b7900 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f98600097a0 tx=0x7f986002a180 comp rx=0 tx=0).stop 2024-05-11T08:06:05.356 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:05.354+0000 7f987821c640 1 -- 172.21.15.167:0/885292684 shutdown_connections 2024-05-11T08:06:05.356 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:05.354+0000 7f987821c640 1 --2- 172.21.15.167:0/885292684 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f98700b74c0 0x7f98700b7900 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:05.356 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:05.354+0000 7f987821c640 1 --2- 172.21.15.167:0/885292684 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f987015f220 0x7f98700d1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:05.356 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:05.354+0000 7f987821c640 1 -- 172.21.15.167:0/885292684 >> 172.21.15.167:0/885292684 conn(0x7f98700b4ae0 msgr2=0x7f98700b4ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:05.356 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:05.354+0000 7f987821c640 1 -- 172.21.15.167:0/885292684 shutdown_connections 2024-05-11T08:06:05.356 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:05.354+0000 7f987821c640 1 -- 172.21.15.167:0/885292684 wait complete. 2024-05-11T08:06:05.356 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:05.355+0000 7f987821c640 1 Processor -- start 2024-05-11T08:06:05.356 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:05.355+0000 7f987821c640 1 -- start start 2024-05-11T08:06:05.358 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:05.356+0000 7f987821c640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f98700b74c0 0x7f987015c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:05.358 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:05.356+0000 7f987821c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f98700cd580 0x7f987015db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:05.358 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:05.356+0000 7f987821c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f987015f220 0x7f98701f4a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:05.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:05.356+0000 7f987821c640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f98700ce2e0 con 0x7f98700cd580 2024-05-11T08:06:05.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:05.356+0000 7f987821c640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f98700cf0e0 con 0x7f987015f220 2024-05-11T08:06:05.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:05.356+0000 7f987821c640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f98700d2400 con 0x7f98700b74c0 2024-05-11T08:06:05.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:05.356+0000 7f98757b7640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f98700cd580 0x7f987015db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:05.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:05.356+0000 7f98757b7640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f98700cd580 0x7f987015db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:45680/0 (socket says 172.21.15.167:45680) 2024-05-11T08:06:05.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:05.356+0000 7f98757b7640 1 -- 172.21.15.167:0/3047423598 learned_addr learned my addr 172.21.15.167:0/3047423598 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:05.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:05.356+0000 7f9875fb8640 1 --2- 172.21.15.167:0/3047423598 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f98700b74c0 0x7f987015c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:05.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:05.356+0000 7f98767b9640 1 --2- 172.21.15.167:0/3047423598 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f987015f220 0x7f98701f4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:05.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:05.357+0000 7f98757b7640 1 -- 172.21.15.167:0/3047423598 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f98700b74c0 msgr2=0x7f987015c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:05.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:05.357+0000 7f98757b7640 1 --2- 172.21.15.167:0/3047423598 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f98700b74c0 0x7f987015c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:05.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:05.357+0000 7f98757b7640 1 -- 172.21.15.167:0/3047423598 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f987015f220 msgr2=0x7f98701f4a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:05.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:05.357+0000 7f98757b7640 1 --2- 172.21.15.167:0/3047423598 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f987015f220 0x7f98701f4a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:05.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:05.357+0000 7f98757b7640 1 -- 172.21.15.167:0/3047423598 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98701555f0 con 0x7f98700cd580 2024-05-11T08:06:05.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:05.357+0000 7f98767b9640 1 --2- 172.21.15.167:0/3047423598 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f987015f220 0x7f98701f4a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T08:06:05.360 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:05.357+0000 7f98757b7640 1 --2- 172.21.15.167:0/3047423598 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f98700cd580 0x7f987015db70 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f98600098c0 tx=0x7f98600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:05.360 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:05.357+0000 7f98667fc640 1 -- 172.21.15.167:0/3047423598 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9860037040 con 0x7f98700cd580 2024-05-11T08:06:05.360 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:05.357+0000 7f98667fc640 1 -- 172.21.15.167:0/3047423598 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9860030770 con 0x7f98700cd580 2024-05-11T08:06:05.360 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:05.357+0000 7f98667fc640 1 -- 172.21.15.167:0/3047423598 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9860030b40 con 0x7f98700cd580 2024-05-11T08:06:05.360 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:05.357+0000 7f987821c640 1 -- 172.21.15.167:0/3047423598 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f987015e2e0 con 0x7f98700cd580 2024-05-11T08:06:05.360 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:05.357+0000 7f987821c640 1 -- 172.21.15.167:0/3047423598 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98700ca2e0 con 0x7f98700cd580 2024-05-11T08:06:05.360 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:05.358+0000 7f987821c640 1 -- 172.21.15.167:0/3047423598 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9870069bc0 con 0x7f98700cd580 2024-05-11T08:06:05.360 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:05.359+0000 7f98667fc640 1 -- 172.21.15.167:0/3047423598 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f9860044020 con 0x7f98700cd580 2024-05-11T08:06:05.361 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:05.359+0000 7f98667fc640 1 --2- 172.21.15.167:0/3047423598 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f98540762d0 0x7f9854078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:05.361 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:05.359+0000 7f98667fc640 1 -- 172.21.15.167:0/3047423598 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f98600b7d80 con 0x7f98700cd580 2024-05-11T08:06:05.361 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:05.359+0000 7f9875fb8640 1 --2- 172.21.15.167:0/3047423598 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f98540762d0 0x7f9854078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:05.361 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:05.359+0000 7f9875fb8640 1 --2- 172.21.15.167:0/3047423598 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f98540762d0 0x7f9854078790 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f986c0079a0 tx=0x7f986c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:05.363 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:05.363+0000 7f98667fc640 1 -- 172.21.15.167:0/3047423598 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f98701555f0 con 0x7f98700cd580 2024-05-11T08:06:05.556 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:05.556+0000 7f987821c640 1 -- 172.21.15.167:0/3047423598 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f987015a510 con 0x7f98700cd580 2024-05-11T08:06:05.556 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:05.556+0000 7f98667fc640 1 -- 172.21.15.167:0/3047423598 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) ==== 80+0+0 (secure 0 0 0) 0x7f9860032e20 con 0x7f98700cd580 2024-05-11T08:06:05.557 INFO:teuthology.orchestra.run.smithi167.stderr:listed 0 entries 2024-05-11T08:06:05.559 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:05.558+0000 7f987821c640 1 -- 172.21.15.167:0/3047423598 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f98540762d0 msgr2=0x7f9854078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:05.559 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:05.558+0000 7f987821c640 1 --2- 172.21.15.167:0/3047423598 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f98540762d0 0x7f9854078790 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f986c0079a0 tx=0x7f986c008040 comp rx=0 tx=0).stop 2024-05-11T08:06:05.559 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:05.558+0000 7f987821c640 1 -- 172.21.15.167:0/3047423598 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f98700cd580 msgr2=0x7f987015db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:05.559 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:05.558+0000 7f987821c640 1 --2- 172.21.15.167:0/3047423598 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f98700cd580 0x7f987015db70 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f98600098c0 tx=0x7f98600046c0 comp rx=0 tx=0).stop 2024-05-11T08:06:05.559 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:05.558+0000 7f987821c640 1 -- 172.21.15.167:0/3047423598 shutdown_connections 2024-05-11T08:06:05.559 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:05.558+0000 7f987821c640 1 --2- 172.21.15.167:0/3047423598 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f98540762d0 0x7f9854078790 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:05.559 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:05.558+0000 7f987821c640 1 --2- 172.21.15.167:0/3047423598 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f987015f220 0x7f98701f4a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:05.559 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:05.558+0000 7f987821c640 1 --2- 172.21.15.167:0/3047423598 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f98700cd580 0x7f987015db70 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:05.559 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:05.558+0000 7f987821c640 1 --2- 172.21.15.167:0/3047423598 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f98700b74c0 0x7f987015c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:05.559 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:05.558+0000 7f987821c640 1 -- 172.21.15.167:0/3047423598 >> 172.21.15.167:0/3047423598 conn(0x7f98700b4ae0 msgr2=0x7f98701548c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:05.560 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:05.559+0000 7f987821c640 1 -- 172.21.15.167:0/3047423598 shutdown_connections 2024-05-11T08:06:05.560 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:05.559+0000 7f987821c640 1 -- 172.21.15.167:0/3047423598 wait complete. 2024-05-11T08:06:05.569 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi167.front.sepia.ceph.com... 2024-05-11T08:06:05.570 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T08:06:05.570 DEBUG:teuthology.orchestra.run.smithi167:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-05-11T08:06:05.593 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi167.stderr:ceph-fuse[42259]: fuse finished with error 0 and tester_r 0 2024-05-11T08:06:05.594 INFO:teuthology.orchestra.run:waiting for 300 2024-05-11T08:06:11.696 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi167.front.sepia.ceph.com 2024-05-11T08:06:11.696 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T08:06:11.696 DEBUG:teuthology.orchestra.run.smithi167:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-05-11T08:06:11.727 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-11T08:06:11.877 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:11.877+0000 7f69677f0640 1 Processor -- start 2024-05-11T08:06:11.877 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:11.877+0000 7f69677f0640 1 -- start start 2024-05-11T08:06:11.879 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:11.878+0000 7f69677f0640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f69600b8a40 0x7f6960160540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:11.879 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:11.878+0000 7f69677f0640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6960160a80 con 0x7f69600d0ec0 2024-05-11T08:06:11.879 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:11.878+0000 7f69677f0640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69600d24a0 con 0x7f69600b5e90 2024-05-11T08:06:11.879 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:11.878+0000 7f69677f0640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f69600cf3e0 con 0x7f69600b8a40 2024-05-11T08:06:11.879 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:11.878+0000 7f6965d8d640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f69600d0ec0 0x7f69600b7ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:36344/0 (socket says 172.21.15.167:36344) 2024-05-11T08:06:11.879 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:11.878+0000 7f6965d8d640 1 -- 172.21.15.167:0/864143199 learned_addr learned my addr 172.21.15.167:0/864143199 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T08:06:11.879 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:11.878+0000 7f696558c640 1 --2- 172.21.15.167:0/864143199 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f69600b8a40 0x7f6960160540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:11.879 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:11.879+0000 7f6957fff640 1 -- 172.21.15.167:0/864143199 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3484033092 0 0) 0x7f6958001110 con 0x7f69600d0ec0 2024-05-11T08:06:11.880 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:11.879+0000 7f6957fff640 1 -- 172.21.15.167:0/864143199 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f694c003520 con 0x7f69600d0ec0 2024-05-11T08:06:11.880 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:11.879+0000 7f6957fff640 1 -- 172.21.15.167:0/864143199 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2168520887 0 0) 0x7f69600d24a0 con 0x7f69600b5e90 2024-05-11T08:06:11.880 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:11.879+0000 7f6957fff640 1 -- 172.21.15.167:0/864143199 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6958001110 con 0x7f69600b5e90 2024-05-11T08:06:11.880 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:11.879+0000 7f6957fff640 1 -- 172.21.15.167:0/864143199 <== mon.0 v1:172.21.15.167:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 897033573 0 0) 0x7f694c003520 con 0x7f69600d0ec0 2024-05-11T08:06:11.880 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:11.879+0000 7f696558c640 1 -- 172.21.15.167:0/864143199 >> v1:172.21.15.204:6789/0 conn(0x7f69600b5e90 legacy=0x7f69600cfba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:11.880 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:11.879+0000 7f696558c640 1 -- 172.21.15.167:0/864143199 >> v1:172.21.15.167:6789/0 conn(0x7f69600d0ec0 legacy=0x7f69600b7ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:11.880 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:11.879+0000 7f696558c640 1 -- 172.21.15.167:0/864143199 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69600d2740 con 0x7f69600b8a40 2024-05-11T08:06:11.881 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:11.880+0000 7f696558c640 1 --2- 172.21.15.167:0/864143199 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f69600b8a40 0x7f6960160540 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f695c009dc0 tx=0x7f695c02a5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d2238be76ee14f75 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:11.881 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:11.880+0000 7f6957fff640 1 -- 172.21.15.167:0/864143199 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69580018c0 con 0x7f69600b8a40 2024-05-11T08:06:11.881 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:11.880+0000 7f6957fff640 1 -- 172.21.15.167:0/864143199 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f695c007540 con 0x7f69600b8a40 2024-05-11T08:06:11.882 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:11.880+0000 7f6957fff640 1 -- 172.21.15.167:0/864143199 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f695c0078f0 con 0x7f69600b8a40 2024-05-11T08:06:11.882 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:11.880+0000 7f69677f0640 1 -- 172.21.15.167:0/864143199 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f69600b8a40 msgr2=0x7f6960160540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:11.882 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:11.880+0000 7f69677f0640 1 --2- 172.21.15.167:0/864143199 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f69600b8a40 0x7f6960160540 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f695c009dc0 tx=0x7f695c02a5d0 comp rx=0 tx=0).stop 2024-05-11T08:06:11.882 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:11.880+0000 7f69677f0640 1 -- 172.21.15.167:0/864143199 shutdown_connections 2024-05-11T08:06:11.882 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:11.880+0000 7f69677f0640 1 --2- 172.21.15.167:0/864143199 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f69600b8a40 0x7f6960160540 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:11.882 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:11.880+0000 7f69677f0640 1 -- 172.21.15.167:0/864143199 >> 172.21.15.167:0/864143199 conn(0x7f6960072730 msgr2=0x7f69600d2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:11.882 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:11.880+0000 7f69677f0640 1 -- 172.21.15.167:0/864143199 shutdown_connections 2024-05-11T08:06:11.882 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:11.880+0000 7f69677f0640 1 -- 172.21.15.167:0/864143199 wait complete. 2024-05-11T08:06:11.882 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:11.882+0000 7f69677f0640 1 Processor -- start 2024-05-11T08:06:11.882 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:11.882+0000 7f69677f0640 1 -- start start 2024-05-11T08:06:11.884 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:11.882+0000 7f69677f0640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f69600b5e90 0x7f6960151440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:11.885 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:11.882+0000 7f69677f0640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f69600b8a40 0x7f696015d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:11.885 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:11.882+0000 7f69677f0640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f69600d0ec0 0x7f696015d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:11.885 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:11.882+0000 7f69677f0640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f69600d02b0 con 0x7f69600b8a40 2024-05-11T08:06:11.885 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:11.882+0000 7f69677f0640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f69600cf680 con 0x7f69600b5e90 2024-05-11T08:06:11.885 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:11.882+0000 7f69677f0640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f69600b8e20 con 0x7f69600d0ec0 2024-05-11T08:06:11.885 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:11.882+0000 7f6964d8b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f69600b8a40 0x7f696015d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:11.885 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:11.883+0000 7f6964d8b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f69600b8a40 0x7f696015d250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:45690/0 (socket says 172.21.15.167:45690) 2024-05-11T08:06:11.885 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:11.883+0000 7f6964d8b640 1 -- 172.21.15.167:0/3703479877 learned_addr learned my addr 172.21.15.167:0/3703479877 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:11.885 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:11.883+0000 7f6965d8d640 1 --2- 172.21.15.167:0/3703479877 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f69600d0ec0 0x7f696015d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:11.885 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:11.883+0000 7f696558c640 1 --2- 172.21.15.167:0/3703479877 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f69600b5e90 0x7f6960151440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:11.885 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:11.883+0000 7f6964d8b640 1 -- 172.21.15.167:0/3703479877 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f69600d0ec0 msgr2=0x7f696015d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:11.885 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:11.883+0000 7f6964d8b640 1 --2- 172.21.15.167:0/3703479877 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f69600d0ec0 0x7f696015d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:11.885 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:11.883+0000 7f6964d8b640 1 -- 172.21.15.167:0/3703479877 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f69600b5e90 msgr2=0x7f6960151440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:11.886 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:11.883+0000 7f6964d8b640 1 --2- 172.21.15.167:0/3703479877 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f69600b5e90 0x7f6960151440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:11.886 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:11.883+0000 7f6964d8b640 1 -- 172.21.15.167:0/3703479877 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6960151980 con 0x7f69600b8a40 2024-05-11T08:06:11.886 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:11.883+0000 7f6965d8d640 1 --2- 172.21.15.167:0/3703479877 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f69600d0ec0 0x7f696015d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T08:06:11.886 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:11.883+0000 7f696558c640 1 --2- 172.21.15.167:0/3703479877 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f69600b5e90 0x7f6960151440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T08:06:11.886 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:11.883+0000 7f6964d8b640 1 --2- 172.21.15.167:0/3703479877 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f69600b8a40 0x7f696015d250 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f695000c430 tx=0x7f695000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:11.886 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:11.883+0000 7f6955ffb640 1 -- 172.21.15.167:0/3703479877 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6950017070 con 0x7f69600b8a40 2024-05-11T08:06:11.886 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:11.883+0000 7f6955ffb640 1 -- 172.21.15.167:0/3703479877 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69500052f0 con 0x7f69600b8a40 2024-05-11T08:06:11.886 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:11.883+0000 7f69677f0640 1 -- 172.21.15.167:0/3703479877 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69600ce940 con 0x7f69600b8a40 2024-05-11T08:06:11.886 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:11.883+0000 7f6955ffb640 1 -- 172.21.15.167:0/3703479877 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6950003830 con 0x7f69600b8a40 2024-05-11T08:06:11.886 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:11.883+0000 7f69677f0640 1 -- 172.21.15.167:0/3703479877 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f696015abb0 con 0x7f69600b8a40 2024-05-11T08:06:11.886 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:11.884+0000 7f69677f0640 1 -- 172.21.15.167:0/3703479877 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6960069dd0 con 0x7f69600b8a40 2024-05-11T08:06:11.887 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:11.885+0000 7f6955ffb640 1 -- 172.21.15.167:0/3703479877 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f695001a070 con 0x7f69600b8a40 2024-05-11T08:06:11.887 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:11.886+0000 7f6955ffb640 1 --2- 172.21.15.167:0/3703479877 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f69440762d0 0x7f6944078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:11.887 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:11.886+0000 7f696558c640 1 --2- 172.21.15.167:0/3703479877 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f69440762d0 0x7f6944078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:11.887 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:11.886+0000 7f6955ffb640 1 -- 172.21.15.167:0/3703479877 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f6950096d00 con 0x7f69600b8a40 2024-05-11T08:06:11.887 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:11.886+0000 7f696558c640 1 --2- 172.21.15.167:0/3703479877 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f69440762d0 0x7f6944078790 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f695c034000 tx=0x7f695c02f000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:11.890 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:11.890+0000 7f6955ffb640 1 -- 172.21.15.167:0/3703479877 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f6960069dd0 con 0x7f69600b8a40 2024-05-11T08:06:12.072 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:12.072+0000 7f69677f0640 1 -- 172.21.15.167:0/3703479877 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6960069590 con 0x7f69600b8a40 2024-05-11T08:06:12.073 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:12.073+0000 7f6955ffb640 1 -- 172.21.15.167:0/3703479877 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18562 (secure 0 0 0) 0x7f6960151980 con 0x7f69600b8a40 2024-05-11T08:06:12.074 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T08:06:12.074 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":27,"fsid":"2034778d-8ff1-4823-aa44-ded32b518241","created":"2024-05-11T08:05:31.612421+0000","modified":"2024-05-11T08:05:59.756806+0000","last_up_change":"2024-05-11T08:05:42.557382+0000","last_in_change":"2024-05-11T08:05:33.757311+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-11T08:05:43.931263+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-11T08:05:52.573726+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-11T08:05:53.670746+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-11T08:05:57.380272+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"587eaced-5ded-406d-884b-30c1caa8686b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":3863307356},{"type":"v1","addr":"172.21.15.167:6806","nonce":3863307356}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":3863307356},{"type":"v1","addr":"172.21.15.167:6810","nonce":3863307356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":3863307356},{"type":"v1","addr":"172.21.15.167:6815","nonce":3863307356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":3863307356},{"type":"v1","addr":"172.21.15.167:6813","nonce":3863307356}]},"public_addr":"172.21.15.167:6806/3863307356","cluster_addr":"172.21.15.167:6810/3863307356","heartbeat_back_addr":"172.21.15.167:6815/3863307356","heartbeat_front_addr":"172.21.15.167:6813/3863307356","state":["exists","up"]},{"osd":1,"uuid":"37c59104-2dd0-4cf9-85ea-56636f6b02a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":1585024048},{"type":"v1","addr":"172.21.15.167:6817","nonce":1585024048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1585024048},{"type":"v1","addr":"172.21.15.167:6819","nonce":1585024048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1585024048},{"type":"v1","addr":"172.21.15.167:6823","nonce":1585024048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":1585024048},{"type":"v1","addr":"172.21.15.167:6821","nonce":1585024048}]},"public_addr":"172.21.15.167:6817/1585024048","cluster_addr":"172.21.15.167:6819/1585024048","heartbeat_back_addr":"172.21.15.167:6823/1585024048","heartbeat_front_addr":"172.21.15.167:6821/1585024048","state":["exists","up"]},{"osd":2,"uuid":"2bb4f7f5-c698-451b-b4bb-78e716921291","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":3242166900},{"type":"v1","addr":"172.21.15.167:6801","nonce":3242166900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":3242166900},{"type":"v1","addr":"172.21.15.167:6803","nonce":3242166900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6809","nonce":3242166900},{"type":"v1","addr":"172.21.15.167:6811","nonce":3242166900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6805","nonce":3242166900},{"type":"v1","addr":"172.21.15.167:6807","nonce":3242166900}]},"public_addr":"172.21.15.167:6801/3242166900","cluster_addr":"172.21.15.167:6803/3242166900","heartbeat_back_addr":"172.21.15.167:6811/3242166900","heartbeat_front_addr":"172.21.15.167:6807/3242166900","state":["exists","up"]},{"osd":3,"uuid":"9dd26759-a338-49d2-8103-45d068b9c000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":453689293},{"type":"v1","addr":"172.21.15.167:6825","nonce":453689293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":453689293},{"type":"v1","addr":"172.21.15.167:6827","nonce":453689293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":453689293},{"type":"v1","addr":"172.21.15.167:6831","nonce":453689293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":453689293},{"type":"v1","addr":"172.21.15.167:6829","nonce":453689293}]},"public_addr":"172.21.15.167:6825/453689293","cluster_addr":"172.21.15.167:6827/453689293","heartbeat_back_addr":"172.21.15.167:6831/453689293","heartbeat_front_addr":"172.21.15.167:6829/453689293","state":["exists","up"]},{"osd":4,"uuid":"cb27250d-2a80-4c7b-8cef-3a7af05c8a16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1311831500},{"type":"v1","addr":"172.21.15.204:6801","nonce":1311831500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1311831500},{"type":"v1","addr":"172.21.15.204:6803","nonce":1311831500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1311831500},{"type":"v1","addr":"172.21.15.204:6807","nonce":1311831500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1311831500},{"type":"v1","addr":"172.21.15.204:6805","nonce":1311831500}]},"public_addr":"172.21.15.204:6801/1311831500","cluster_addr":"172.21.15.204:6803/1311831500","heartbeat_back_addr":"172.21.15.204:6807/1311831500","heartbeat_front_addr":"172.21.15.204:6805/1311831500","state":["exists","up"]},{"osd":5,"uuid":"92998541-ffec-430b-ab76-8c3cb7c2752d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2921847521},{"type":"v1","addr":"172.21.15.204:6825","nonce":2921847521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2921847521},{"type":"v1","addr":"172.21.15.204:6827","nonce":2921847521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2921847521},{"type":"v1","addr":"172.21.15.204:6831","nonce":2921847521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2921847521},{"type":"v1","addr":"172.21.15.204:6829","nonce":2921847521}]},"public_addr":"172.21.15.204:6825/2921847521","cluster_addr":"172.21.15.204:6827/2921847521","heartbeat_back_addr":"172.21.15.204:6831/2921847521","heartbeat_front_addr":"172.21.15.204:6829/2921847521","state":["exists","up"]},{"osd":6,"uuid":"9543479c-2ec5-42a8-8933-707bfed2e048","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":2889851077},{"type":"v1","addr":"172.21.15.204:6817","nonce":2889851077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":2889851077},{"type":"v1","addr":"172.21.15.204:6819","nonce":2889851077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":2889851077},{"type":"v1","addr":"172.21.15.204:6823","nonce":2889851077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":2889851077},{"type":"v1","addr":"172.21.15.204:6821","nonce":2889851077}]},"public_addr":"172.21.15.204:6817/2889851077","cluster_addr":"172.21.15.204:6819/2889851077","heartbeat_back_addr":"172.21.15.204:6823/2889851077","heartbeat_front_addr":"172.21.15.204:6821/2889851077","state":["exists","up"]},{"osd":7,"uuid":"c45a5647-8358-409e-af91-c05dda65f0b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":1949534184},{"type":"v1","addr":"172.21.15.204:6809","nonce":1949534184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":1949534184},{"type":"v1","addr":"172.21.15.204:6811","nonce":1949534184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1949534184},{"type":"v1","addr":"172.21.15.204:6815","nonce":1949534184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1949534184},{"type":"v1","addr":"172.21.15.204:6813","nonce":1949534184}]},"public_addr":"172.21.15.204:6809/1949534184","cluster_addr":"172.21.15.204:6811/1949534184","heartbeat_back_addr":"172.21.15.204:6815/1949534184","heartbeat_front_addr":"172.21.15.204:6813/1949534184","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.952253+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.940689+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.860682+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.929350+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.901627+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.957451+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.987249+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.951197+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-11T08:06:12.076 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:12.076+0000 7f69677f0640 1 -- 172.21.15.167:0/3703479877 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f69440762d0 msgr2=0x7f6944078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:12.076 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:12.076+0000 7f69677f0640 1 --2- 172.21.15.167:0/3703479877 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f69440762d0 0x7f6944078790 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f695c034000 tx=0x7f695c02f000 comp rx=0 tx=0).stop 2024-05-11T08:06:12.076 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:12.076+0000 7f69677f0640 1 -- 172.21.15.167:0/3703479877 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f69600b8a40 msgr2=0x7f696015d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:12.076 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:12.076+0000 7f69677f0640 1 --2- 172.21.15.167:0/3703479877 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f69600b8a40 0x7f696015d250 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f695000c430 tx=0x7f695000c8f0 comp rx=0 tx=0).stop 2024-05-11T08:06:12.076 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:12.076+0000 7f69677f0640 1 -- 172.21.15.167:0/3703479877 shutdown_connections 2024-05-11T08:06:12.077 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:12.076+0000 7f69677f0640 1 --2- 172.21.15.167:0/3703479877 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f69440762d0 0x7f6944078790 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:12.077 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:12.076+0000 7f69677f0640 1 --2- 172.21.15.167:0/3703479877 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f69600d0ec0 0x7f696015d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:12.077 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:12.076+0000 7f69677f0640 1 --2- 172.21.15.167:0/3703479877 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f69600b8a40 0x7f696015d250 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:12.077 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:12.076+0000 7f69677f0640 1 --2- 172.21.15.167:0/3703479877 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f69600b5e90 0x7f6960151440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:12.077 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:12.076+0000 7f69677f0640 1 -- 172.21.15.167:0/3703479877 >> 172.21.15.167:0/3703479877 conn(0x7f6960072730 msgr2=0x7f696006ddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:12.077 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:12.076+0000 7f69677f0640 1 -- 172.21.15.167:0/3703479877 shutdown_connections 2024-05-11T08:06:12.077 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:12.076+0000 7f69677f0640 1 -- 172.21.15.167:0/3703479877 wait complete. 2024-05-11T08:06:12.087 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-11T08:06:12.188 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:12.188+0000 7f1521f22640 1 Processor -- start 2024-05-11T08:06:12.188 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:12.188+0000 7f1521f22640 1 -- start start 2024-05-11T08:06:12.189 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:12.189+0000 7f1521f22640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f151c0d0ec0 0x7f151c0d1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:12.190 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:12.189+0000 7f1521f22640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f151c0d24a0 con 0x7f151c0b5e90 2024-05-11T08:06:12.190 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:12.189+0000 7f1521f22640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f151c0cf3e0 con 0x7f151c0b8a40 2024-05-11T08:06:12.190 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:12.189+0000 7f1521f22640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f151c0d1850 con 0x7f151c0d0ec0 2024-05-11T08:06:12.190 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:12.189+0000 7f151bfff640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f151c0d0ec0 0x7f151c0d1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:12.190 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:12.189+0000 7f151affd640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f151c0b5e90 0x7f151c0cfba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:36350/0 (socket says 172.21.15.167:36350) 2024-05-11T08:06:12.190 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:12.189+0000 7f151affd640 1 -- 172.21.15.167:0/3054338270 learned_addr learned my addr 172.21.15.167:0/3054338270 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T08:06:12.190 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:12.189+0000 7f151bfff640 1 -- 172.21.15.167:0/3054338270 >> v1:172.21.15.204:6789/0 conn(0x7f151c0b8a40 legacy=0x7f151c160540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:12.190 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:12.189+0000 7f151bfff640 1 -- 172.21.15.167:0/3054338270 >> v1:172.21.15.167:6789/0 conn(0x7f151c0b5e90 legacy=0x7f151c0cfba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:12.190 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:12.189+0000 7f151bfff640 1 -- 172.21.15.167:0/3054338270 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f151c0d2740 con 0x7f151c0d0ec0 2024-05-11T08:06:12.191 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:12.190+0000 7f151bfff640 1 --2- 172.21.15.167:0/3054338270 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f151c0d0ec0 0x7f151c0d1310 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f151c0b67e0 tx=0x7f151402a590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ea2e718f7595bc6 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:12.191 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:12.190+0000 7f151a7fc640 1 -- 172.21.15.167:0/3054338270 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1514007740 con 0x7f151c0d0ec0 2024-05-11T08:06:12.192 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:12.190+0000 7f151a7fc640 1 -- 172.21.15.167:0/3054338270 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15140078e0 con 0x7f151c0d0ec0 2024-05-11T08:06:12.192 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:12.190+0000 7f151a7fc640 1 -- 172.21.15.167:0/3054338270 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1514007cb0 con 0x7f151c0d0ec0 2024-05-11T08:06:12.192 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:12.191+0000 7f1521f22640 1 -- 172.21.15.167:0/3054338270 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f151c0d0ec0 msgr2=0x7f151c0d1310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:12.192 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:12.191+0000 7f1521f22640 1 --2- 172.21.15.167:0/3054338270 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f151c0d0ec0 0x7f151c0d1310 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f151c0b67e0 tx=0x7f151402a590 comp rx=0 tx=0).stop 2024-05-11T08:06:12.192 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:12.191+0000 7f1521f22640 1 -- 172.21.15.167:0/3054338270 shutdown_connections 2024-05-11T08:06:12.192 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:12.191+0000 7f1521f22640 1 --2- 172.21.15.167:0/3054338270 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f151c0d0ec0 0x7f151c0d1310 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:12.192 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:12.191+0000 7f1521f22640 1 -- 172.21.15.167:0/3054338270 >> 172.21.15.167:0/3054338270 conn(0x7f151c072730 msgr2=0x7f151c0d2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:12.192 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:12.191+0000 7f1521f22640 1 -- 172.21.15.167:0/3054338270 shutdown_connections 2024-05-11T08:06:12.192 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:12.191+0000 7f1521f22640 1 -- 172.21.15.167:0/3054338270 wait complete. 2024-05-11T08:06:12.195 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:12.192+0000 7f1521f22640 1 Processor -- start 2024-05-11T08:06:12.195 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:12.192+0000 7f1521f22640 1 -- start start 2024-05-11T08:06:12.195 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:12.193+0000 7f1521f22640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f151c0b5e90 0x7f151c14d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:12.195 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:12.193+0000 7f1521f22640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f151c0b8a40 0x7f151c0c76a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:12.195 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:12.193+0000 7f1521f22640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f151c0d0ec0 0x7f151c1553d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:12.195 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:12.193+0000 7f1521f22640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f151c0d02b0 con 0x7f151c0d0ec0 2024-05-11T08:06:12.195 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:12.193+0000 7f1521f22640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f151c0cf680 con 0x7f151c0b8a40 2024-05-11T08:06:12.195 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:12.193+0000 7f1521f22640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f151c0b8e20 con 0x7f151c0b5e90 2024-05-11T08:06:12.195 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:12.193+0000 7f151bfff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f151c0d0ec0 0x7f151c1553d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:12.195 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:12.193+0000 7f151bfff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f151c0d0ec0 0x7f151c1553d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:45718/0 (socket says 172.21.15.167:45718) 2024-05-11T08:06:12.195 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:12.193+0000 7f151bfff640 1 -- 172.21.15.167:0/2758038688 learned_addr learned my addr 172.21.15.167:0/2758038688 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:12.195 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:12.193+0000 7f151b7fe640 1 --2- 172.21.15.167:0/2758038688 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f151c0b5e90 0x7f151c14d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:12.195 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:12.193+0000 7f151affd640 1 --2- 172.21.15.167:0/2758038688 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f151c0b8a40 0x7f151c0c76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:12.196 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:12.193+0000 7f151bfff640 1 -- 172.21.15.167:0/2758038688 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f151c0b5e90 msgr2=0x7f151c14d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:12.196 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:12.193+0000 7f151bfff640 1 --2- 172.21.15.167:0/2758038688 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f151c0b5e90 0x7f151c14d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:12.196 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:12.193+0000 7f151bfff640 1 -- 172.21.15.167:0/2758038688 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f151c0b8a40 msgr2=0x7f151c0c76a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:12.196 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:12.193+0000 7f151bfff640 1 --2- 172.21.15.167:0/2758038688 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f151c0b8a40 0x7f151c0c76a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:12.196 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:12.193+0000 7f151bfff640 1 -- 172.21.15.167:0/2758038688 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f151c0c8060 con 0x7f151c0d0ec0 2024-05-11T08:06:12.196 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:12.193+0000 7f151affd640 1 --2- 172.21.15.167:0/2758038688 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f151c0b8a40 0x7f151c0c76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T08:06:12.196 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:12.193+0000 7f151bfff640 1 --2- 172.21.15.167:0/2758038688 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f151c0d0ec0 0x7f151c1553d0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f151402ab80 tx=0x7f1514004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:12.196 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:12.194+0000 7f1503fff640 1 -- 172.21.15.167:0/2758038688 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1514040070 con 0x7f151c0d0ec0 2024-05-11T08:06:12.196 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:12.194+0000 7f1503fff640 1 -- 172.21.15.167:0/2758038688 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1514031bc0 con 0x7f151c0d0ec0 2024-05-11T08:06:12.196 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:12.194+0000 7f1521f22640 1 -- 172.21.15.167:0/2758038688 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f151c0c8340 con 0x7f151c0d0ec0 2024-05-11T08:06:12.196 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:12.194+0000 7f1503fff640 1 -- 172.21.15.167:0/2758038688 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1514045460 con 0x7f151c0d0ec0 2024-05-11T08:06:12.196 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:12.194+0000 7f1521f22640 1 -- 172.21.15.167:0/2758038688 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f151c0c8900 con 0x7f151c0d0ec0 2024-05-11T08:06:12.196 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:12.195+0000 7f1521f22640 1 -- 172.21.15.167:0/2758038688 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f151c069590 con 0x7f151c0d0ec0 2024-05-11T08:06:12.197 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:12.195+0000 7f1503fff640 1 -- 172.21.15.167:0/2758038688 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f1514039040 con 0x7f151c0d0ec0 2024-05-11T08:06:12.197 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:12.196+0000 7f1503fff640 1 --2- 172.21.15.167:0/2758038688 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f15080762d0 0x7f1508078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:12.197 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:12.196+0000 7f1503fff640 1 -- 172.21.15.167:0/2758038688 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f15140b91a0 con 0x7f151c0d0ec0 2024-05-11T08:06:12.198 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:12.196+0000 7f151b7fe640 1 --2- 172.21.15.167:0/2758038688 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f15080762d0 0x7f1508078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:12.198 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:12.196+0000 7f151b7fe640 1 --2- 172.21.15.167:0/2758038688 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f15080762d0 0x7f1508078790 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1510004f60 tx=0x7f1510007e30 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:12.200 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:12.200+0000 7f1503fff640 1 -- 172.21.15.167:0/2758038688 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f1514082850 con 0x7f151c0d0ec0 2024-05-11T08:06:12.386 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:12.386+0000 7f1521f22640 1 -- 172.21.15.167:0/2758038688 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f151c1e5270 con 0x7f151c0d0ec0 2024-05-11T08:06:12.387 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:12.387+0000 7f1503fff640 1 -- 172.21.15.167:0/2758038688 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18562 (secure 0 0 0) 0x7f1514082150 con 0x7f151c0d0ec0 2024-05-11T08:06:12.387 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T08:06:12.387 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":27,"fsid":"2034778d-8ff1-4823-aa44-ded32b518241","created":"2024-05-11T08:05:31.612421+0000","modified":"2024-05-11T08:05:59.756806+0000","last_up_change":"2024-05-11T08:05:42.557382+0000","last_in_change":"2024-05-11T08:05:33.757311+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-11T08:05:43.931263+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-11T08:05:52.573726+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-11T08:05:53.670746+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-11T08:05:57.380272+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"587eaced-5ded-406d-884b-30c1caa8686b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":3863307356},{"type":"v1","addr":"172.21.15.167:6806","nonce":3863307356}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":3863307356},{"type":"v1","addr":"172.21.15.167:6810","nonce":3863307356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":3863307356},{"type":"v1","addr":"172.21.15.167:6815","nonce":3863307356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":3863307356},{"type":"v1","addr":"172.21.15.167:6813","nonce":3863307356}]},"public_addr":"172.21.15.167:6806/3863307356","cluster_addr":"172.21.15.167:6810/3863307356","heartbeat_back_addr":"172.21.15.167:6815/3863307356","heartbeat_front_addr":"172.21.15.167:6813/3863307356","state":["exists","up"]},{"osd":1,"uuid":"37c59104-2dd0-4cf9-85ea-56636f6b02a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":1585024048},{"type":"v1","addr":"172.21.15.167:6817","nonce":1585024048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1585024048},{"type":"v1","addr":"172.21.15.167:6819","nonce":1585024048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1585024048},{"type":"v1","addr":"172.21.15.167:6823","nonce":1585024048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":1585024048},{"type":"v1","addr":"172.21.15.167:6821","nonce":1585024048}]},"public_addr":"172.21.15.167:6817/1585024048","cluster_addr":"172.21.15.167:6819/1585024048","heartbeat_back_addr":"172.21.15.167:6823/1585024048","heartbeat_front_addr":"172.21.15.167:6821/1585024048","state":["exists","up"]},{"osd":2,"uuid":"2bb4f7f5-c698-451b-b4bb-78e716921291","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":3242166900},{"type":"v1","addr":"172.21.15.167:6801","nonce":3242166900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":3242166900},{"type":"v1","addr":"172.21.15.167:6803","nonce":3242166900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6809","nonce":3242166900},{"type":"v1","addr":"172.21.15.167:6811","nonce":3242166900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6805","nonce":3242166900},{"type":"v1","addr":"172.21.15.167:6807","nonce":3242166900}]},"public_addr":"172.21.15.167:6801/3242166900","cluster_addr":"172.21.15.167:6803/3242166900","heartbeat_back_addr":"172.21.15.167:6811/3242166900","heartbeat_front_addr":"172.21.15.167:6807/3242166900","state":["exists","up"]},{"osd":3,"uuid":"9dd26759-a338-49d2-8103-45d068b9c000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":453689293},{"type":"v1","addr":"172.21.15.167:6825","nonce":453689293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":453689293},{"type":"v1","addr":"172.21.15.167:6827","nonce":453689293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":453689293},{"type":"v1","addr":"172.21.15.167:6831","nonce":453689293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":453689293},{"type":"v1","addr":"172.21.15.167:6829","nonce":453689293}]},"public_addr":"172.21.15.167:6825/453689293","cluster_addr":"172.21.15.167:6827/453689293","heartbeat_back_addr":"172.21.15.167:6831/453689293","heartbeat_front_addr":"172.21.15.167:6829/453689293","state":["exists","up"]},{"osd":4,"uuid":"cb27250d-2a80-4c7b-8cef-3a7af05c8a16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1311831500},{"type":"v1","addr":"172.21.15.204:6801","nonce":1311831500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1311831500},{"type":"v1","addr":"172.21.15.204:6803","nonce":1311831500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1311831500},{"type":"v1","addr":"172.21.15.204:6807","nonce":1311831500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1311831500},{"type":"v1","addr":"172.21.15.204:6805","nonce":1311831500}]},"public_addr":"172.21.15.204:6801/1311831500","cluster_addr":"172.21.15.204:6803/1311831500","heartbeat_back_addr":"172.21.15.204:6807/1311831500","heartbeat_front_addr":"172.21.15.204:6805/1311831500","state":["exists","up"]},{"osd":5,"uuid":"92998541-ffec-430b-ab76-8c3cb7c2752d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2921847521},{"type":"v1","addr":"172.21.15.204:6825","nonce":2921847521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2921847521},{"type":"v1","addr":"172.21.15.204:6827","nonce":2921847521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2921847521},{"type":"v1","addr":"172.21.15.204:6831","nonce":2921847521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2921847521},{"type":"v1","addr":"172.21.15.204:6829","nonce":2921847521}]},"public_addr":"172.21.15.204:6825/2921847521","cluster_addr":"172.21.15.204:6827/2921847521","heartbeat_back_addr":"172.21.15.204:6831/2921847521","heartbeat_front_addr":"172.21.15.204:6829/2921847521","state":["exists","up"]},{"osd":6,"uuid":"9543479c-2ec5-42a8-8933-707bfed2e048","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":2889851077},{"type":"v1","addr":"172.21.15.204:6817","nonce":2889851077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":2889851077},{"type":"v1","addr":"172.21.15.204:6819","nonce":2889851077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":2889851077},{"type":"v1","addr":"172.21.15.204:6823","nonce":2889851077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":2889851077},{"type":"v1","addr":"172.21.15.204:6821","nonce":2889851077}]},"public_addr":"172.21.15.204:6817/2889851077","cluster_addr":"172.21.15.204:6819/2889851077","heartbeat_back_addr":"172.21.15.204:6823/2889851077","heartbeat_front_addr":"172.21.15.204:6821/2889851077","state":["exists","up"]},{"osd":7,"uuid":"c45a5647-8358-409e-af91-c05dda65f0b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":1949534184},{"type":"v1","addr":"172.21.15.204:6809","nonce":1949534184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":1949534184},{"type":"v1","addr":"172.21.15.204:6811","nonce":1949534184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1949534184},{"type":"v1","addr":"172.21.15.204:6815","nonce":1949534184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1949534184},{"type":"v1","addr":"172.21.15.204:6813","nonce":1949534184}]},"public_addr":"172.21.15.204:6809/1949534184","cluster_addr":"172.21.15.204:6811/1949534184","heartbeat_back_addr":"172.21.15.204:6815/1949534184","heartbeat_front_addr":"172.21.15.204:6813/1949534184","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.952253+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.940689+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.860682+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.929350+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.901627+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.957451+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.987249+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.951197+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-11T08:06:12.389 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:12.389+0000 7f1521f22640 1 -- 172.21.15.167:0/2758038688 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f15080762d0 msgr2=0x7f1508078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:12.389 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:12.389+0000 7f1521f22640 1 --2- 172.21.15.167:0/2758038688 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f15080762d0 0x7f1508078790 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1510004f60 tx=0x7f1510007e30 comp rx=0 tx=0).stop 2024-05-11T08:06:12.389 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:12.389+0000 7f1521f22640 1 -- 172.21.15.167:0/2758038688 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f151c0d0ec0 msgr2=0x7f151c1553d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:12.389 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:12.389+0000 7f1521f22640 1 --2- 172.21.15.167:0/2758038688 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f151c0d0ec0 0x7f151c1553d0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f151402ab80 tx=0x7f1514004a40 comp rx=0 tx=0).stop 2024-05-11T08:06:12.390 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:12.389+0000 7f1521f22640 1 -- 172.21.15.167:0/2758038688 shutdown_connections 2024-05-11T08:06:12.390 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:12.389+0000 7f1521f22640 1 --2- 172.21.15.167:0/2758038688 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f15080762d0 0x7f1508078790 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:12.390 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:12.389+0000 7f1521f22640 1 --2- 172.21.15.167:0/2758038688 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f151c0d0ec0 0x7f151c1553d0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:12.390 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:12.389+0000 7f1521f22640 1 --2- 172.21.15.167:0/2758038688 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f151c0b8a40 0x7f151c0c76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:12.390 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:12.389+0000 7f1521f22640 1 --2- 172.21.15.167:0/2758038688 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f151c0b5e90 0x7f151c14d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:12.390 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:12.389+0000 7f1521f22640 1 -- 172.21.15.167:0/2758038688 >> 172.21.15.167:0/2758038688 conn(0x7f151c072730 msgr2=0x7f151c0cc620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:12.390 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:12.389+0000 7f1521f22640 1 -- 172.21.15.167:0/2758038688 shutdown_connections 2024-05-11T08:06:12.390 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:12.389+0000 7f1521f22640 1 -- 172.21.15.167:0/2758038688 wait complete. 2024-05-11T08:06:12.401 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-11T08:06:12.504 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:12.503+0000 7fb69e8e1640 1 Processor -- start 2024-05-11T08:06:12.504 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:12.504+0000 7fb69e8e1640 1 -- start start 2024-05-11T08:06:12.506 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:12.504+0000 7fb69e8e1640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb69815d170 0x7fb69806e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:12.506 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:12.504+0000 7fb69e8e1640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb69815c6d0 con 0x7fb698073270 2024-05-11T08:06:12.506 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:12.504+0000 7fb69e8e1640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6980700b0 con 0x7fb6980ce8f0 2024-05-11T08:06:12.506 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:12.504+0000 7fb69e8e1640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fb698160b00 con 0x7fb69815d170 2024-05-11T08:06:12.506 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:12.505+0000 7fb697fff640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7fb698073270 0x7fb698073650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:36366/0 (socket says 172.21.15.167:36366) 2024-05-11T08:06:12.506 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:12.505+0000 7fb697fff640 1 -- 172.21.15.167:0/3782247246 learned_addr learned my addr 172.21.15.167:0/3782247246 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T08:06:12.506 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:12.505+0000 7fb68ffff640 1 --2- 172.21.15.167:0/3782247246 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb69815d170 0x7fb69806e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:12.506 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:12.505+0000 7fb6977fe640 1 -- 172.21.15.167:0/3782247246 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1390216976 0 0) 0x7fb69815c6d0 con 0x7fb698073270 2024-05-11T08:06:12.506 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:12.505+0000 7fb6977fe640 1 -- 172.21.15.167:0/3782247246 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb684003520 con 0x7fb698073270 2024-05-11T08:06:12.506 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:12.505+0000 7fb6977fe640 1 -- 172.21.15.167:0/3782247246 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 753632006 0 0) 0x7fb6980700b0 con 0x7fb6980ce8f0 2024-05-11T08:06:12.507 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:12.505+0000 7fb6977fe640 1 -- 172.21.15.167:0/3782247246 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb69815c6d0 con 0x7fb6980ce8f0 2024-05-11T08:06:12.507 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:12.505+0000 7fb68ffff640 1 -- 172.21.15.167:0/3782247246 >> v1:172.21.15.204:6789/0 conn(0x7fb6980ce8f0 legacy=0x7fb6980ced80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:12.511 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:12.505+0000 7fb68ffff640 1 -- 172.21.15.167:0/3782247246 >> v1:172.21.15.167:6789/0 conn(0x7fb698073270 legacy=0x7fb698073650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:12.511 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:12.505+0000 7fb68ffff640 1 -- 172.21.15.167:0/3782247246 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb69815c970 con 0x7fb69815d170 2024-05-11T08:06:12.511 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:12.506+0000 7fb68ffff640 1 --2- 172.21.15.167:0/3782247246 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb69815d170 0x7fb69806e3c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb6800097d0 tx=0x7fb68002a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=26030e3d8d2c4091 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:12.511 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:12.506+0000 7fb6977fe640 1 -- 172.21.15.167:0/3782247246 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb680037070 con 0x7fb69815d170 2024-05-11T08:06:12.511 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:12.506+0000 7fb6977fe640 1 -- 172.21.15.167:0/3782247246 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb680032440 con 0x7fb69815d170 2024-05-11T08:06:12.511 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:12.506+0000 7fb6977fe640 1 -- 172.21.15.167:0/3782247246 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb680032810 con 0x7fb69815d170 2024-05-11T08:06:12.514 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:12.506+0000 7fb69e8e1640 1 -- 172.21.15.167:0/3782247246 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb69815d170 msgr2=0x7fb69806e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:12.514 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:12.506+0000 7fb69e8e1640 1 --2- 172.21.15.167:0/3782247246 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb69815d170 0x7fb69806e3c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb6800097d0 tx=0x7fb68002a210 comp rx=0 tx=0).stop 2024-05-11T08:06:12.514 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:12.507+0000 7fb69e8e1640 1 -- 172.21.15.167:0/3782247246 shutdown_connections 2024-05-11T08:06:12.515 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:12.507+0000 7fb69e8e1640 1 --2- 172.21.15.167:0/3782247246 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb69815d170 0x7fb69806e3c0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:12.515 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:12.507+0000 7fb69e8e1640 1 -- 172.21.15.167:0/3782247246 >> 172.21.15.167:0/3782247246 conn(0x7fb69806c450 msgr2=0x7fb69806c860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:12.515 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:12.507+0000 7fb69e8e1640 1 -- 172.21.15.167:0/3782247246 shutdown_connections 2024-05-11T08:06:12.515 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:12.507+0000 7fb69e8e1640 1 -- 172.21.15.167:0/3782247246 wait complete. 2024-05-11T08:06:12.515 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:12.508+0000 7fb69e8e1640 1 Processor -- start 2024-05-11T08:06:12.515 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:12.508+0000 7fb69e8e1640 1 -- start start 2024-05-11T08:06:12.515 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:12.508+0000 7fb69e8e1640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb698073270 0x7fb69815b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:12.515 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:12.508+0000 7fb69e8e1640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb6980ce8f0 0x7fb6980cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:12.515 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:12.508+0000 7fb69e8e1640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb69815d170 0x7fb6980cfd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:12.515 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:12.508+0000 7fb69e8e1640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb69806fd60 con 0x7fb698073270 2024-05-11T08:06:12.515 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:12.508+0000 7fb69e8e1640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb69815cce0 con 0x7fb6980ce8f0 2024-05-11T08:06:12.515 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:12.508+0000 7fb69e8e1640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fb698145bb0 con 0x7fb69815d170 2024-05-11T08:06:12.515 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:12.509+0000 7fb697fff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb698073270 0x7fb69815b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:12.516 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:12.509+0000 7fb697fff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb698073270 0x7fb69815b940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:45732/0 (socket says 172.21.15.167:45732) 2024-05-11T08:06:12.516 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:12.509+0000 7fb697fff640 1 -- 172.21.15.167:0/3486329233 learned_addr learned my addr 172.21.15.167:0/3486329233 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:12.516 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:12.509+0000 7fb68ffff640 1 --2- 172.21.15.167:0/3486329233 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb6980ce8f0 0x7fb6980cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:12.516 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:12.509+0000 7fb69ce7e640 1 --2- 172.21.15.167:0/3486329233 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb69815d170 0x7fb6980cfd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:12.516 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:12.509+0000 7fb697fff640 1 -- 172.21.15.167:0/3486329233 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb69815d170 msgr2=0x7fb6980cfd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:12.516 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:12.509+0000 7fb697fff640 1 --2- 172.21.15.167:0/3486329233 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb69815d170 0x7fb6980cfd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:12.516 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:12.509+0000 7fb697fff640 1 -- 172.21.15.167:0/3486329233 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb6980ce8f0 msgr2=0x7fb6980cf7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:12.516 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:12.509+0000 7fb697fff640 1 --2- 172.21.15.167:0/3486329233 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb6980ce8f0 0x7fb6980cf7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:12.516 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:12.509+0000 7fb697fff640 1 -- 172.21.15.167:0/3486329233 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb698160530 con 0x7fb698073270 2024-05-11T08:06:12.516 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:12.509+0000 7fb697fff640 1 --2- 172.21.15.167:0/3486329233 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb698073270 0x7fb69815b940 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fb68800c500 tx=0x7fb68800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:12.516 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:12.509+0000 7fb6957fa640 1 -- 172.21.15.167:0/3486329233 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb688017070 con 0x7fb698073270 2024-05-11T08:06:12.516 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:12.509+0000 7fb6957fa640 1 -- 172.21.15.167:0/3486329233 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb688005ce0 con 0x7fb698073270 2024-05-11T08:06:12.516 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:12.509+0000 7fb69e8e1640 1 -- 172.21.15.167:0/3486329233 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb69813b250 con 0x7fb698073270 2024-05-11T08:06:12.517 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:12.509+0000 7fb6957fa640 1 -- 172.21.15.167:0/3486329233 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb688007270 con 0x7fb698073270 2024-05-11T08:06:12.517 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:12.509+0000 7fb69e8e1640 1 -- 172.21.15.167:0/3486329233 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb69813b810 con 0x7fb698073270 2024-05-11T08:06:12.517 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:12.510+0000 7fb69e8e1640 1 -- 172.21.15.167:0/3486329233 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb698069520 con 0x7fb698073270 2024-05-11T08:06:12.517 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:12.511+0000 7fb6957fa640 1 -- 172.21.15.167:0/3486329233 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fb68801a070 con 0x7fb698073270 2024-05-11T08:06:12.517 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:12.511+0000 7fb6957fa640 1 --2- 172.21.15.167:0/3486329233 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fb6780762d0 0x7fb678078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:12.517 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:12.511+0000 7fb6957fa640 1 -- 172.21.15.167:0/3486329233 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fb688096bf0 con 0x7fb698073270 2024-05-11T08:06:12.517 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:12.511+0000 7fb68ffff640 1 --2- 172.21.15.167:0/3486329233 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fb6780762d0 0x7fb678078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:12.517 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:12.512+0000 7fb68ffff640 1 --2- 172.21.15.167:0/3486329233 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fb6780762d0 0x7fb678078790 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb680034000 tx=0x7fb68002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:12.517 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:12.516+0000 7fb6957fa640 1 -- 172.21.15.167:0/3486329233 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fb68805fbd0 con 0x7fb698073270 2024-05-11T08:06:12.733 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:12.734+0000 7fb69e8e1640 1 -- 172.21.15.167:0/3486329233 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fb6980d0a60 con 0x7fb698073270 2024-05-11T08:06:13.429 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:13.429+0000 7fb6957fa640 1 -- 172.21.15.167:0/3486329233 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v8) ==== 118+0+0 (secure 0 0 0) 0x7fb688063ed0 con 0x7fb698073270 2024-05-11T08:06:13.432 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:13.431+0000 7fb69e8e1640 1 -- 172.21.15.167:0/3486329233 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fb6780762d0 msgr2=0x7fb678078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:13.432 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:13.431+0000 7fb69e8e1640 1 --2- 172.21.15.167:0/3486329233 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fb6780762d0 0x7fb678078790 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb680034000 tx=0x7fb68002e000 comp rx=0 tx=0).stop 2024-05-11T08:06:13.432 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:13.431+0000 7fb69e8e1640 1 -- 172.21.15.167:0/3486329233 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb698073270 msgr2=0x7fb69815b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:13.432 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:13.431+0000 7fb69e8e1640 1 --2- 172.21.15.167:0/3486329233 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb698073270 0x7fb69815b940 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fb68800c500 tx=0x7fb68800c9c0 comp rx=0 tx=0).stop 2024-05-11T08:06:13.432 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:13.431+0000 7fb69e8e1640 1 -- 172.21.15.167:0/3486329233 shutdown_connections 2024-05-11T08:06:13.432 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:13.431+0000 7fb69e8e1640 1 --2- 172.21.15.167:0/3486329233 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fb6780762d0 0x7fb678078790 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:13.432 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:13.431+0000 7fb69e8e1640 1 --2- 172.21.15.167:0/3486329233 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb69815d170 0x7fb6980cfd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:13.432 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:13.431+0000 7fb69e8e1640 1 --2- 172.21.15.167:0/3486329233 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb6980ce8f0 0x7fb6980cf7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:13.432 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:13.431+0000 7fb69e8e1640 1 --2- 172.21.15.167:0/3486329233 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb698073270 0x7fb69815b940 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:13.432 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:13.431+0000 7fb69e8e1640 1 -- 172.21.15.167:0/3486329233 >> 172.21.15.167:0/3486329233 conn(0x7fb69806c450 msgr2=0x7fb6980b22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:13.432 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:13.431+0000 7fb69e8e1640 1 -- 172.21.15.167:0/3486329233 shutdown_connections 2024-05-11T08:06:13.432 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:13.431+0000 7fb69e8e1640 1 -- 172.21.15.167:0/3486329233 wait complete. 2024-05-11T08:06:13.444 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']} 2024-05-11T08:06:13.444 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-11T08:06:13.549 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:13.549+0000 7ff5613dd640 1 Processor -- start 2024-05-11T08:06:13.549 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:13.549+0000 7ff5613dd640 1 -- start start 2024-05-11T08:06:13.550 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:13.550+0000 7ff5613dd640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff55c0d40a0 0x7ff55c15fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:13.550 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:13.550+0000 7ff5613dd640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7ff55c160230 0x7ff55c168ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:13.550 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:13.550+0000 7ff5613dd640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff55c06dcc0 con 0x7ff55c15f910 2024-05-11T08:06:13.550 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:13.550+0000 7ff5613dd640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7ff55c156820 con 0x7ff55c160230 2024-05-11T08:06:13.551 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:13.550+0000 7ff5613dd640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7ff55c06d140 con 0x7ff55c0d40a0 2024-05-11T08:06:13.551 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:13.550+0000 7ff55affd640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7ff55c15f910 0x7ff55c06c6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:36380/0 (socket says 172.21.15.167:36380) 2024-05-11T08:06:13.551 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:13.550+0000 7ff55b7fe640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7ff55c160230 0x7ff55c168ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:13.551 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:13.550+0000 7ff55affd640 1 -- 172.21.15.167:0/1181766690 learned_addr learned my addr 172.21.15.167:0/1181766690 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T08:06:13.551 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:13.550+0000 7ff55a7fc640 1 --2- 172.21.15.167:0/1181766690 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff55c0d40a0 0x7ff55c15fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:13.551 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:13.550+0000 7ff55b7fe640 1 -- 172.21.15.167:0/1181766690 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff55c0d40a0 msgr2=0x7ff55c15fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:13.551 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:13.550+0000 7ff55b7fe640 1 --2- 172.21.15.167:0/1181766690 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff55c0d40a0 0x7ff55c15fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:13.551 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:13.550+0000 7ff559ffb640 1 -- 172.21.15.167:0/1181766690 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4237770065 0 0) 0x7ff55c06dcc0 con 0x7ff55c15f910 2024-05-11T08:06:13.551 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:13.550+0000 7ff55b7fe640 1 -- 172.21.15.167:0/1181766690 >> v1:172.21.15.167:6789/0 conn(0x7ff55c15f910 legacy=0x7ff55c06c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:13.551 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:13.550+0000 7ff55b7fe640 1 -- 172.21.15.167:0/1181766690 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff55c06d420 con 0x7ff55c160230 2024-05-11T08:06:13.551 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:13.550+0000 7ff55a7fc640 1 --2- 172.21.15.167:0/1181766690 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff55c0d40a0 0x7ff55c15fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T08:06:13.552 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:13.551+0000 7ff55b7fe640 1 --2- 172.21.15.167:0/1181766690 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff55c160230 0x7ff55c168ec0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff554009d90 tx=0x7ff55402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86b0d9c16e8f4c23 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:13.552 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:13.551+0000 7ff559ffb640 1 -- 172.21.15.167:0/1181766690 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff554007740 con 0x7ff55c160230 2024-05-11T08:06:13.552 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:13.551+0000 7ff559ffb640 1 -- 172.21.15.167:0/1181766690 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5540078e0 con 0x7ff55c160230 2024-05-11T08:06:13.553 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:13.551+0000 7ff559ffb640 1 -- 172.21.15.167:0/1181766690 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff554007cb0 con 0x7ff55c160230 2024-05-11T08:06:13.553 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:13.552+0000 7ff5613dd640 1 -- 172.21.15.167:0/1181766690 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff55c160230 msgr2=0x7ff55c168ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:13.553 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:13.552+0000 7ff5613dd640 1 --2- 172.21.15.167:0/1181766690 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff55c160230 0x7ff55c168ec0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff554009d90 tx=0x7ff55402a660 comp rx=0 tx=0).stop 2024-05-11T08:06:13.553 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:13.552+0000 7ff5613dd640 1 -- 172.21.15.167:0/1181766690 shutdown_connections 2024-05-11T08:06:13.553 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:13.552+0000 7ff5613dd640 1 --2- 172.21.15.167:0/1181766690 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff55c160230 0x7ff55c168ec0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:13.553 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:13.552+0000 7ff5613dd640 1 --2- 172.21.15.167:0/1181766690 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff55c0d40a0 0x7ff55c15fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:13.553 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:13.552+0000 7ff5613dd640 1 -- 172.21.15.167:0/1181766690 >> 172.21.15.167:0/1181766690 conn(0x7ff55c0d51e0 msgr2=0x7ff55c0d55f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:13.553 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:13.552+0000 7ff5613dd640 1 -- 172.21.15.167:0/1181766690 shutdown_connections 2024-05-11T08:06:13.553 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:13.552+0000 7ff5613dd640 1 -- 172.21.15.167:0/1181766690 wait complete. 2024-05-11T08:06:13.555 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:13.553+0000 7ff5613dd640 1 Processor -- start 2024-05-11T08:06:13.555 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:13.553+0000 7ff5613dd640 1 -- start start 2024-05-11T08:06:13.555 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:13.554+0000 7ff5613dd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff55c0d40a0 0x7ff55c147100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:13.555 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:13.554+0000 7ff5613dd640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff55c15f910 0x7ff55c1f4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:13.555 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:13.554+0000 7ff5613dd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff55c160230 0x7ff55c15e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:13.555 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:13.554+0000 7ff5613dd640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff55c1606c0 con 0x7ff55c160230 2024-05-11T08:06:13.555 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:13.554+0000 7ff5613dd640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff55c06d9e0 con 0x7ff55c0d40a0 2024-05-11T08:06:13.555 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:13.554+0000 7ff5613dd640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7ff55c06ce00 con 0x7ff55c15f910 2024-05-11T08:06:13.556 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:13.554+0000 7ff55b7fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff55c160230 0x7ff55c15e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:13.556 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:13.554+0000 7ff55b7fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff55c160230 0x7ff55c15e5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:45744/0 (socket says 172.21.15.167:45744) 2024-05-11T08:06:13.556 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:13.554+0000 7ff55b7fe640 1 -- 172.21.15.167:0/3222127333 learned_addr learned my addr 172.21.15.167:0/3222127333 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:13.556 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:13.554+0000 7ff55a7fc640 1 --2- 172.21.15.167:0/3222127333 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff55c15f910 0x7ff55c1f4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:13.556 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:13.554+0000 7ff55affd640 1 --2- 172.21.15.167:0/3222127333 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff55c0d40a0 0x7ff55c147100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:13.556 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:13.554+0000 7ff55b7fe640 1 -- 172.21.15.167:0/3222127333 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff55c15f910 msgr2=0x7ff55c1f4770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:13.556 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:13.554+0000 7ff55b7fe640 1 --2- 172.21.15.167:0/3222127333 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff55c15f910 0x7ff55c1f4770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:13.556 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:13.554+0000 7ff55b7fe640 1 -- 172.21.15.167:0/3222127333 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff55c0d40a0 msgr2=0x7ff55c147100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:13.556 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:13.554+0000 7ff55b7fe640 1 --2- 172.21.15.167:0/3222127333 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff55c0d40a0 0x7ff55c147100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:13.556 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:13.554+0000 7ff55b7fe640 1 -- 172.21.15.167:0/3222127333 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff55c147640 con 0x7ff55c160230 2024-05-11T08:06:13.556 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:13.555+0000 7ff55b7fe640 1 --2- 172.21.15.167:0/3222127333 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff55c160230 0x7ff55c15e5a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff554009eb0 tx=0x7ff554004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:13.556 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:13.555+0000 7ff5437fe640 1 -- 172.21.15.167:0/3222127333 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff554040070 con 0x7ff55c160230 2024-05-11T08:06:13.556 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:13.555+0000 7ff5437fe640 1 -- 172.21.15.167:0/3222127333 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff554031c40 con 0x7ff55c160230 2024-05-11T08:06:13.556 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:13.555+0000 7ff5437fe640 1 -- 172.21.15.167:0/3222127333 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff554003990 con 0x7ff55c160230 2024-05-11T08:06:13.556 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:13.555+0000 7ff5613dd640 1 -- 172.21.15.167:0/3222127333 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff55c0cc500 con 0x7ff55c160230 2024-05-11T08:06:13.557 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:13.555+0000 7ff5613dd640 1 -- 172.21.15.167:0/3222127333 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff55c0ccac0 con 0x7ff55c160230 2024-05-11T08:06:13.557 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:13.556+0000 7ff5613dd640 1 -- 172.21.15.167:0/3222127333 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff55c069520 con 0x7ff55c160230 2024-05-11T08:06:13.557 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:13.556+0000 7ff5437fe640 1 -- 172.21.15.167:0/3222127333 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7ff554039040 con 0x7ff55c160230 2024-05-11T08:06:13.558 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:13.557+0000 7ff5437fe640 1 --2- 172.21.15.167:0/3222127333 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7ff5480762d0 0x7ff548078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:13.558 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:13.557+0000 7ff5437fe640 1 -- 172.21.15.167:0/3222127333 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7ff5540b87a0 con 0x7ff55c160230 2024-05-11T08:06:13.558 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:13.557+0000 7ff55affd640 1 --2- 172.21.15.167:0/3222127333 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7ff5480762d0 0x7ff548078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:13.558 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:13.557+0000 7ff55affd640 1 --2- 172.21.15.167:0/3222127333 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7ff5480762d0 0x7ff548078790 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff54c000f60 tx=0x7ff54c007cf0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:13.561 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:13.561+0000 7ff5437fe640 1 -- 172.21.15.167:0/3222127333 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7ff554081800 con 0x7ff55c160230 2024-05-11T08:06:13.779 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:13.779+0000 7ff5613dd640 1 -- 172.21.15.167:0/3222127333 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7ff55c15ffc0 con 0x7ff55c160230 2024-05-11T08:06:14.434 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.434+0000 7ff5437fe640 1 -- 172.21.15.167:0/3222127333 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v9) ==== 122+0+0 (secure 0 0 0) 0x7ff554085b00 con 0x7ff55c160230 2024-05-11T08:06:14.438 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.437+0000 7ff5613dd640 1 -- 172.21.15.167:0/3222127333 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7ff5480762d0 msgr2=0x7ff548078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:14.438 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.437+0000 7ff5613dd640 1 --2- 172.21.15.167:0/3222127333 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7ff5480762d0 0x7ff548078790 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff54c000f60 tx=0x7ff54c007cf0 comp rx=0 tx=0).stop 2024-05-11T08:06:14.438 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.437+0000 7ff5613dd640 1 -- 172.21.15.167:0/3222127333 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff55c160230 msgr2=0x7ff55c15e5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:14.438 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.437+0000 7ff5613dd640 1 --2- 172.21.15.167:0/3222127333 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff55c160230 0x7ff55c15e5a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff554009eb0 tx=0x7ff554004a40 comp rx=0 tx=0).stop 2024-05-11T08:06:14.438 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.437+0000 7ff5613dd640 1 -- 172.21.15.167:0/3222127333 shutdown_connections 2024-05-11T08:06:14.438 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.437+0000 7ff5613dd640 1 --2- 172.21.15.167:0/3222127333 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7ff5480762d0 0x7ff548078790 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:14.438 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.437+0000 7ff5613dd640 1 --2- 172.21.15.167:0/3222127333 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff55c160230 0x7ff55c15e5a0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:14.438 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.437+0000 7ff5613dd640 1 --2- 172.21.15.167:0/3222127333 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff55c15f910 0x7ff55c1f4770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:14.438 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.437+0000 7ff5613dd640 1 --2- 172.21.15.167:0/3222127333 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff55c0d40a0 0x7ff55c147100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:14.438 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.437+0000 7ff5613dd640 1 -- 172.21.15.167:0/3222127333 >> 172.21.15.167:0/3222127333 conn(0x7ff55c0d51e0 msgr2=0x7ff55c0b83d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:14.438 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.437+0000 7ff5613dd640 1 -- 172.21.15.167:0/3222127333 shutdown_connections 2024-05-11T08:06:14.439 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.437+0000 7ff5613dd640 1 -- 172.21.15.167:0/3222127333 wait complete. 2024-05-11T08:06:14.449 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-11T08:06:14.449 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-11T08:06:14.549 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.549+0000 7f39eb81d640 1 Processor -- start 2024-05-11T08:06:14.549 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.549+0000 7f39eb81d640 1 -- start start 2024-05-11T08:06:14.552 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.550+0000 7f39eb81d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f39e40d2cf0 0x7f39e40cafb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:14.552 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.550+0000 7f39eb81d640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39e40d1190 con 0x7f39e41598d0 2024-05-11T08:06:14.552 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.550+0000 7f39eb81d640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39e406d500 con 0x7f39e40cb4f0 2024-05-11T08:06:14.552 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.550+0000 7f39eb81d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f39e40d3150 con 0x7f39e40d2cf0 2024-05-11T08:06:14.552 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.550+0000 7f39e95b9640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f39e41598d0 0x7f39e40d0640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:60722/0 (socket says 172.21.15.167:60722) 2024-05-11T08:06:14.552 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.550+0000 7f39e95b9640 1 -- 172.21.15.167:0/65856252 learned_addr learned my addr 172.21.15.167:0/65856252 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T08:06:14.552 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.550+0000 7f39e8db8640 1 --2- 172.21.15.167:0/65856252 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f39e40d2cf0 0x7f39e40cafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:14.552 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.550+0000 7f39e3fff640 1 -- 172.21.15.167:0/65856252 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 93195388 0 0) 0x7f39e40d1190 con 0x7f39e41598d0 2024-05-11T08:06:14.552 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.550+0000 7f39e3fff640 1 -- 172.21.15.167:0/65856252 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f39cc003520 con 0x7f39e41598d0 2024-05-11T08:06:14.553 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.550+0000 7f39e3fff640 1 -- 172.21.15.167:0/65856252 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 230515745 0 0) 0x7f39e406d500 con 0x7f39e40cb4f0 2024-05-11T08:06:14.553 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.551+0000 7f39e3fff640 1 -- 172.21.15.167:0/65856252 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f39e40d1190 con 0x7f39e40cb4f0 2024-05-11T08:06:14.553 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.551+0000 7f39e3fff640 1 -- 172.21.15.167:0/65856252 <== mon.0 v1:172.21.15.167:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3376910187 0 0) 0x7f39cc003520 con 0x7f39e41598d0 2024-05-11T08:06:14.553 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.551+0000 7f39e3fff640 1 -- 172.21.15.167:0/65856252 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f39e40d2cf0 msgr2=0x7f39e40cafb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:14.553 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.551+0000 7f39e3fff640 1 --2- 172.21.15.167:0/65856252 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f39e40d2cf0 0x7f39e40cafb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:14.553 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.551+0000 7f39e3fff640 1 -- 172.21.15.167:0/65856252 >> v1:172.21.15.204:6789/0 conn(0x7f39e40cb4f0 legacy=0x7f39e406dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:14.553 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.551+0000 7f39e3fff640 1 -- 172.21.15.167:0/65856252 --> v1:172.21.15.167:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39e406d7e0 con 0x7f39e41598d0 2024-05-11T08:06:14.553 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.551+0000 7f39e3fff640 1 -- 172.21.15.167:0/65856252 <== mon.0 v1:172.21.15.167:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3602277846 0 0) 0x7f39dc0016f0 con 0x7f39e41598d0 2024-05-11T08:06:14.553 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.551+0000 7f39e3fff640 1 -- 172.21.15.167:0/65856252 >> v1:172.21.15.167:6789/0 conn(0x7f39e41598d0 legacy=0x7f39e40d0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:14.553 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.551+0000 7f39e3fff640 1 --2- 172.21.15.167:0/65856252 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f39e40d2cf0 0x7f39cc003c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:14.553 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.551+0000 7f39e3fff640 1 --2- 172.21.15.167:0/65856252 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39cc0041c0 0x7f39cc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:14.553 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.551+0000 7f39e3fff640 1 --2- 172.21.15.167:0/65856252 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f39cc004d40 0x7f39cc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:14.553 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.551+0000 7f39e3fff640 1 -- 172.21.15.167:0/65856252 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f39cc00bb50 con 0x7f39cc004d40 2024-05-11T08:06:14.553 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.551+0000 7f39e3fff640 1 -- 172.21.15.167:0/65856252 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f39cc00bd00 con 0x7f39cc0041c0 2024-05-11T08:06:14.553 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.551+0000 7f39e3fff640 1 -- 172.21.15.167:0/65856252 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f39cc00bf00 con 0x7f39e40d2cf0 2024-05-11T08:06:14.553 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.551+0000 7f39e9dba640 1 --2- 172.21.15.167:0/65856252 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f39cc004d40 0x7f39cc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:14.554 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.551+0000 7f39e95b9640 1 --2- 172.21.15.167:0/65856252 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f39e40d2cf0 0x7f39cc003c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:14.554 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.551+0000 7f39e8db8640 1 --2- 172.21.15.167:0/65856252 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39cc0041c0 0x7f39cc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:14.554 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.551+0000 7f39e9dba640 1 -- 172.21.15.167:0/65856252 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f39e40d2cf0 msgr2=0x7f39cc003c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:14.554 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.551+0000 7f39e9dba640 1 --2- 172.21.15.167:0/65856252 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f39e40d2cf0 0x7f39cc003c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:14.554 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.551+0000 7f39e9dba640 1 -- 172.21.15.167:0/65856252 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39cc0041c0 msgr2=0x7f39cc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:14.554 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.551+0000 7f39e9dba640 1 --2- 172.21.15.167:0/65856252 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39cc0041c0 0x7f39cc004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:14.554 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.551+0000 7f39e9dba640 1 -- 172.21.15.167:0/65856252 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39cc003520 con 0x7f39cc004d40 2024-05-11T08:06:14.554 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.552+0000 7f39e9dba640 1 --2- 172.21.15.167:0/65856252 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f39cc004d40 0x7f39cc0051c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f39d4002000 tx=0x7f39cc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d81d11afd8779b0b server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:14.554 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.552+0000 7f39e3fff640 1 -- 172.21.15.167:0/65856252 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39d4001c20 con 0x7f39cc004d40 2024-05-11T08:06:14.554 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.552+0000 7f39e3fff640 1 -- 172.21.15.167:0/65856252 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39d4006d70 con 0x7f39cc004d40 2024-05-11T08:06:14.554 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.552+0000 7f39e3fff640 1 -- 172.21.15.167:0/65856252 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39d4003bb0 con 0x7f39cc004d40 2024-05-11T08:06:14.557 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.552+0000 7f39eb81d640 1 -- 172.21.15.167:0/65856252 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f39cc004d40 msgr2=0x7f39cc0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:14.557 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.552+0000 7f39eb81d640 1 --2- 172.21.15.167:0/65856252 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f39cc004d40 0x7f39cc0051c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f39d4002000 tx=0x7f39cc005830 comp rx=0 tx=0).stop 2024-05-11T08:06:14.557 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.552+0000 7f39e9dba640 1 -- 172.21.15.167:0/65856252 reap_dead start 2024-05-11T08:06:14.557 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.553+0000 7f39eb81d640 1 -- 172.21.15.167:0/65856252 shutdown_connections 2024-05-11T08:06:14.557 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.553+0000 7f39eb81d640 1 -- 172.21.15.167:0/65856252 >> 172.21.15.167:0/65856252 conn(0x7f39e40d1ba0 msgr2=0x7f39e40d1fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:14.557 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.553+0000 7f39eb81d640 1 -- 172.21.15.167:0/65856252 shutdown_connections 2024-05-11T08:06:14.557 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.553+0000 7f39eb81d640 1 -- 172.21.15.167:0/65856252 wait complete. 2024-05-11T08:06:14.557 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.554+0000 7f39eb81d640 1 Processor -- start 2024-05-11T08:06:14.557 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.554+0000 7f39eb81d640 1 -- start start 2024-05-11T08:06:14.557 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.554+0000 7f39eb81d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f39cc004d40 0x7f39e415ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:14.557 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.554+0000 7f39eb81d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f39e40b9e10 0x7f39e40ba270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:14.557 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.555+0000 7f39eb81d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39e4148bd0 0x7f39e4149050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:14.557 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.555+0000 7f39eb81d640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f39e41607c0 con 0x7f39cc004d40 2024-05-11T08:06:14.557 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.555+0000 7f39eb81d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f39e40cb940 con 0x7f39e4148bd0 2024-05-11T08:06:14.558 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.555+0000 7f39eb81d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f39e4159cb0 con 0x7f39e40b9e10 2024-05-11T08:06:14.558 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.555+0000 7f39e95b9640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f39cc004d40 0x7f39e415ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:14.558 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.555+0000 7f39e95b9640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f39cc004d40 0x7f39e415ec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:54114/0 (socket says 172.21.15.167:54114) 2024-05-11T08:06:14.558 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.555+0000 7f39e95b9640 1 -- 172.21.15.167:0/3625270469 learned_addr learned my addr 172.21.15.167:0/3625270469 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:14.558 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.555+0000 7f39e8db8640 1 --2- 172.21.15.167:0/3625270469 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f39e40b9e10 0x7f39e40ba270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:14.558 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.555+0000 7f39e9dba640 1 --2- 172.21.15.167:0/3625270469 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39e4148bd0 0x7f39e4149050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:14.558 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.555+0000 7f39e95b9640 1 -- 172.21.15.167:0/3625270469 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f39e40b9e10 msgr2=0x7f39e40ba270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:14.558 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.555+0000 7f39e95b9640 1 --2- 172.21.15.167:0/3625270469 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f39e40b9e10 0x7f39e40ba270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:14.558 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.555+0000 7f39e95b9640 1 -- 172.21.15.167:0/3625270469 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39e4148bd0 msgr2=0x7f39e4149050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:14.558 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.555+0000 7f39e95b9640 1 --2- 172.21.15.167:0/3625270469 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39e4148bd0 0x7f39e4149050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:14.558 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.555+0000 7f39e95b9640 1 -- 172.21.15.167:0/3625270469 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39e4146a90 con 0x7f39cc004d40 2024-05-11T08:06:14.558 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.555+0000 7f39e95b9640 1 --2- 172.21.15.167:0/3625270469 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f39cc004d40 0x7f39e415ec80 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f39dc0072f0 tx=0x7f39dc0076f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:14.558 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.555+0000 7f39e1ffb640 1 -- 172.21.15.167:0/3625270469 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39dc01a070 con 0x7f39cc004d40 2024-05-11T08:06:14.558 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.555+0000 7f39e1ffb640 1 -- 172.21.15.167:0/3625270469 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39dc00f9a0 con 0x7f39cc004d40 2024-05-11T08:06:14.558 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.555+0000 7f39eb81d640 1 -- 172.21.15.167:0/3625270469 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39e4146d00 con 0x7f39cc004d40 2024-05-11T08:06:14.559 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.555+0000 7f39e1ffb640 1 -- 172.21.15.167:0/3625270469 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39dc00fd70 con 0x7f39cc004d40 2024-05-11T08:06:14.559 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.555+0000 7f39eb81d640 1 -- 172.21.15.167:0/3625270469 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39e415f1c0 con 0x7f39cc004d40 2024-05-11T08:06:14.559 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.556+0000 7f39eb81d640 1 -- 172.21.15.167:0/3625270469 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39e4076450 con 0x7f39cc004d40 2024-05-11T08:06:14.559 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.557+0000 7f39e1ffb640 1 -- 172.21.15.167:0/3625270469 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f39dc01d070 con 0x7f39cc004d40 2024-05-11T08:06:14.559 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.557+0000 7f39e1ffb640 1 --2- 172.21.15.167:0/3625270469 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f39d0076300 0x7f39d00787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:14.559 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.557+0000 7f39e1ffb640 1 -- 172.21.15.167:0/3625270469 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f39dc097470 con 0x7f39cc004d40 2024-05-11T08:06:14.559 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.557+0000 7f39e8db8640 1 --2- 172.21.15.167:0/3625270469 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f39d0076300 0x7f39d00787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:14.560 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.558+0000 7f39e8db8640 1 --2- 172.21.15.167:0/3625270469 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f39d0076300 0x7f39d00787c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f39e41590e0 tx=0x7f39d8009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:14.561 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.562+0000 7f39e1ffb640 1 -- 172.21.15.167:0/3625270469 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f39e4076450 con 0x7f39cc004d40 2024-05-11T08:06:14.780 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.780+0000 7f39eb81d640 1 -- 172.21.15.167:0/3625270469 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f39e40cb2e0 con 0x7f39cc004d40 2024-05-11T08:06:14.781 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.781+0000 7f39e1ffb640 1 -- 172.21.15.167:0/3625270469 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5183 (secure 0 0 0) 0x7f39e40cb2e0 con 0x7f39cc004d40 2024-05-11T08:06:14.781 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T08:06:14.781 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6834/506789040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":506789040},{"type":"v1","addr":"172.21.15.204:6834","nonce":506789040}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4287,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1448126000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":1448126000},{"type":"v1","addr":"172.21.15.167:6837","nonce":1448126000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4290,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/766834562","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":766834562},{"type":"v1","addr":"172.21.15.167:6836","nonce":766834562}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-11T08:05:54.711448+0000","modified":"2024-05-11T08:06:13.780553+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4305},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4305":{"gid":4305,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.204:6835/1247774569","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6833","nonce":1247774569},{"type":"v1","addr":"172.21.15.204:6835","nonce":1247774569}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4305,"qdb_cluster":[4305]},"id":1}]} 2024-05-11T08:06:14.781 INFO:teuthology.orchestra.run.smithi167.stderr:dumped fsmap epoch 9 2024-05-11T08:06:14.783 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.783+0000 7f39eb81d640 1 -- 172.21.15.167:0/3625270469 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f39d0076300 msgr2=0x7f39d00787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:14.783 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.783+0000 7f39eb81d640 1 --2- 172.21.15.167:0/3625270469 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f39d0076300 0x7f39d00787c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f39e41590e0 tx=0x7f39d8009190 comp rx=0 tx=0).stop 2024-05-11T08:06:14.783 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.783+0000 7f39eb81d640 1 -- 172.21.15.167:0/3625270469 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f39cc004d40 msgr2=0x7f39e415ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:14.783 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.783+0000 7f39eb81d640 1 --2- 172.21.15.167:0/3625270469 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f39cc004d40 0x7f39e415ec80 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f39dc0072f0 tx=0x7f39dc0076f0 comp rx=0 tx=0).stop 2024-05-11T08:06:14.784 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.783+0000 7f39eb81d640 1 -- 172.21.15.167:0/3625270469 shutdown_connections 2024-05-11T08:06:14.784 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.783+0000 7f39eb81d640 1 --2- 172.21.15.167:0/3625270469 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f39d0076300 0x7f39d00787c0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:14.784 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.783+0000 7f39eb81d640 1 --2- 172.21.15.167:0/3625270469 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39e4148bd0 0x7f39e4149050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:14.784 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.783+0000 7f39eb81d640 1 --2- 172.21.15.167:0/3625270469 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f39e40b9e10 0x7f39e40ba270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:14.784 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.783+0000 7f39eb81d640 1 --2- 172.21.15.167:0/3625270469 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f39cc004d40 0x7f39e415ec80 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:14.784 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.783+0000 7f39eb81d640 1 -- 172.21.15.167:0/3625270469 >> 172.21.15.167:0/3625270469 conn(0x7f39e40d1ba0 msgr2=0x7f39e41609d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:14.784 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.783+0000 7f39eb81d640 1 -- 172.21.15.167:0/3625270469 shutdown_connections 2024-05-11T08:06:14.784 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.783+0000 7f39eb81d640 1 -- 172.21.15.167:0/3625270469 wait complete. 2024-05-11T08:06:14.794 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-11T08:06:14.895 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.894+0000 7f590fda5640 1 Processor -- start 2024-05-11T08:06:14.895 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.895+0000 7f590fda5640 1 -- start start 2024-05-11T08:06:14.897 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.895+0000 7f590fda5640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f59081488a0 0x7f590815f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:14.897 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.895+0000 7f590fda5640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f590815fe50 0x7f59080b5a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:14.898 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.895+0000 7f590fda5640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f59080b5f50 0x7f59080710f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:14.898 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.895+0000 7f590fda5640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f59080b5580 con 0x7f59080b5f50 2024-05-11T08:06:14.898 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.895+0000 7f590fda5640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f5908163e60 con 0x7f590815fe50 2024-05-11T08:06:14.898 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.895+0000 7f590fda5640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f59080b4f70 con 0x7f59081488a0 2024-05-11T08:06:14.898 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.896+0000 7f590e342640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f59080b5f50 0x7f59080710f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:14.898 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.896+0000 7f590e342640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f59080b5f50 0x7f59080710f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:54120/0 (socket says 172.21.15.167:54120) 2024-05-11T08:06:14.898 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.896+0000 7f590e342640 1 -- 172.21.15.167:0/1301461305 learned_addr learned my addr 172.21.15.167:0/1301461305 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:14.898 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.896+0000 7f590d340640 1 --2- 172.21.15.167:0/1301461305 >> v2:172.21.15.204:3300/0 conn(0x7f590815fe50 0x7f59080b5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:14.898 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.896+0000 7f590db41640 1 --2- 172.21.15.167:0/1301461305 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f59081488a0 0x7f590815f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:14.898 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.896+0000 7f590e342640 1 -- 172.21.15.167:0/1301461305 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f59081488a0 msgr2=0x7f590815f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:14.898 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.896+0000 7f590e342640 1 --2- 172.21.15.167:0/1301461305 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f59081488a0 0x7f590815f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:14.898 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.896+0000 7f590e342640 1 -- 172.21.15.167:0/1301461305 >> v2:172.21.15.204:3300/0 conn(0x7f590815fe50 msgr2=0x7f59080b5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:14.898 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.896+0000 7f590e342640 1 --2- 172.21.15.167:0/1301461305 >> v2:172.21.15.204:3300/0 conn(0x7f590815fe50 0x7f59080b5a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:14.898 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.896+0000 7f590e342640 1 -- 172.21.15.167:0/1301461305 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5908163b80 con 0x7f59080b5f50 2024-05-11T08:06:14.899 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.896+0000 7f590db41640 1 --2- 172.21.15.167:0/1301461305 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f59081488a0 0x7f590815f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T08:06:14.899 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.897+0000 7f590e342640 1 --2- 172.21.15.167:0/1301461305 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f59080b5f50 0x7f59080710f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f59080d2cb0 tx=0x7f590002a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55815f7c6b43adcb server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:14.899 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.897+0000 7f590cb3f640 1 -- 172.21.15.167:0/1301461305 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5900036070 con 0x7f59080b5f50 2024-05-11T08:06:14.899 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.897+0000 7f590cb3f640 1 -- 172.21.15.167:0/1301461305 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5900031600 con 0x7f59080b5f50 2024-05-11T08:06:14.899 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.897+0000 7f590cb3f640 1 -- 172.21.15.167:0/1301461305 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59000319d0 con 0x7f59080b5f50 2024-05-11T08:06:14.899 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.897+0000 7f590fda5640 1 -- 172.21.15.167:0/1301461305 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f59080b5f50 msgr2=0x7f59080710f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:14.899 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.897+0000 7f590fda5640 1 --2- 172.21.15.167:0/1301461305 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f59080b5f50 0x7f59080710f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f59080d2cb0 tx=0x7f590002a610 comp rx=0 tx=0).stop 2024-05-11T08:06:14.899 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.897+0000 7f590fda5640 1 -- 172.21.15.167:0/1301461305 shutdown_connections 2024-05-11T08:06:14.899 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.897+0000 7f590fda5640 1 --2- 172.21.15.167:0/1301461305 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f59080b5f50 0x7f59080710f0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:14.899 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.897+0000 7f590fda5640 1 --2- 172.21.15.167:0/1301461305 >> v2:172.21.15.204:3300/0 conn(0x7f590815fe50 0x7f59080b5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:14.899 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.897+0000 7f590fda5640 1 --2- 172.21.15.167:0/1301461305 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f59081488a0 0x7f590815f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:14.899 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.897+0000 7f590fda5640 1 -- 172.21.15.167:0/1301461305 >> 172.21.15.167:0/1301461305 conn(0x7f59080d2550 msgr2=0x7f59080ce890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:14.899 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.897+0000 7f590fda5640 1 -- 172.21.15.167:0/1301461305 shutdown_connections 2024-05-11T08:06:14.899 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.897+0000 7f590fda5640 1 -- 172.21.15.167:0/1301461305 wait complete. 2024-05-11T08:06:14.901 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.899+0000 7f590fda5640 1 Processor -- start 2024-05-11T08:06:14.901 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.899+0000 7f590fda5640 1 -- start start 2024-05-11T08:06:14.901 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.899+0000 7f590fda5640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f59080b5f50 0x7f59081f93e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:14.902 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.899+0000 7f590fda5640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f59081488a0 0x7f59080ac3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:14.902 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.899+0000 7f590fda5640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f590815fe50 0x7f59080ad260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:14.902 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.899+0000 7f590fda5640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5908148e00 con 0x7f59080b5f50 2024-05-11T08:06:14.902 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.899+0000 7f590fda5640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5908148c80 con 0x7f590815fe50 2024-05-11T08:06:14.902 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.899+0000 7f590fda5640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f5908071630 con 0x7f59081488a0 2024-05-11T08:06:14.902 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.899+0000 7f590db41640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f59080b5f50 0x7f59081f93e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:14.902 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.899+0000 7f590db41640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f59080b5f50 0x7f59081f93e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:54132/0 (socket says 172.21.15.167:54132) 2024-05-11T08:06:14.902 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.899+0000 7f590db41640 1 -- 172.21.15.167:0/2480905132 learned_addr learned my addr 172.21.15.167:0/2480905132 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:14.902 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.900+0000 7f590d340640 1 --2- 172.21.15.167:0/2480905132 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f59081488a0 0x7f59080ac3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:14.902 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.900+0000 7f590e342640 1 --2- 172.21.15.167:0/2480905132 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f590815fe50 0x7f59080ad260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:14.902 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.900+0000 7f590db41640 1 -- 172.21.15.167:0/2480905132 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f59081488a0 msgr2=0x7f59080ac3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:14.902 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.900+0000 7f590db41640 1 --2- 172.21.15.167:0/2480905132 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f59081488a0 0x7f59080ac3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:14.902 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.900+0000 7f590db41640 1 -- 172.21.15.167:0/2480905132 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f590815fe50 msgr2=0x7f59080ad260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:14.903 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.900+0000 7f590db41640 1 --2- 172.21.15.167:0/2480905132 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f590815fe50 0x7f59080ad260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:14.903 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.900+0000 7f590db41640 1 -- 172.21.15.167:0/2480905132 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59081ea660 con 0x7f59080b5f50 2024-05-11T08:06:14.903 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.900+0000 7f590e342640 1 --2- 172.21.15.167:0/2480905132 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f590815fe50 0x7f59080ad260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T08:06:14.903 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.900+0000 7f590db41640 1 --2- 172.21.15.167:0/2480905132 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f59080b5f50 0x7f59081f93e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f58fc00e760 tx=0x7f58fc00ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:14.903 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.900+0000 7f58fa7fc640 1 -- 172.21.15.167:0/2480905132 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58fc00f040 con 0x7f59080b5f50 2024-05-11T08:06:14.903 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.900+0000 7f58fa7fc640 1 -- 172.21.15.167:0/2480905132 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58fc004230 con 0x7f59080b5f50 2024-05-11T08:06:14.903 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.900+0000 7f590fda5640 1 -- 172.21.15.167:0/2480905132 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5908143c40 con 0x7f59080b5f50 2024-05-11T08:06:14.903 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.900+0000 7f58fa7fc640 1 -- 172.21.15.167:0/2480905132 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58fc00ee70 con 0x7f59080b5f50 2024-05-11T08:06:14.903 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.900+0000 7f590fda5640 1 -- 172.21.15.167:0/2480905132 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59080ab8e0 con 0x7f59080b5f50 2024-05-11T08:06:14.903 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.901+0000 7f590fda5640 1 -- 172.21.15.167:0/2480905132 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5908069590 con 0x7f59080b5f50 2024-05-11T08:06:14.903 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.902+0000 7f58fa7fc640 1 -- 172.21.15.167:0/2480905132 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f58fc00c6d0 con 0x7f59080b5f50 2024-05-11T08:06:14.904 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.902+0000 7f58fa7fc640 1 --2- 172.21.15.167:0/2480905132 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f58ec0762d0 0x7f58ec078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:14.904 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.902+0000 7f58fa7fc640 1 -- 172.21.15.167:0/2480905132 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f58fc0975a0 con 0x7f59080b5f50 2024-05-11T08:06:14.904 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.902+0000 7f590d340640 1 --2- 172.21.15.167:0/2480905132 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f58ec0762d0 0x7f58ec078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:14.904 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.903+0000 7f590d340640 1 --2- 172.21.15.167:0/2480905132 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f58ec0762d0 0x7f58ec078790 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5904006fd0 tx=0x7f5904008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:14.906 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:14.906+0000 7f58fa7fc640 1 -- 172.21.15.167:0/2480905132 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f58fc017020 con 0x7f59080b5f50 2024-05-11T08:06:15.091 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.091+0000 7f590fda5640 1 -- 172.21.15.167:0/2480905132 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f59080d1220 con 0x7f59080b5f50 2024-05-11T08:06:15.092 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.092+0000 7f58fa7fc640 1 -- 172.21.15.167:0/2480905132 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18562 (secure 0 0 0) 0x7f58fc060c50 con 0x7f59080b5f50 2024-05-11T08:06:15.093 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T08:06:15.093 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":27,"fsid":"2034778d-8ff1-4823-aa44-ded32b518241","created":"2024-05-11T08:05:31.612421+0000","modified":"2024-05-11T08:05:59.756806+0000","last_up_change":"2024-05-11T08:05:42.557382+0000","last_in_change":"2024-05-11T08:05:33.757311+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-11T08:05:43.931263+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-11T08:05:52.573726+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-11T08:05:53.670746+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-11T08:05:57.380272+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"587eaced-5ded-406d-884b-30c1caa8686b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":3863307356},{"type":"v1","addr":"172.21.15.167:6806","nonce":3863307356}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":3863307356},{"type":"v1","addr":"172.21.15.167:6810","nonce":3863307356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":3863307356},{"type":"v1","addr":"172.21.15.167:6815","nonce":3863307356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":3863307356},{"type":"v1","addr":"172.21.15.167:6813","nonce":3863307356}]},"public_addr":"172.21.15.167:6806/3863307356","cluster_addr":"172.21.15.167:6810/3863307356","heartbeat_back_addr":"172.21.15.167:6815/3863307356","heartbeat_front_addr":"172.21.15.167:6813/3863307356","state":["exists","up"]},{"osd":1,"uuid":"37c59104-2dd0-4cf9-85ea-56636f6b02a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":1585024048},{"type":"v1","addr":"172.21.15.167:6817","nonce":1585024048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1585024048},{"type":"v1","addr":"172.21.15.167:6819","nonce":1585024048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1585024048},{"type":"v1","addr":"172.21.15.167:6823","nonce":1585024048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":1585024048},{"type":"v1","addr":"172.21.15.167:6821","nonce":1585024048}]},"public_addr":"172.21.15.167:6817/1585024048","cluster_addr":"172.21.15.167:6819/1585024048","heartbeat_back_addr":"172.21.15.167:6823/1585024048","heartbeat_front_addr":"172.21.15.167:6821/1585024048","state":["exists","up"]},{"osd":2,"uuid":"2bb4f7f5-c698-451b-b4bb-78e716921291","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":3242166900},{"type":"v1","addr":"172.21.15.167:6801","nonce":3242166900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":3242166900},{"type":"v1","addr":"172.21.15.167:6803","nonce":3242166900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6809","nonce":3242166900},{"type":"v1","addr":"172.21.15.167:6811","nonce":3242166900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6805","nonce":3242166900},{"type":"v1","addr":"172.21.15.167:6807","nonce":3242166900}]},"public_addr":"172.21.15.167:6801/3242166900","cluster_addr":"172.21.15.167:6803/3242166900","heartbeat_back_addr":"172.21.15.167:6811/3242166900","heartbeat_front_addr":"172.21.15.167:6807/3242166900","state":["exists","up"]},{"osd":3,"uuid":"9dd26759-a338-49d2-8103-45d068b9c000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":453689293},{"type":"v1","addr":"172.21.15.167:6825","nonce":453689293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":453689293},{"type":"v1","addr":"172.21.15.167:6827","nonce":453689293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":453689293},{"type":"v1","addr":"172.21.15.167:6831","nonce":453689293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":453689293},{"type":"v1","addr":"172.21.15.167:6829","nonce":453689293}]},"public_addr":"172.21.15.167:6825/453689293","cluster_addr":"172.21.15.167:6827/453689293","heartbeat_back_addr":"172.21.15.167:6831/453689293","heartbeat_front_addr":"172.21.15.167:6829/453689293","state":["exists","up"]},{"osd":4,"uuid":"cb27250d-2a80-4c7b-8cef-3a7af05c8a16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1311831500},{"type":"v1","addr":"172.21.15.204:6801","nonce":1311831500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1311831500},{"type":"v1","addr":"172.21.15.204:6803","nonce":1311831500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1311831500},{"type":"v1","addr":"172.21.15.204:6807","nonce":1311831500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1311831500},{"type":"v1","addr":"172.21.15.204:6805","nonce":1311831500}]},"public_addr":"172.21.15.204:6801/1311831500","cluster_addr":"172.21.15.204:6803/1311831500","heartbeat_back_addr":"172.21.15.204:6807/1311831500","heartbeat_front_addr":"172.21.15.204:6805/1311831500","state":["exists","up"]},{"osd":5,"uuid":"92998541-ffec-430b-ab76-8c3cb7c2752d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2921847521},{"type":"v1","addr":"172.21.15.204:6825","nonce":2921847521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2921847521},{"type":"v1","addr":"172.21.15.204:6827","nonce":2921847521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2921847521},{"type":"v1","addr":"172.21.15.204:6831","nonce":2921847521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2921847521},{"type":"v1","addr":"172.21.15.204:6829","nonce":2921847521}]},"public_addr":"172.21.15.204:6825/2921847521","cluster_addr":"172.21.15.204:6827/2921847521","heartbeat_back_addr":"172.21.15.204:6831/2921847521","heartbeat_front_addr":"172.21.15.204:6829/2921847521","state":["exists","up"]},{"osd":6,"uuid":"9543479c-2ec5-42a8-8933-707bfed2e048","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":2889851077},{"type":"v1","addr":"172.21.15.204:6817","nonce":2889851077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":2889851077},{"type":"v1","addr":"172.21.15.204:6819","nonce":2889851077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":2889851077},{"type":"v1","addr":"172.21.15.204:6823","nonce":2889851077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":2889851077},{"type":"v1","addr":"172.21.15.204:6821","nonce":2889851077}]},"public_addr":"172.21.15.204:6817/2889851077","cluster_addr":"172.21.15.204:6819/2889851077","heartbeat_back_addr":"172.21.15.204:6823/2889851077","heartbeat_front_addr":"172.21.15.204:6821/2889851077","state":["exists","up"]},{"osd":7,"uuid":"c45a5647-8358-409e-af91-c05dda65f0b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":1949534184},{"type":"v1","addr":"172.21.15.204:6809","nonce":1949534184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":1949534184},{"type":"v1","addr":"172.21.15.204:6811","nonce":1949534184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1949534184},{"type":"v1","addr":"172.21.15.204:6815","nonce":1949534184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1949534184},{"type":"v1","addr":"172.21.15.204:6813","nonce":1949534184}]},"public_addr":"172.21.15.204:6809/1949534184","cluster_addr":"172.21.15.204:6811/1949534184","heartbeat_back_addr":"172.21.15.204:6815/1949534184","heartbeat_front_addr":"172.21.15.204:6813/1949534184","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.952253+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.940689+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.860682+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.929350+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.901627+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.957451+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.987249+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.951197+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-11T08:06:15.095 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.094+0000 7f590fda5640 1 -- 172.21.15.167:0/2480905132 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f58ec0762d0 msgr2=0x7f58ec078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:15.095 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.094+0000 7f590fda5640 1 --2- 172.21.15.167:0/2480905132 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f58ec0762d0 0x7f58ec078790 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5904006fd0 tx=0x7f5904008040 comp rx=0 tx=0).stop 2024-05-11T08:06:15.095 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.094+0000 7f590fda5640 1 -- 172.21.15.167:0/2480905132 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f59080b5f50 msgr2=0x7f59081f93e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:15.095 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.094+0000 7f590fda5640 1 --2- 172.21.15.167:0/2480905132 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f59080b5f50 0x7f59081f93e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f58fc00e760 tx=0x7f58fc00ec20 comp rx=0 tx=0).stop 2024-05-11T08:06:15.095 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.094+0000 7f590fda5640 1 -- 172.21.15.167:0/2480905132 shutdown_connections 2024-05-11T08:06:15.095 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.094+0000 7f590fda5640 1 --2- 172.21.15.167:0/2480905132 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f58ec0762d0 0x7f58ec078790 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:15.095 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.094+0000 7f590fda5640 1 --2- 172.21.15.167:0/2480905132 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f590815fe50 0x7f59080ad260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:15.095 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.095+0000 7f590fda5640 1 --2- 172.21.15.167:0/2480905132 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f59081488a0 0x7f59080ac3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:15.095 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.095+0000 7f590fda5640 1 --2- 172.21.15.167:0/2480905132 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f59080b5f50 0x7f59081f93e0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:15.095 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.095+0000 7f590fda5640 1 -- 172.21.15.167:0/2480905132 >> 172.21.15.167:0/2480905132 conn(0x7f59080d2550 msgr2=0x7f59080d05a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:15.096 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.095+0000 7f590fda5640 1 -- 172.21.15.167:0/2480905132 shutdown_connections 2024-05-11T08:06:15.096 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.095+0000 7f590fda5640 1 -- 172.21.15.167:0/2480905132 wait complete. 2024-05-11T08:06:15.106 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-11T08:06:15.106 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-11T08:06:15.207 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.207+0000 7f418ecb4640 1 Processor -- start 2024-05-11T08:06:15.208 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.207+0000 7f418ecb4640 1 -- start start 2024-05-11T08:06:15.209 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.208+0000 7f418ecb4640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f41881598d0 0x7f41880d2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:15.209 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.208+0000 7f418ecb4640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f41880d0640 0x7f41880d0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:15.209 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.208+0000 7f418ecb4640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41880d1190 con 0x7f41880cafb0 2024-05-11T08:06:15.209 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.208+0000 7f418ecb4640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f418806d500 con 0x7f41881598d0 2024-05-11T08:06:15.209 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.208+0000 7f418ecb4640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f418806d7e0 con 0x7f41880d0640 2024-05-11T08:06:15.209 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.208+0000 7f418d251640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f41880cafb0 0x7f418806dfe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:60728/0 (socket says 172.21.15.167:60728) 2024-05-11T08:06:15.209 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.208+0000 7f418ca50640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f41881598d0 0x7f41880d2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:15.209 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.208+0000 7f418d251640 1 -- 172.21.15.167:0/192345520 learned_addr learned my addr 172.21.15.167:0/192345520 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T08:06:15.209 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.209+0000 7f417ffff640 1 --2- 172.21.15.167:0/192345520 >> v2:172.21.15.204:3300/0 conn(0x7f41880d0640 0x7f41880d0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:15.209 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.209+0000 7f418ca50640 1 -- 172.21.15.167:0/192345520 >> v2:172.21.15.204:3300/0 conn(0x7f41880d0640 msgr2=0x7f41880d0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:15.209 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.209+0000 7f418ca50640 1 --2- 172.21.15.167:0/192345520 >> v2:172.21.15.204:3300/0 conn(0x7f41880d0640 0x7f41880d0ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:15.209 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.209+0000 7f418ca50640 1 -- 172.21.15.167:0/192345520 >> v1:172.21.15.167:6789/0 conn(0x7f41880cafb0 legacy=0x7f418806dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:15.210 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.209+0000 7f417f7fe640 1 -- 172.21.15.167:0/192345520 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1965621113 0 0) 0x7f41880d1190 con 0x7f41880cafb0 2024-05-11T08:06:15.210 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.209+0000 7f418ca50640 1 -- 172.21.15.167:0/192345520 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f418806dac0 con 0x7f41881598d0 2024-05-11T08:06:15.210 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.209+0000 7f417ffff640 1 --2- 172.21.15.167:0/192345520 >> v2:172.21.15.204:3300/0 conn(0x7f41880d0640 0x7f41880d0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T08:06:15.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.209+0000 7f418ca50640 1 --2- 172.21.15.167:0/192345520 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f41881598d0 0x7f41880d2cf0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f41880b4c50 tx=0x7f418402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c23fe28ed76c0f5 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:15.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.209+0000 7f417f7fe640 1 -- 172.21.15.167:0/192345520 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4184037070 con 0x7f41881598d0 2024-05-11T08:06:15.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.209+0000 7f417f7fe640 1 -- 172.21.15.167:0/192345520 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4184032440 con 0x7f41881598d0 2024-05-11T08:06:15.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.209+0000 7f417f7fe640 1 -- 172.21.15.167:0/192345520 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4184032810 con 0x7f41881598d0 2024-05-11T08:06:15.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.210+0000 7f418ecb4640 1 -- 172.21.15.167:0/192345520 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f41881598d0 msgr2=0x7f41880d2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:15.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.210+0000 7f418ecb4640 1 --2- 172.21.15.167:0/192345520 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f41881598d0 0x7f41880d2cf0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f41880b4c50 tx=0x7f418402a1a0 comp rx=0 tx=0).stop 2024-05-11T08:06:15.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.210+0000 7f418ecb4640 1 -- 172.21.15.167:0/192345520 shutdown_connections 2024-05-11T08:06:15.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.210+0000 7f418ecb4640 1 --2- 172.21.15.167:0/192345520 >> v2:172.21.15.204:3300/0 conn(0x7f41880d0640 0x7f41880d0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:15.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.210+0000 7f418ecb4640 1 --2- 172.21.15.167:0/192345520 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f41881598d0 0x7f41880d2cf0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:15.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.210+0000 7f418ecb4640 1 -- 172.21.15.167:0/192345520 >> 172.21.15.167:0/192345520 conn(0x7f41880d1ba0 msgr2=0x7f41880d1fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:15.212 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.210+0000 7f418ecb4640 1 -- 172.21.15.167:0/192345520 shutdown_connections 2024-05-11T08:06:15.212 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.210+0000 7f418ecb4640 1 -- 172.21.15.167:0/192345520 wait complete. 2024-05-11T08:06:15.212 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.211+0000 7f418ecb4640 1 Processor -- start 2024-05-11T08:06:15.212 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.211+0000 7f418ecb4640 1 -- start start 2024-05-11T08:06:15.214 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.212+0000 7f418ecb4640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f41880cafb0 0x7f418814d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:15.214 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.212+0000 7f418ecb4640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f41880d0640 0x7f41881f8740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:15.214 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.212+0000 7f418ecb4640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f41881598d0 0x7f41881f8c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:15.214 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.212+0000 7f418ecb4640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f418806d1b0 con 0x7f41880cafb0 2024-05-11T08:06:15.214 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.212+0000 7f418ecb4640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4188159cb0 con 0x7f41880d0640 2024-05-11T08:06:15.214 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.212+0000 7f418ecb4640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f41880cb3e0 con 0x7f41881598d0 2024-05-11T08:06:15.214 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.212+0000 7f418ca50640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f41880cafb0 0x7f418814d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:15.214 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.212+0000 7f418ca50640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f41880cafb0 0x7f418814d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:54156/0 (socket says 172.21.15.167:54156) 2024-05-11T08:06:15.214 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.212+0000 7f418ca50640 1 -- 172.21.15.167:0/1791741388 learned_addr learned my addr 172.21.15.167:0/1791741388 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:15.214 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.212+0000 7f418d251640 1 --2- 172.21.15.167:0/1791741388 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f41881598d0 0x7f41881f8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:15.214 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.212+0000 7f418ca50640 1 -- 172.21.15.167:0/1791741388 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f41881598d0 msgr2=0x7f41881f8c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:15.214 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.212+0000 7f417ffff640 1 --2- 172.21.15.167:0/1791741388 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f41880d0640 0x7f41881f8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:15.215 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.212+0000 7f418ca50640 1 --2- 172.21.15.167:0/1791741388 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f41881598d0 0x7f41881f8c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:15.215 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.212+0000 7f418ca50640 1 -- 172.21.15.167:0/1791741388 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f41880d0640 msgr2=0x7f41881f8740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:15.215 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.212+0000 7f418ca50640 1 --2- 172.21.15.167:0/1791741388 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f41880d0640 0x7f41881f8740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:15.215 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.212+0000 7f418ca50640 1 -- 172.21.15.167:0/1791741388 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f418814e030 con 0x7f41880cafb0 2024-05-11T08:06:15.215 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.213+0000 7f418ca50640 1 --2- 172.21.15.167:0/1791741388 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f41880cafb0 0x7f418814d550 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f4184035040 tx=0x7f41840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:15.215 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.213+0000 7f417d7fa640 1 -- 172.21.15.167:0/1791741388 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4184037050 con 0x7f41880cafb0 2024-05-11T08:06:15.215 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.213+0000 7f417d7fa640 1 -- 172.21.15.167:0/1791741388 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4184038040 con 0x7f41880cafb0 2024-05-11T08:06:15.215 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.213+0000 7f418ecb4640 1 -- 172.21.15.167:0/1791741388 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4188150e10 con 0x7f41880cafb0 2024-05-11T08:06:15.215 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.213+0000 7f417d7fa640 1 -- 172.21.15.167:0/1791741388 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4184004450 con 0x7f41880cafb0 2024-05-11T08:06:15.215 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.213+0000 7f418ecb4640 1 -- 172.21.15.167:0/1791741388 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41881513d0 con 0x7f41880cafb0 2024-05-11T08:06:15.215 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.214+0000 7f418ecb4640 1 -- 172.21.15.167:0/1791741388 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4188069590 con 0x7f41880cafb0 2024-05-11T08:06:15.216 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.214+0000 7f417d7fa640 1 -- 172.21.15.167:0/1791741388 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f4184030980 con 0x7f41880cafb0 2024-05-11T08:06:15.216 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.215+0000 7f417d7fa640 1 --2- 172.21.15.167:0/1791741388 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f41740762d0 0x7f4174078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:15.216 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.215+0000 7f417d7fa640 1 -- 172.21.15.167:0/1791741388 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f41840b8520 con 0x7f41880cafb0 2024-05-11T08:06:15.216 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.215+0000 7f417ffff640 1 --2- 172.21.15.167:0/1791741388 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f41740762d0 0x7f4174078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:15.216 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.215+0000 7f417ffff640 1 --2- 172.21.15.167:0/1791741388 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f41740762d0 0x7f4174078790 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f41881630e0 tx=0x7f4178008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:15.219 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.219+0000 7f417d7fa640 1 -- 172.21.15.167:0/1791741388 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f4184081b40 con 0x7f41880cafb0 2024-05-11T08:06:15.399 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.398+0000 7f418ecb4640 1 -- 172.21.15.167:0/1791741388 --> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f41881573e0 con 0x7f41740762d0 2024-05-11T08:06:15.408 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.407+0000 7f417d7fa640 1 -- 172.21.15.167:0/1791741388 <== mgr.4113 v2:172.21.15.167:6832/253569496 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332523 (secure 0 0 0) 0x7f41881573e0 con 0x7f41740762d0 2024-05-11T08:06:15.412 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T08:06:15.414 INFO:teuthology.orchestra.run.smithi167.stderr:dumped all 2024-05-11T08:06:15.415 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.411+0000 7f418ecb4640 1 -- 172.21.15.167:0/1791741388 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f41740762d0 msgr2=0x7f4174078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:15.416 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.411+0000 7f418ecb4640 1 --2- 172.21.15.167:0/1791741388 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f41740762d0 0x7f4174078790 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f41881630e0 tx=0x7f4178008040 comp rx=0 tx=0).stop 2024-05-11T08:06:15.416 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.411+0000 7f418ecb4640 1 -- 172.21.15.167:0/1791741388 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f41880cafb0 msgr2=0x7f418814d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:15.416 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.411+0000 7f418ecb4640 1 --2- 172.21.15.167:0/1791741388 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f41880cafb0 0x7f418814d550 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f4184035040 tx=0x7f41840046c0 comp rx=0 tx=0).stop 2024-05-11T08:06:15.416 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.411+0000 7f418ecb4640 1 -- 172.21.15.167:0/1791741388 shutdown_connections 2024-05-11T08:06:15.416 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.411+0000 7f418ecb4640 1 --2- 172.21.15.167:0/1791741388 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f41740762d0 0x7f4174078790 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:15.416 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.411+0000 7f418ecb4640 1 --2- 172.21.15.167:0/1791741388 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f41881598d0 0x7f41881f8c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:15.416 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.411+0000 7f418ecb4640 1 --2- 172.21.15.167:0/1791741388 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f41880d0640 0x7f41881f8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:15.416 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.411+0000 7f418ecb4640 1 --2- 172.21.15.167:0/1791741388 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f41880cafb0 0x7f418814d550 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:15.416 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.411+0000 7f418ecb4640 1 -- 172.21.15.167:0/1791741388 >> 172.21.15.167:0/1791741388 conn(0x7f41880d1ba0 msgr2=0x7f4188163630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:15.416 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.412+0000 7f418ecb4640 1 -- 172.21.15.167:0/1791741388 shutdown_connections 2024-05-11T08:06:15.416 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.412+0000 7f418ecb4640 1 -- 172.21.15.167:0/1791741388 wait complete. 2024-05-11T08:06:15.423 INFO:teuthology.orchestra.run.smithi167.stdout:{"pg_ready":true,"pg_map":{"version":37,"stamp":"2024-05-11T08:06:13.921061+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":149,"num_write_kb":1646,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":114,"ondisk_log_size":114,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219604,"kb_used_data":3432,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755116,"statfs":{"total":773094113280,"available":772869238784,"internally_reserved":0,"allocated":3514368,"data_stored":2074472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003049"},"pg_stats":[{"pgid":"2.1f","version":"22'4","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.392985+0000","last_change":"2024-05-11T08:05:54.368188+0000","last_active":"2024-05-11T08:06:00.392985+0000","last_peered":"2024-05-11T08:06:00.392985+0000","last_clean":"2024-05-11T08:06:00.392985+0000","last_became_active":"2024-05-11T08:05:54.367881+0000","last_became_peered":"2024-05-11T08:05:54.367881+0000","last_unstale":"2024-05-11T08:06:00.392985+0000","last_undegraded":"2024-05-11T08:06:00.392985+0000","last_fullsized":"2024-05-11T08:06:00.392985+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_clean_scrub_stamp":"2024-05-11T08:05:53.331701+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:22:53.142453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.645941+0000","last_change":"2024-05-11T08:05:55.367955+0000","last_active":"2024-05-11T08:06:01.645941+0000","last_peered":"2024-05-11T08:06:01.645941+0000","last_clean":"2024-05-11T08:06:01.645941+0000","last_became_active":"2024-05-11T08:05:55.366260+0000","last_became_peered":"2024-05-11T08:05:55.366260+0000","last_unstale":"2024-05-11T08:06:01.645941+0000","last_undegraded":"2024-05-11T08:06:01.645941+0000","last_fullsized":"2024-05-11T08:06:01.645941+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:44:43.387850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.647113+0000","last_change":"2024-05-11T08:05:59.558909+0000","last_active":"2024-05-11T08:06:01.647113+0000","last_peered":"2024-05-11T08:06:01.647113+0000","last_clean":"2024-05-11T08:06:01.647113+0000","last_became_active":"2024-05-11T08:05:59.558344+0000","last_became_peered":"2024-05-11T08:05:59.558344+0000","last_unstale":"2024-05-11T08:06:01.647113+0000","last_undegraded":"2024-05-11T08:06:01.647113+0000","last_fullsized":"2024-05-11T08:06:01.647113+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_clean_scrub_stamp":"2024-05-11T08:05:58.374552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:50:45.024927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"22'1","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.646170+0000","last_change":"2024-05-11T08:05:54.371497+0000","last_active":"2024-05-11T08:06:01.646170+0000","last_peered":"2024-05-11T08:06:01.646170+0000","last_clean":"2024-05-11T08:06:01.646170+0000","last_became_active":"2024-05-11T08:05:54.371241+0000","last_became_peered":"2024-05-11T08:05:54.371241+0000","last_unstale":"2024-05-11T08:06:01.646170+0000","last_undegraded":"2024-05-11T08:06:01.646170+0000","last_fullsized":"2024-05-11T08:06:01.646170+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_clean_scrub_stamp":"2024-05-11T08:05:53.331701+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:33:25.907598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.392845+0000","last_change":"2024-05-11T08:05:55.369937+0000","last_active":"2024-05-11T08:06:00.392845+0000","last_peered":"2024-05-11T08:06:00.392845+0000","last_clean":"2024-05-11T08:06:00.392845+0000","last_became_active":"2024-05-11T08:05:55.369476+0000","last_became_peered":"2024-05-11T08:05:55.369476+0000","last_unstale":"2024-05-11T08:06:00.392845+0000","last_undegraded":"2024-05-11T08:06:00.392845+0000","last_fullsized":"2024-05-11T08:06:00.392845+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:16:46.423362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.823013+0000","last_change":"2024-05-11T08:05:59.407301+0000","last_active":"2024-05-11T08:06:00.823013+0000","last_peered":"2024-05-11T08:06:00.823013+0000","last_clean":"2024-05-11T08:06:00.823013+0000","last_became_active":"2024-05-11T08:05:59.406627+0000","last_became_peered":"2024-05-11T08:05:59.406627+0000","last_unstale":"2024-05-11T08:06:00.823013+0000","last_undegraded":"2024-05-11T08:06:00.823013+0000","last_fullsized":"2024-05-11T08:06:00.823013+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_clean_scrub_stamp":"2024-05-11T08:05:58.374552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:41:42.964013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"22'2","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.388522+0000","last_change":"2024-05-11T08:05:54.367998+0000","last_active":"2024-05-11T08:06:00.388522+0000","last_peered":"2024-05-11T08:06:00.388522+0000","last_clean":"2024-05-11T08:06:00.388522+0000","last_became_active":"2024-05-11T08:05:54.367681+0000","last_became_peered":"2024-05-11T08:05:54.367681+0000","last_unstale":"2024-05-11T08:06:00.388522+0000","last_undegraded":"2024-05-11T08:06:00.388522+0000","last_fullsized":"2024-05-11T08:06:00.388522+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_clean_scrub_stamp":"2024-05-11T08:05:53.331701+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:50:17.617231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.389537+0000","last_change":"2024-05-11T08:05:55.379931+0000","last_active":"2024-05-11T08:06:00.389537+0000","last_peered":"2024-05-11T08:06:00.389537+0000","last_clean":"2024-05-11T08:06:00.389537+0000","last_became_active":"2024-05-11T08:05:55.379656+0000","last_became_peered":"2024-05-11T08:05:55.379656+0000","last_unstale":"2024-05-11T08:06:00.389537+0000","last_undegraded":"2024-05-11T08:06:00.389537+0000","last_fullsized":"2024-05-11T08:06:00.389537+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:14:09.302889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.823238+0000","last_change":"2024-05-11T08:05:59.404405+0000","last_active":"2024-05-11T08:06:00.823238+0000","last_peered":"2024-05-11T08:06:00.823238+0000","last_clean":"2024-05-11T08:06:00.823238+0000","last_became_active":"2024-05-11T08:05:59.403407+0000","last_became_peered":"2024-05-11T08:05:59.403407+0000","last_unstale":"2024-05-11T08:06:00.823238+0000","last_undegraded":"2024-05-11T08:06:00.823238+0000","last_fullsized":"2024-05-11T08:06:00.823238+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_clean_scrub_stamp":"2024-05-11T08:05:58.374552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:57:03.981207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"22'2","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.388963+0000","last_change":"2024-05-11T08:05:54.369560+0000","last_active":"2024-05-11T08:06:00.388963+0000","last_peered":"2024-05-11T08:06:00.388963+0000","last_clean":"2024-05-11T08:06:00.388963+0000","last_became_active":"2024-05-11T08:05:54.369150+0000","last_became_peered":"2024-05-11T08:05:54.369150+0000","last_unstale":"2024-05-11T08:06:00.388963+0000","last_undegraded":"2024-05-11T08:06:00.388963+0000","last_fullsized":"2024-05-11T08:06:00.388963+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_clean_scrub_stamp":"2024-05-11T08:05:53.331701+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:17:22.080661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.389718+0000","last_change":"2024-05-11T08:05:55.379813+0000","last_active":"2024-05-11T08:06:00.389718+0000","last_peered":"2024-05-11T08:06:00.389718+0000","last_clean":"2024-05-11T08:06:00.389718+0000","last_became_active":"2024-05-11T08:05:55.379575+0000","last_became_peered":"2024-05-11T08:05:55.379575+0000","last_unstale":"2024-05-11T08:06:00.389718+0000","last_undegraded":"2024-05-11T08:06:00.389718+0000","last_fullsized":"2024-05-11T08:06:00.389718+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:28:24.740747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.823228+0000","last_change":"2024-05-11T08:05:59.412477+0000","last_active":"2024-05-11T08:06:00.823228+0000","last_peered":"2024-05-11T08:06:00.823228+0000","last_clean":"2024-05-11T08:06:00.823228+0000","last_became_active":"2024-05-11T08:05:59.411969+0000","last_became_peered":"2024-05-11T08:05:59.411969+0000","last_unstale":"2024-05-11T08:06:00.823228+0000","last_undegraded":"2024-05-11T08:06:00.823228+0000","last_fullsized":"2024-05-11T08:06:00.823228+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_clean_scrub_stamp":"2024-05-11T08:05:58.374552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:38:06.426124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.645499+0000","last_change":"2024-05-11T08:05:55.376503+0000","last_active":"2024-05-11T08:06:01.645499+0000","last_peered":"2024-05-11T08:06:01.645499+0000","last_clean":"2024-05-11T08:06:01.645499+0000","last_became_active":"2024-05-11T08:05:55.376198+0000","last_became_peered":"2024-05-11T08:05:55.376198+0000","last_unstale":"2024-05-11T08:06:01.645499+0000","last_undegraded":"2024-05-11T08:06:01.645499+0000","last_fullsized":"2024-05-11T08:06:01.645499+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:20:43.832246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"22'2","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.388669+0000","last_change":"2024-05-11T08:05:54.370164+0000","last_active":"2024-05-11T08:06:00.388669+0000","last_peered":"2024-05-11T08:06:00.388669+0000","last_clean":"2024-05-11T08:06:00.388669+0000","last_became_active":"2024-05-11T08:05:54.369893+0000","last_became_peered":"2024-05-11T08:05:54.369893+0000","last_unstale":"2024-05-11T08:06:00.388669+0000","last_undegraded":"2024-05-11T08:06:00.388669+0000","last_fullsized":"2024-05-11T08:06:00.388669+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_clean_scrub_stamp":"2024-05-11T08:05:53.331701+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:53:10.612688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.823618+0000","last_change":"2024-05-11T08:05:59.409212+0000","last_active":"2024-05-11T08:06:00.823618+0000","last_peered":"2024-05-11T08:06:00.823618+0000","last_clean":"2024-05-11T08:06:00.823618+0000","last_became_active":"2024-05-11T08:05:59.408832+0000","last_became_peered":"2024-05-11T08:05:59.408832+0000","last_unstale":"2024-05-11T08:06:00.823618+0000","last_undegraded":"2024-05-11T08:06:00.823618+0000","last_fullsized":"2024-05-11T08:06:00.823618+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_clean_scrub_stamp":"2024-05-11T08:05:58.374552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:45:28.584065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.646456+0000","last_change":"2024-05-11T08:05:55.362770+0000","last_active":"2024-05-11T08:06:01.646456+0000","last_peered":"2024-05-11T08:06:01.646456+0000","last_clean":"2024-05-11T08:06:01.646456+0000","last_became_active":"2024-05-11T08:05:55.362481+0000","last_became_peered":"2024-05-11T08:05:55.362481+0000","last_unstale":"2024-05-11T08:06:01.646456+0000","last_undegraded":"2024-05-11T08:06:01.646456+0000","last_fullsized":"2024-05-11T08:06:01.646456+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:22:00.631022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.645921+0000","last_change":"2024-05-11T08:05:54.369496+0000","last_active":"2024-05-11T08:06:01.645921+0000","last_peered":"2024-05-11T08:06:01.645921+0000","last_clean":"2024-05-11T08:06:01.645921+0000","last_became_active":"2024-05-11T08:05:54.369246+0000","last_became_peered":"2024-05-11T08:05:54.369246+0000","last_unstale":"2024-05-11T08:06:01.645921+0000","last_undegraded":"2024-05-11T08:06:01.645921+0000","last_fullsized":"2024-05-11T08:06:01.645921+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_clean_scrub_stamp":"2024-05-11T08:05:53.331701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:30:23.111497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.823390+0000","last_change":"2024-05-11T08:05:59.403693+0000","last_active":"2024-05-11T08:06:00.823390+0000","last_peered":"2024-05-11T08:06:00.823390+0000","last_clean":"2024-05-11T08:06:00.823390+0000","last_became_active":"2024-05-11T08:05:59.403195+0000","last_became_peered":"2024-05-11T08:05:59.403195+0000","last_unstale":"2024-05-11T08:06:00.823390+0000","last_undegraded":"2024-05-11T08:06:00.823390+0000","last_fullsized":"2024-05-11T08:06:00.823390+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_clean_scrub_stamp":"2024-05-11T08:05:58.374552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:28:28.773313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.645822+0000","last_change":"2024-05-11T08:05:55.366582+0000","last_active":"2024-05-11T08:06:01.645822+0000","last_peered":"2024-05-11T08:06:01.645822+0000","last_clean":"2024-05-11T08:06:01.645822+0000","last_became_active":"2024-05-11T08:05:55.366309+0000","last_became_peered":"2024-05-11T08:05:55.366309+0000","last_unstale":"2024-05-11T08:06:01.645822+0000","last_undegraded":"2024-05-11T08:06:01.645822+0000","last_fullsized":"2024-05-11T08:06:01.645822+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:32:48.010809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.645408+0000","last_change":"2024-05-11T08:05:54.368032+0000","last_active":"2024-05-11T08:06:01.645408+0000","last_peered":"2024-05-11T08:06:01.645408+0000","last_clean":"2024-05-11T08:06:01.645408+0000","last_became_active":"2024-05-11T08:05:54.367739+0000","last_became_peered":"2024-05-11T08:05:54.367739+0000","last_unstale":"2024-05-11T08:06:01.645408+0000","last_undegraded":"2024-05-11T08:06:01.645408+0000","last_fullsized":"2024-05-11T08:06:01.645408+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_clean_scrub_stamp":"2024-05-11T08:05:53.331701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:01:58.573898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.646280+0000","last_change":"2024-05-11T08:05:59.560219+0000","last_active":"2024-05-11T08:06:01.646280+0000","last_peered":"2024-05-11T08:06:01.646280+0000","last_clean":"2024-05-11T08:06:01.646280+0000","last_became_active":"2024-05-11T08:05:59.559821+0000","last_became_peered":"2024-05-11T08:05:59.559821+0000","last_unstale":"2024-05-11T08:06:01.646280+0000","last_undegraded":"2024-05-11T08:06:01.646280+0000","last_fullsized":"2024-05-11T08:06:01.646280+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_clean_scrub_stamp":"2024-05-11T08:05:58.374552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:51:30.882512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.823321+0000","last_change":"2024-05-11T08:05:55.370564+0000","last_active":"2024-05-11T08:06:00.823321+0000","last_peered":"2024-05-11T08:06:00.823321+0000","last_clean":"2024-05-11T08:06:00.823321+0000","last_became_active":"2024-05-11T08:05:55.370277+0000","last_became_peered":"2024-05-11T08:05:55.370277+0000","last_unstale":"2024-05-11T08:06:00.823321+0000","last_undegraded":"2024-05-11T08:06:00.823321+0000","last_fullsized":"2024-05-11T08:06:00.823321+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:30:24.121782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.387998+0000","last_change":"2024-05-11T08:05:54.371153+0000","last_active":"2024-05-11T08:06:00.387998+0000","last_peered":"2024-05-11T08:06:00.387998+0000","last_clean":"2024-05-11T08:06:00.387998+0000","last_became_active":"2024-05-11T08:05:54.369944+0000","last_became_peered":"2024-05-11T08:05:54.369944+0000","last_unstale":"2024-05-11T08:06:00.387998+0000","last_undegraded":"2024-05-11T08:06:00.387998+0000","last_fullsized":"2024-05-11T08:06:00.387998+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_clean_scrub_stamp":"2024-05-11T08:05:53.331701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:30:23.111497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.823584+0000","last_change":"2024-05-11T08:05:59.403650+0000","last_active":"2024-05-11T08:06:00.823584+0000","last_peered":"2024-05-11T08:06:00.823584+0000","last_clean":"2024-05-11T08:06:00.823584+0000","last_became_active":"2024-05-11T08:05:59.403181+0000","last_became_peered":"2024-05-11T08:05:59.403181+0000","last_unstale":"2024-05-11T08:06:00.823584+0000","last_undegraded":"2024-05-11T08:06:00.823584+0000","last_fullsized":"2024-05-11T08:06:00.823584+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_clean_scrub_stamp":"2024-05-11T08:05:58.374552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:51:30.882512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.392942+0000","last_change":"2024-05-11T08:05:55.365766+0000","last_active":"2024-05-11T08:06:00.392942+0000","last_peered":"2024-05-11T08:06:00.392942+0000","last_clean":"2024-05-11T08:06:00.392942+0000","last_became_active":"2024-05-11T08:05:55.365513+0000","last_became_peered":"2024-05-11T08:05:55.365513+0000","last_unstale":"2024-05-11T08:06:00.392942+0000","last_undegraded":"2024-05-11T08:06:00.392942+0000","last_fullsized":"2024-05-11T08:06:00.392942+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:43:37.948904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"22'2","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.646133+0000","last_change":"2024-05-11T08:05:54.368015+0000","last_active":"2024-05-11T08:06:01.646133+0000","last_peered":"2024-05-11T08:06:01.646133+0000","last_clean":"2024-05-11T08:06:01.646133+0000","last_became_active":"2024-05-11T08:05:54.367740+0000","last_became_peered":"2024-05-11T08:05:54.367740+0000","last_unstale":"2024-05-11T08:06:01.646133+0000","last_undegraded":"2024-05-11T08:06:01.646133+0000","last_fullsized":"2024-05-11T08:06:01.646133+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_clean_scrub_stamp":"2024-05-11T08:05:53.331701+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:30:23.111497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.823061+0000","last_change":"2024-05-11T08:05:59.559986+0000","last_active":"2024-05-11T08:06:00.823061+0000","last_peered":"2024-05-11T08:06:00.823061+0000","last_clean":"2024-05-11T08:06:00.823061+0000","last_became_active":"2024-05-11T08:05:59.559594+0000","last_became_peered":"2024-05-11T08:05:59.559594+0000","last_unstale":"2024-05-11T08:06:00.823061+0000","last_undegraded":"2024-05-11T08:06:00.823061+0000","last_fullsized":"2024-05-11T08:06:00.823061+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_clean_scrub_stamp":"2024-05-11T08:05:58.374552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:14:13.335454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.646670+0000","last_change":"2024-05-11T08:05:55.374876+0000","last_active":"2024-05-11T08:06:01.646670+0000","last_peered":"2024-05-11T08:06:01.646670+0000","last_clean":"2024-05-11T08:06:01.646670+0000","last_became_active":"2024-05-11T08:05:55.374617+0000","last_became_peered":"2024-05-11T08:05:55.374617+0000","last_unstale":"2024-05-11T08:06:01.646670+0000","last_undegraded":"2024-05-11T08:06:01.646670+0000","last_fullsized":"2024-05-11T08:06:01.646670+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:41:38.931447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"23'1","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.646291+0000","last_change":"2024-05-11T08:05:54.365008+0000","last_active":"2024-05-11T08:06:01.646291+0000","last_peered":"2024-05-11T08:06:01.646291+0000","last_clean":"2024-05-11T08:06:01.646291+0000","last_became_active":"2024-05-11T08:05:54.364698+0000","last_became_peered":"2024-05-11T08:05:54.364698+0000","last_unstale":"2024-05-11T08:06:01.646291+0000","last_undegraded":"2024-05-11T08:06:01.646291+0000","last_fullsized":"2024-05-11T08:06:01.646291+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_clean_scrub_stamp":"2024-05-11T08:05:53.331701+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:17:22.080661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.645075+0000","last_change":"2024-05-11T08:05:59.562139+0000","last_active":"2024-05-11T08:06:01.645075+0000","last_peered":"2024-05-11T08:06:01.645075+0000","last_clean":"2024-05-11T08:06:01.645075+0000","last_became_active":"2024-05-11T08:05:59.561081+0000","last_became_peered":"2024-05-11T08:05:59.561081+0000","last_unstale":"2024-05-11T08:06:01.645075+0000","last_undegraded":"2024-05-11T08:06:01.645075+0000","last_fullsized":"2024-05-11T08:06:01.645075+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_clean_scrub_stamp":"2024-05-11T08:05:58.374552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:41:33.826242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.645886+0000","last_change":"2024-05-11T08:05:55.371779+0000","last_active":"2024-05-11T08:06:01.645886+0000","last_peered":"2024-05-11T08:06:01.645886+0000","last_clean":"2024-05-11T08:06:01.645886+0000","last_became_active":"2024-05-11T08:05:55.368726+0000","last_became_peered":"2024-05-11T08:05:55.368726+0000","last_unstale":"2024-05-11T08:06:01.645886+0000","last_undegraded":"2024-05-11T08:06:01.645886+0000","last_fullsized":"2024-05-11T08:06:01.645886+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:32:48.010809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.388659+0000","last_change":"2024-05-11T08:05:54.370339+0000","last_active":"2024-05-11T08:06:00.388659+0000","last_peered":"2024-05-11T08:06:00.388659+0000","last_clean":"2024-05-11T08:06:00.388659+0000","last_became_active":"2024-05-11T08:05:54.370026+0000","last_became_peered":"2024-05-11T08:05:54.370026+0000","last_unstale":"2024-05-11T08:06:00.388659+0000","last_undegraded":"2024-05-11T08:06:00.388659+0000","last_fullsized":"2024-05-11T08:06:00.388659+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_clean_scrub_stamp":"2024-05-11T08:05:53.331701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:32:43.591947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.392567+0000","last_change":"2024-05-11T08:05:59.558387+0000","last_active":"2024-05-11T08:06:00.392567+0000","last_peered":"2024-05-11T08:06:00.392567+0000","last_clean":"2024-05-11T08:06:00.392567+0000","last_became_active":"2024-05-11T08:05:59.557980+0000","last_became_peered":"2024-05-11T08:05:59.557980+0000","last_unstale":"2024-05-11T08:06:00.392567+0000","last_undegraded":"2024-05-11T08:06:00.392567+0000","last_fullsized":"2024-05-11T08:06:00.392567+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_clean_scrub_stamp":"2024-05-11T08:05:58.374552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:12:42.864479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.389813+0000","last_change":"2024-05-11T08:05:55.373737+0000","last_active":"2024-05-11T08:06:00.389813+0000","last_peered":"2024-05-11T08:06:00.389813+0000","last_clean":"2024-05-11T08:06:00.389813+0000","last_became_active":"2024-05-11T08:05:55.373459+0000","last_became_peered":"2024-05-11T08:05:55.373459+0000","last_unstale":"2024-05-11T08:06:00.389813+0000","last_undegraded":"2024-05-11T08:06:00.389813+0000","last_fullsized":"2024-05-11T08:06:00.389813+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:51:26.849946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.645985+0000","last_change":"2024-05-11T08:05:54.367679+0000","last_active":"2024-05-11T08:06:01.645985+0000","last_peered":"2024-05-11T08:06:01.645985+0000","last_clean":"2024-05-11T08:06:01.645985+0000","last_became_active":"2024-05-11T08:05:54.367393+0000","last_became_peered":"2024-05-11T08:05:54.367393+0000","last_unstale":"2024-05-11T08:06:01.645985+0000","last_undegraded":"2024-05-11T08:06:01.645985+0000","last_fullsized":"2024-05-11T08:06:01.645985+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_clean_scrub_stamp":"2024-05-11T08:05:53.331701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:50:17.617231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.645628+0000","last_change":"2024-05-11T08:05:59.413104+0000","last_active":"2024-05-11T08:06:01.645628+0000","last_peered":"2024-05-11T08:06:01.645628+0000","last_clean":"2024-05-11T08:06:01.645628+0000","last_became_active":"2024-05-11T08:05:59.412442+0000","last_became_peered":"2024-05-11T08:05:59.412442+0000","last_unstale":"2024-05-11T08:06:01.645628+0000","last_undegraded":"2024-05-11T08:06:01.645628+0000","last_fullsized":"2024-05-11T08:06:01.645628+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_clean_scrub_stamp":"2024-05-11T08:05:58.374552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:57:03.981207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.823506+0000","last_change":"2024-05-11T08:05:55.377393+0000","last_active":"2024-05-11T08:06:00.823506+0000","last_peered":"2024-05-11T08:06:00.823506+0000","last_clean":"2024-05-11T08:06:00.823506+0000","last_became_active":"2024-05-11T08:05:55.377129+0000","last_became_peered":"2024-05-11T08:05:55.377129+0000","last_unstale":"2024-05-11T08:06:00.823506+0000","last_undegraded":"2024-05-11T08:06:00.823506+0000","last_fullsized":"2024-05-11T08:06:00.823506+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:05:55.494912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.389461+0000","last_change":"2024-05-11T08:05:54.361751+0000","last_active":"2024-05-11T08:06:00.389461+0000","last_peered":"2024-05-11T08:06:00.389461+0000","last_clean":"2024-05-11T08:06:00.389461+0000","last_became_active":"2024-05-11T08:05:54.361484+0000","last_became_peered":"2024-05-11T08:05:54.361484+0000","last_unstale":"2024-05-11T08:06:00.389461+0000","last_undegraded":"2024-05-11T08:06:00.389461+0000","last_fullsized":"2024-05-11T08:06:00.389461+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_clean_scrub_stamp":"2024-05-11T08:05:53.331701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:53:10.612688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.645837+0000","last_change":"2024-05-11T08:05:59.409532+0000","last_active":"2024-05-11T08:06:01.645837+0000","last_peered":"2024-05-11T08:06:01.645837+0000","last_clean":"2024-05-11T08:06:01.645837+0000","last_became_active":"2024-05-11T08:05:59.409096+0000","last_became_peered":"2024-05-11T08:05:59.409096+0000","last_unstale":"2024-05-11T08:06:01.645837+0000","last_undegraded":"2024-05-11T08:06:01.645837+0000","last_fullsized":"2024-05-11T08:06:01.645837+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_clean_scrub_stamp":"2024-05-11T08:05:58.374552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:12:42.864479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.645313+0000","last_change":"2024-05-11T08:05:55.368600+0000","last_active":"2024-05-11T08:06:01.645313+0000","last_peered":"2024-05-11T08:06:01.645313+0000","last_clean":"2024-05-11T08:06:01.645313+0000","last_became_active":"2024-05-11T08:05:55.368322+0000","last_became_peered":"2024-05-11T08:05:55.368322+0000","last_unstale":"2024-05-11T08:06:01.645313+0000","last_undegraded":"2024-05-11T08:06:01.645313+0000","last_fullsized":"2024-05-11T08:06:01.645313+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:38:18.066779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.645325+0000","last_change":"2024-05-11T08:05:54.369698+0000","last_active":"2024-05-11T08:06:01.645325+0000","last_peered":"2024-05-11T08:06:01.645325+0000","last_clean":"2024-05-11T08:06:01.645325+0000","last_became_active":"2024-05-11T08:05:54.369387+0000","last_became_peered":"2024-05-11T08:05:54.369387+0000","last_unstale":"2024-05-11T08:06:01.645325+0000","last_undegraded":"2024-05-11T08:06:01.645325+0000","last_fullsized":"2024-05-11T08:06:01.645325+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_clean_scrub_stamp":"2024-05-11T08:05:53.331701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:14:18.956461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.646283+0000","last_change":"2024-05-11T08:05:59.413264+0000","last_active":"2024-05-11T08:06:01.646283+0000","last_peered":"2024-05-11T08:06:01.646283+0000","last_clean":"2024-05-11T08:06:01.646283+0000","last_became_active":"2024-05-11T08:05:59.412704+0000","last_became_peered":"2024-05-11T08:05:59.412704+0000","last_unstale":"2024-05-11T08:06:01.646283+0000","last_undegraded":"2024-05-11T08:06:01.646283+0000","last_fullsized":"2024-05-11T08:06:01.646283+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_clean_scrub_stamp":"2024-05-11T08:05:58.374552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:45:28.584065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.392303+0000","last_change":"2024-05-11T08:05:55.373518+0000","last_active":"2024-05-11T08:06:00.392303+0000","last_peered":"2024-05-11T08:06:00.392303+0000","last_clean":"2024-05-11T08:06:00.392303+0000","last_became_active":"2024-05-11T08:05:55.373067+0000","last_became_peered":"2024-05-11T08:05:55.373067+0000","last_unstale":"2024-05-11T08:06:00.392303+0000","last_undegraded":"2024-05-11T08:06:00.392303+0000","last_fullsized":"2024-05-11T08:06:00.392303+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:35:55.478239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.388011+0000","last_change":"2024-05-11T08:05:54.369563+0000","last_active":"2024-05-11T08:06:00.388011+0000","last_peered":"2024-05-11T08:06:00.388011+0000","last_clean":"2024-05-11T08:06:00.388011+0000","last_became_active":"2024-05-11T08:05:54.369257+0000","last_became_peered":"2024-05-11T08:05:54.369257+0000","last_unstale":"2024-05-11T08:06:00.388011+0000","last_undegraded":"2024-05-11T08:06:00.388011+0000","last_fullsized":"2024-05-11T08:06:00.388011+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_clean_scrub_stamp":"2024-05-11T08:05:53.331701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:05:54.484627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.392895+0000","last_change":"2024-05-11T08:05:59.560554+0000","last_active":"2024-05-11T08:06:00.392895+0000","last_peered":"2024-05-11T08:06:00.392895+0000","last_clean":"2024-05-11T08:06:00.392895+0000","last_became_active":"2024-05-11T08:05:59.560163+0000","last_became_peered":"2024-05-11T08:05:59.560163+0000","last_unstale":"2024-05-11T08:06:00.392895+0000","last_undegraded":"2024-05-11T08:06:00.392895+0000","last_fullsized":"2024-05-11T08:06:00.392895+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_clean_scrub_stamp":"2024-05-11T08:05:58.374552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:19:03.458410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.645474+0000","last_change":"2024-05-11T08:05:55.371707+0000","last_active":"2024-05-11T08:06:01.645474+0000","last_peered":"2024-05-11T08:06:01.645474+0000","last_clean":"2024-05-11T08:06:01.645474+0000","last_became_active":"2024-05-11T08:05:55.371443+0000","last_became_peered":"2024-05-11T08:05:55.371443+0000","last_unstale":"2024-05-11T08:06:01.645474+0000","last_undegraded":"2024-05-11T08:06:01.645474+0000","last_fullsized":"2024-05-11T08:06:01.645474+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:16:46.423362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.389333+0000","last_change":"2024-05-11T08:05:54.361406+0000","last_active":"2024-05-11T08:06:00.389333+0000","last_peered":"2024-05-11T08:06:00.389333+0000","last_clean":"2024-05-11T08:06:00.389333+0000","last_became_active":"2024-05-11T08:05:54.361065+0000","last_became_peered":"2024-05-11T08:05:54.361065+0000","last_unstale":"2024-05-11T08:06:00.389333+0000","last_undegraded":"2024-05-11T08:06:00.389333+0000","last_fullsized":"2024-05-11T08:06:00.389333+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_clean_scrub_stamp":"2024-05-11T08:05:53.331701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:33:25.907598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":118,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.388646+0000","last_change":"2024-05-11T08:05:45.772368+0000","last_active":"2024-05-11T08:06:00.388646+0000","last_peered":"2024-05-11T08:06:00.388646+0000","last_clean":"2024-05-11T08:06:00.388646+0000","last_became_active":"2024-05-11T08:05:45.771373+0000","last_became_peered":"2024-05-11T08:05:45.771373+0000","last_unstale":"2024-05-11T08:06:00.388646+0000","last_undegraded":"2024-05-11T08:06:00.388646+0000","last_fullsized":"2024-05-11T08:06:00.388646+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:44.616617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:44.616617+0000","last_clean_scrub_stamp":"2024-05-11T08:05:44.616617+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:02:08.078748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.645881+0000","last_change":"2024-05-11T08:05:59.558739+0000","last_active":"2024-05-11T08:06:01.645881+0000","last_peered":"2024-05-11T08:06:01.645881+0000","last_clean":"2024-05-11T08:06:01.645881+0000","last_became_active":"2024-05-11T08:05:59.558307+0000","last_became_peered":"2024-05-11T08:05:59.558307+0000","last_unstale":"2024-05-11T08:06:01.645881+0000","last_undegraded":"2024-05-11T08:06:01.645881+0000","last_fullsized":"2024-05-11T08:06:01.645881+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_clean_scrub_stamp":"2024-05-11T08:05:58.374552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:28:28.773313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.388334+0000","last_change":"2024-05-11T08:05:55.379286+0000","last_active":"2024-05-11T08:06:00.388334+0000","last_peered":"2024-05-11T08:06:00.388334+0000","last_clean":"2024-05-11T08:06:00.388334+0000","last_became_active":"2024-05-11T08:05:55.378804+0000","last_became_peered":"2024-05-11T08:05:55.378804+0000","last_unstale":"2024-05-11T08:06:00.388334+0000","last_undegraded":"2024-05-11T08:06:00.388334+0000","last_fullsized":"2024-05-11T08:06:00.388334+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:56:59.948641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.646359+0000","last_change":"2024-05-11T08:05:54.364902+0000","last_active":"2024-05-11T08:06:01.646359+0000","last_peered":"2024-05-11T08:06:01.646359+0000","last_clean":"2024-05-11T08:06:01.646359+0000","last_became_active":"2024-05-11T08:05:54.364619+0000","last_became_peered":"2024-05-11T08:05:54.364619+0000","last_unstale":"2024-05-11T08:06:01.646359+0000","last_undegraded":"2024-05-11T08:06:01.646359+0000","last_fullsized":"2024-05-11T08:06:01.646359+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_clean_scrub_stamp":"2024-05-11T08:05:53.331701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:10:56.295408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.645635+0000","last_change":"2024-05-11T08:05:59.558409+0000","last_active":"2024-05-11T08:06:01.645635+0000","last_peered":"2024-05-11T08:06:01.645635+0000","last_clean":"2024-05-11T08:06:01.645635+0000","last_became_active":"2024-05-11T08:05:59.557939+0000","last_became_peered":"2024-05-11T08:05:59.557939+0000","last_unstale":"2024-05-11T08:06:01.645635+0000","last_undegraded":"2024-05-11T08:06:01.645635+0000","last_fullsized":"2024-05-11T08:06:01.645635+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_clean_scrub_stamp":"2024-05-11T08:05:58.374552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:57:03.981207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.389620+0000","last_change":"2024-05-11T08:05:55.376614+0000","last_active":"2024-05-11T08:06:00.389620+0000","last_peered":"2024-05-11T08:06:00.389620+0000","last_clean":"2024-05-11T08:06:00.389620+0000","last_became_active":"2024-05-11T08:05:55.376350+0000","last_became_peered":"2024-05-11T08:05:55.376350+0000","last_unstale":"2024-05-11T08:06:00.389620+0000","last_undegraded":"2024-05-11T08:06:00.389620+0000","last_fullsized":"2024-05-11T08:06:00.389620+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:12:38.831913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"22'1","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.645871+0000","last_change":"2024-05-11T08:05:54.369427+0000","last_active":"2024-05-11T08:06:01.645871+0000","last_peered":"2024-05-11T08:06:01.645871+0000","last_clean":"2024-05-11T08:06:01.645871+0000","last_became_active":"2024-05-11T08:05:54.369181+0000","last_became_peered":"2024-05-11T08:05:54.369181+0000","last_unstale":"2024-05-11T08:06:01.645871+0000","last_undegraded":"2024-05-11T08:06:01.645871+0000","last_fullsized":"2024-05-11T08:06:01.645871+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_clean_scrub_stamp":"2024-05-11T08:05:53.331701+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:05:54.484627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.392744+0000","last_change":"2024-05-11T08:05:59.411982+0000","last_active":"2024-05-11T08:06:00.392744+0000","last_peered":"2024-05-11T08:06:00.392744+0000","last_clean":"2024-05-11T08:06:00.392744+0000","last_became_active":"2024-05-11T08:05:59.411561+0000","last_became_peered":"2024-05-11T08:05:59.411561+0000","last_unstale":"2024-05-11T08:06:00.392744+0000","last_undegraded":"2024-05-11T08:06:00.392744+0000","last_fullsized":"2024-05-11T08:06:00.392744+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_clean_scrub_stamp":"2024-05-11T08:05:58.374552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:45:28.584065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.389445+0000","last_change":"2024-05-11T08:05:55.386175+0000","last_active":"2024-05-11T08:06:00.389445+0000","last_peered":"2024-05-11T08:06:00.389445+0000","last_clean":"2024-05-11T08:06:00.389445+0000","last_became_active":"2024-05-11T08:05:55.385883+0000","last_became_peered":"2024-05-11T08:05:55.385883+0000","last_unstale":"2024-05-11T08:06:00.389445+0000","last_undegraded":"2024-05-11T08:06:00.389445+0000","last_fullsized":"2024-05-11T08:06:00.389445+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:43:20.023451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"22'2","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.645669+0000","last_change":"2024-05-11T08:05:54.369810+0000","last_active":"2024-05-11T08:06:01.645669+0000","last_peered":"2024-05-11T08:06:01.645669+0000","last_clean":"2024-05-11T08:06:01.645669+0000","last_became_active":"2024-05-11T08:05:54.369512+0000","last_became_peered":"2024-05-11T08:05:54.369512+0000","last_unstale":"2024-05-11T08:06:01.645669+0000","last_undegraded":"2024-05-11T08:06:01.645669+0000","last_fullsized":"2024-05-11T08:06:01.645669+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_clean_scrub_stamp":"2024-05-11T08:05:53.331701+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:39:04.019231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.388134+0000","last_change":"2024-05-11T08:05:59.562447+0000","last_active":"2024-05-11T08:06:00.388134+0000","last_peered":"2024-05-11T08:06:00.388134+0000","last_clean":"2024-05-11T08:06:00.388134+0000","last_became_active":"2024-05-11T08:05:59.562033+0000","last_became_peered":"2024-05-11T08:05:59.562033+0000","last_unstale":"2024-05-11T08:06:00.388134+0000","last_undegraded":"2024-05-11T08:06:00.388134+0000","last_fullsized":"2024-05-11T08:06:00.388134+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_clean_scrub_stamp":"2024-05-11T08:05:58.374552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:41:33.826242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.388740+0000","last_change":"2024-05-11T08:05:55.386091+0000","last_active":"2024-05-11T08:06:00.388740+0000","last_peered":"2024-05-11T08:06:00.388740+0000","last_clean":"2024-05-11T08:06:00.388740+0000","last_became_active":"2024-05-11T08:05:55.385751+0000","last_became_peered":"2024-05-11T08:05:55.385751+0000","last_unstale":"2024-05-11T08:06:00.388740+0000","last_undegraded":"2024-05-11T08:06:00.388740+0000","last_fullsized":"2024-05-11T08:06:00.388740+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:12:38.831913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"22'1","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.823802+0000","last_change":"2024-05-11T08:05:54.372054+0000","last_active":"2024-05-11T08:06:00.823802+0000","last_peered":"2024-05-11T08:06:00.823802+0000","last_clean":"2024-05-11T08:06:00.823802+0000","last_became_active":"2024-05-11T08:05:54.371721+0000","last_became_peered":"2024-05-11T08:05:54.371721+0000","last_unstale":"2024-05-11T08:06:00.823802+0000","last_undegraded":"2024-05-11T08:06:00.823802+0000","last_fullsized":"2024-05-11T08:06:00.823802+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_clean_scrub_stamp":"2024-05-11T08:05:53.331701+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:40:29.770055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.823074+0000","last_change":"2024-05-11T08:05:59.404048+0000","last_active":"2024-05-11T08:06:00.823074+0000","last_peered":"2024-05-11T08:06:00.823074+0000","last_clean":"2024-05-11T08:06:00.823074+0000","last_became_active":"2024-05-11T08:05:59.403010+0000","last_became_peered":"2024-05-11T08:05:59.403010+0000","last_unstale":"2024-05-11T08:06:00.823074+0000","last_undegraded":"2024-05-11T08:06:00.823074+0000","last_fullsized":"2024-05-11T08:06:00.823074+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_clean_scrub_stamp":"2024-05-11T08:05:58.374552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:44:47.420416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.388312+0000","last_change":"2024-05-11T08:05:55.382224+0000","last_active":"2024-05-11T08:06:00.388312+0000","last_peered":"2024-05-11T08:06:00.388312+0000","last_clean":"2024-05-11T08:06:00.388312+0000","last_became_active":"2024-05-11T08:05:55.381614+0000","last_became_peered":"2024-05-11T08:05:55.381614+0000","last_unstale":"2024-05-11T08:06:00.388312+0000","last_undegraded":"2024-05-11T08:06:00.388312+0000","last_fullsized":"2024-05-11T08:06:00.388312+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:41:38.931447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"22'8","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.645775+0000","last_change":"2024-05-11T08:05:54.365900+0000","last_active":"2024-05-11T08:06:01.645775+0000","last_peered":"2024-05-11T08:06:01.645775+0000","last_clean":"2024-05-11T08:06:01.645775+0000","last_became_active":"2024-05-11T08:05:54.365638+0000","last_became_peered":"2024-05-11T08:05:54.365638+0000","last_unstale":"2024-05-11T08:06:01.645775+0000","last_undegraded":"2024-05-11T08:06:01.645775+0000","last_fullsized":"2024-05-11T08:06:01.645775+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_clean_scrub_stamp":"2024-05-11T08:05:53.331701+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:23:10.014766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.392726+0000","last_change":"2024-05-11T08:05:59.412069+0000","last_active":"2024-05-11T08:06:00.392726+0000","last_peered":"2024-05-11T08:06:00.392726+0000","last_clean":"2024-05-11T08:06:00.392726+0000","last_became_active":"2024-05-11T08:05:59.411674+0000","last_became_peered":"2024-05-11T08:05:59.411674+0000","last_unstale":"2024-05-11T08:06:00.392726+0000","last_undegraded":"2024-05-11T08:06:00.392726+0000","last_fullsized":"2024-05-11T08:06:00.392726+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_clean_scrub_stamp":"2024-05-11T08:05:58.374552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:51:30.882512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.645506+0000","last_change":"2024-05-11T08:05:55.364883+0000","last_active":"2024-05-11T08:06:01.645506+0000","last_peered":"2024-05-11T08:06:01.645506+0000","last_clean":"2024-05-11T08:06:01.645506+0000","last_became_active":"2024-05-11T08:05:55.364622+0000","last_became_peered":"2024-05-11T08:05:55.364622+0000","last_unstale":"2024-05-11T08:06:01.645506+0000","last_undegraded":"2024-05-11T08:06:01.645506+0000","last_fullsized":"2024-05-11T08:06:01.645506+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:35:39.553476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.645631+0000","last_change":"2024-05-11T08:05:54.368776+0000","last_active":"2024-05-11T08:06:01.645631+0000","last_peered":"2024-05-11T08:06:01.645631+0000","last_clean":"2024-05-11T08:06:01.645631+0000","last_became_active":"2024-05-11T08:05:54.368514+0000","last_became_peered":"2024-05-11T08:05:54.368514+0000","last_unstale":"2024-05-11T08:06:01.645631+0000","last_undegraded":"2024-05-11T08:06:01.645631+0000","last_fullsized":"2024-05-11T08:06:01.645631+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_clean_scrub_stamp":"2024-05-11T08:05:53.331701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:01:58.573898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.646534+0000","last_change":"2024-05-11T08:05:59.561579+0000","last_active":"2024-05-11T08:06:01.646534+0000","last_peered":"2024-05-11T08:06:01.646534+0000","last_clean":"2024-05-11T08:06:01.646534+0000","last_became_active":"2024-05-11T08:05:59.561027+0000","last_became_peered":"2024-05-11T08:05:59.561027+0000","last_unstale":"2024-05-11T08:06:01.646534+0000","last_undegraded":"2024-05-11T08:06:01.646534+0000","last_fullsized":"2024-05-11T08:06:01.646534+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_clean_scrub_stamp":"2024-05-11T08:05:58.374552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:06:20.739397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.388603+0000","last_change":"2024-05-11T08:05:55.387188+0000","last_active":"2024-05-11T08:06:00.388603+0000","last_peered":"2024-05-11T08:06:00.388603+0000","last_clean":"2024-05-11T08:06:00.388603+0000","last_became_active":"2024-05-11T08:05:55.386935+0000","last_became_peered":"2024-05-11T08:05:55.386935+0000","last_unstale":"2024-05-11T08:06:00.388603+0000","last_undegraded":"2024-05-11T08:06:00.388603+0000","last_fullsized":"2024-05-11T08:06:00.388603+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:14:09.302889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.389390+0000","last_change":"2024-05-11T08:05:54.370172+0000","last_active":"2024-05-11T08:06:00.389390+0000","last_peered":"2024-05-11T08:06:00.389390+0000","last_clean":"2024-05-11T08:06:00.389390+0000","last_became_active":"2024-05-11T08:05:54.369635+0000","last_became_peered":"2024-05-11T08:05:54.369635+0000","last_unstale":"2024-05-11T08:06:00.389390+0000","last_undegraded":"2024-05-11T08:06:00.389390+0000","last_fullsized":"2024-05-11T08:06:00.389390+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_clean_scrub_stamp":"2024-05-11T08:05:53.331701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:17:22.080661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.647316+0000","last_change":"2024-05-11T08:05:59.559006+0000","last_active":"2024-05-11T08:06:01.647316+0000","last_peered":"2024-05-11T08:06:01.647316+0000","last_clean":"2024-05-11T08:06:01.647316+0000","last_became_active":"2024-05-11T08:05:59.558255+0000","last_became_peered":"2024-05-11T08:05:59.558255+0000","last_unstale":"2024-05-11T08:06:01.647316+0000","last_undegraded":"2024-05-11T08:06:01.647316+0000","last_fullsized":"2024-05-11T08:06:01.647316+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_clean_scrub_stamp":"2024-05-11T08:05:58.374552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:32:53.866284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.392560+0000","last_change":"2024-05-11T08:05:55.375477+0000","last_active":"2024-05-11T08:06:00.392560+0000","last_peered":"2024-05-11T08:06:00.392560+0000","last_clean":"2024-05-11T08:06:00.392560+0000","last_became_active":"2024-05-11T08:05:55.375215+0000","last_became_peered":"2024-05-11T08:05:55.375215+0000","last_unstale":"2024-05-11T08:06:00.392560+0000","last_undegraded":"2024-05-11T08:06:00.392560+0000","last_fullsized":"2024-05-11T08:06:00.392560+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:38:18.066779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"22'2","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.392809+0000","last_change":"2024-05-11T08:05:54.368097+0000","last_active":"2024-05-11T08:06:00.392809+0000","last_peered":"2024-05-11T08:06:00.392809+0000","last_clean":"2024-05-11T08:06:00.392809+0000","last_became_active":"2024-05-11T08:05:54.367812+0000","last_became_peered":"2024-05-11T08:05:54.367812+0000","last_unstale":"2024-05-11T08:06:00.392809+0000","last_undegraded":"2024-05-11T08:06:00.392809+0000","last_fullsized":"2024-05-11T08:06:00.392809+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_clean_scrub_stamp":"2024-05-11T08:05:53.331701+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:00:20.233006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.389793+0000","last_change":"2024-05-11T08:05:59.558753+0000","last_active":"2024-05-11T08:06:00.389793+0000","last_peered":"2024-05-11T08:06:00.389793+0000","last_clean":"2024-05-11T08:06:00.389793+0000","last_became_active":"2024-05-11T08:05:59.558140+0000","last_became_peered":"2024-05-11T08:05:59.558140+0000","last_unstale":"2024-05-11T08:06:00.389793+0000","last_undegraded":"2024-05-11T08:06:00.389793+0000","last_fullsized":"2024-05-11T08:06:00.389793+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_clean_scrub_stamp":"2024-05-11T08:05:58.374552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:54:52.804997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.388229+0000","last_change":"2024-05-11T08:05:55.379691+0000","last_active":"2024-05-11T08:06:00.388229+0000","last_peered":"2024-05-11T08:06:00.388229+0000","last_clean":"2024-05-11T08:06:00.388229+0000","last_became_active":"2024-05-11T08:05:55.378519+0000","last_became_peered":"2024-05-11T08:05:55.378519+0000","last_unstale":"2024-05-11T08:06:00.388229+0000","last_undegraded":"2024-05-11T08:06:00.388229+0000","last_fullsized":"2024-05-11T08:06:00.388229+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:38:02.393559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"27'9","reported_seq":31,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:04.814504+0000","last_change":"2024-05-11T08:05:54.371057+0000","last_active":"2024-05-11T08:06:04.814504+0000","last_peered":"2024-05-11T08:06:04.814504+0000","last_clean":"2024-05-11T08:06:04.814504+0000","last_became_active":"2024-05-11T08:05:54.370746+0000","last_became_peered":"2024-05-11T08:05:54.370746+0000","last_unstale":"2024-05-11T08:06:04.814504+0000","last_undegraded":"2024-05-11T08:06:04.814504+0000","last_fullsized":"2024-05-11T08:06:04.814504+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_clean_scrub_stamp":"2024-05-11T08:05:53.331701+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:17:22.080661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.645768+0000","last_change":"2024-05-11T08:05:59.414118+0000","last_active":"2024-05-11T08:06:01.645768+0000","last_peered":"2024-05-11T08:06:01.645768+0000","last_clean":"2024-05-11T08:06:01.645768+0000","last_became_active":"2024-05-11T08:05:59.413183+0000","last_became_peered":"2024-05-11T08:05:59.413183+0000","last_unstale":"2024-05-11T08:06:01.645768+0000","last_undegraded":"2024-05-11T08:06:01.645768+0000","last_fullsized":"2024-05-11T08:06:01.645768+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_clean_scrub_stamp":"2024-05-11T08:05:58.374552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:28:28.773313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.823505+0000","last_change":"2024-05-11T08:05:55.373641+0000","last_active":"2024-05-11T08:06:00.823505+0000","last_peered":"2024-05-11T08:06:00.823505+0000","last_clean":"2024-05-11T08:06:00.823505+0000","last_became_active":"2024-05-11T08:05:55.373369+0000","last_became_peered":"2024-05-11T08:05:55.373369+0000","last_unstale":"2024-05-11T08:06:00.823505+0000","last_undegraded":"2024-05-11T08:06:00.823505+0000","last_fullsized":"2024-05-11T08:06:00.823505+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:32:44.602233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"22'3","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.645781+0000","last_change":"2024-05-11T08:05:54.366529+0000","last_active":"2024-05-11T08:06:01.645781+0000","last_peered":"2024-05-11T08:06:01.645781+0000","last_clean":"2024-05-11T08:06:01.645781+0000","last_became_active":"2024-05-11T08:05:54.366215+0000","last_became_peered":"2024-05-11T08:05:54.366215+0000","last_unstale":"2024-05-11T08:06:01.645781+0000","last_undegraded":"2024-05-11T08:06:01.645781+0000","last_fullsized":"2024-05-11T08:06:01.645781+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_clean_scrub_stamp":"2024-05-11T08:05:53.331701+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:53:10.612688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.823463+0000","last_change":"2024-05-11T08:05:59.407003+0000","last_active":"2024-05-11T08:06:00.823463+0000","last_peered":"2024-05-11T08:06:00.823463+0000","last_clean":"2024-05-11T08:06:00.823463+0000","last_became_active":"2024-05-11T08:05:59.406437+0000","last_became_peered":"2024-05-11T08:05:59.406437+0000","last_unstale":"2024-05-11T08:06:00.823463+0000","last_undegraded":"2024-05-11T08:06:00.823463+0000","last_fullsized":"2024-05-11T08:06:00.823463+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_clean_scrub_stamp":"2024-05-11T08:05:58.374552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:12:42.864479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.392952+0000","last_change":"2024-05-11T08:05:55.370107+0000","last_active":"2024-05-11T08:06:00.392952+0000","last_peered":"2024-05-11T08:06:00.392952+0000","last_clean":"2024-05-11T08:06:00.392952+0000","last_became_active":"2024-05-11T08:05:55.369823+0000","last_became_peered":"2024-05-11T08:05:55.369823+0000","last_unstale":"2024-05-11T08:06:00.392952+0000","last_undegraded":"2024-05-11T08:06:00.392952+0000","last_fullsized":"2024-05-11T08:06:00.392952+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:22:00.631022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.389731+0000","last_change":"2024-05-11T08:05:54.363934+0000","last_active":"2024-05-11T08:06:00.389731+0000","last_peered":"2024-05-11T08:06:00.389731+0000","last_clean":"2024-05-11T08:06:00.389731+0000","last_became_active":"2024-05-11T08:05:54.363428+0000","last_became_peered":"2024-05-11T08:05:54.363428+0000","last_unstale":"2024-05-11T08:06:00.389731+0000","last_undegraded":"2024-05-11T08:06:00.389731+0000","last_fullsized":"2024-05-11T08:06:00.389731+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_clean_scrub_stamp":"2024-05-11T08:05:53.331701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:30:23.111497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.645683+0000","last_change":"2024-05-11T08:05:59.403691+0000","last_active":"2024-05-11T08:06:01.645683+0000","last_peered":"2024-05-11T08:06:01.645683+0000","last_clean":"2024-05-11T08:06:01.645683+0000","last_became_active":"2024-05-11T08:05:59.403311+0000","last_became_peered":"2024-05-11T08:05:59.403311+0000","last_unstale":"2024-05-11T08:06:01.645683+0000","last_undegraded":"2024-05-11T08:06:01.645683+0000","last_fullsized":"2024-05-11T08:06:01.645683+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_clean_scrub_stamp":"2024-05-11T08:05:58.374552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:54:52.804997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.389689+0000","last_change":"2024-05-11T08:05:55.374886+0000","last_active":"2024-05-11T08:06:00.389689+0000","last_peered":"2024-05-11T08:06:00.389689+0000","last_clean":"2024-05-11T08:06:00.389689+0000","last_became_active":"2024-05-11T08:05:55.374622+0000","last_became_peered":"2024-05-11T08:05:55.374622+0000","last_unstale":"2024-05-11T08:06:00.389689+0000","last_undegraded":"2024-05-11T08:06:00.389689+0000","last_fullsized":"2024-05-11T08:06:00.389689+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:43:37.948904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.646049+0000","last_change":"2024-05-11T08:05:54.364291+0000","last_active":"2024-05-11T08:06:01.646049+0000","last_peered":"2024-05-11T08:06:01.646049+0000","last_clean":"2024-05-11T08:06:01.646049+0000","last_became_active":"2024-05-11T08:05:54.363676+0000","last_became_peered":"2024-05-11T08:05:54.363676+0000","last_unstale":"2024-05-11T08:06:01.646049+0000","last_undegraded":"2024-05-11T08:06:01.646049+0000","last_fullsized":"2024-05-11T08:06:01.646049+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_clean_scrub_stamp":"2024-05-11T08:05:53.331701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:53:10.612688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.647107+0000","last_change":"2024-05-11T08:05:59.558985+0000","last_active":"2024-05-11T08:06:01.647107+0000","last_peered":"2024-05-11T08:06:01.647107+0000","last_clean":"2024-05-11T08:06:01.647107+0000","last_became_active":"2024-05-11T08:05:59.558561+0000","last_became_peered":"2024-05-11T08:05:59.558561+0000","last_unstale":"2024-05-11T08:06:01.647107+0000","last_undegraded":"2024-05-11T08:06:01.647107+0000","last_fullsized":"2024-05-11T08:06:01.647107+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_clean_scrub_stamp":"2024-05-11T08:05:58.374552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:54:53.957903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.392778+0000","last_change":"2024-05-11T08:05:59.561482+0000","last_active":"2024-05-11T08:06:00.392778+0000","last_peered":"2024-05-11T08:06:00.392778+0000","last_clean":"2024-05-11T08:06:00.392778+0000","last_became_active":"2024-05-11T08:05:59.560743+0000","last_became_peered":"2024-05-11T08:05:59.560743+0000","last_unstale":"2024-05-11T08:06:00.392778+0000","last_undegraded":"2024-05-11T08:06:00.392778+0000","last_fullsized":"2024-05-11T08:06:00.392778+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_clean_scrub_stamp":"2024-05-11T08:05:58.374552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:43:24.056017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.645261+0000","last_change":"2024-05-11T08:05:55.367915+0000","last_active":"2024-05-11T08:06:01.645261+0000","last_peered":"2024-05-11T08:06:01.645261+0000","last_clean":"2024-05-11T08:06:01.645261+0000","last_became_active":"2024-05-11T08:05:55.367652+0000","last_became_peered":"2024-05-11T08:05:55.367652+0000","last_unstale":"2024-05-11T08:06:01.645261+0000","last_undegraded":"2024-05-11T08:06:01.645261+0000","last_fullsized":"2024-05-11T08:06:01.645261+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:35:03.601599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.389207+0000","last_change":"2024-05-11T08:05:54.368060+0000","last_active":"2024-05-11T08:06:00.389207+0000","last_peered":"2024-05-11T08:06:00.389207+0000","last_clean":"2024-05-11T08:06:00.389207+0000","last_became_active":"2024-05-11T08:05:54.367770+0000","last_became_peered":"2024-05-11T08:05:54.367770+0000","last_unstale":"2024-05-11T08:06:00.389207+0000","last_undegraded":"2024-05-11T08:06:00.389207+0000","last_fullsized":"2024-05-11T08:06:00.389207+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_clean_scrub_stamp":"2024-05-11T08:05:53.331701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:01:58.573898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.646058+0000","last_change":"2024-05-11T08:05:59.412426+0000","last_active":"2024-05-11T08:06:01.646058+0000","last_peered":"2024-05-11T08:06:01.646058+0000","last_clean":"2024-05-11T08:06:01.646058+0000","last_became_active":"2024-05-11T08:05:59.412011+0000","last_became_peered":"2024-05-11T08:05:59.412011+0000","last_unstale":"2024-05-11T08:06:01.646058+0000","last_undegraded":"2024-05-11T08:06:01.646058+0000","last_fullsized":"2024-05-11T08:06:01.646058+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_clean_scrub_stamp":"2024-05-11T08:05:58.374552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:12:42.864479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.645734+0000","last_change":"2024-05-11T08:05:55.363149+0000","last_active":"2024-05-11T08:06:01.645734+0000","last_peered":"2024-05-11T08:06:01.645734+0000","last_clean":"2024-05-11T08:06:01.645734+0000","last_became_active":"2024-05-11T08:05:55.362892+0000","last_became_peered":"2024-05-11T08:05:55.362892+0000","last_unstale":"2024-05-11T08:06:01.645734+0000","last_undegraded":"2024-05-11T08:06:01.645734+0000","last_fullsized":"2024-05-11T08:06:01.645734+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:38:18.066779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.647209+0000","last_change":"2024-05-11T08:05:54.368847+0000","last_active":"2024-05-11T08:06:01.647209+0000","last_peered":"2024-05-11T08:06:01.647209+0000","last_clean":"2024-05-11T08:06:01.647209+0000","last_became_active":"2024-05-11T08:05:54.368529+0000","last_became_peered":"2024-05-11T08:05:54.368529+0000","last_unstale":"2024-05-11T08:06:01.647209+0000","last_undegraded":"2024-05-11T08:06:01.647209+0000","last_fullsized":"2024-05-11T08:06:01.647209+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_clean_scrub_stamp":"2024-05-11T08:05:53.331701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:22:53.142453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"22'1","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.645710+0000","last_change":"2024-05-11T08:05:54.369435+0000","last_active":"2024-05-11T08:06:01.645710+0000","last_peered":"2024-05-11T08:06:01.645710+0000","last_clean":"2024-05-11T08:06:01.645710+0000","last_became_active":"2024-05-11T08:05:54.369163+0000","last_became_peered":"2024-05-11T08:05:54.369163+0000","last_unstale":"2024-05-11T08:06:01.645710+0000","last_undegraded":"2024-05-11T08:06:01.645710+0000","last_fullsized":"2024-05-11T08:06:01.645710+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_clean_scrub_stamp":"2024-05-11T08:05:53.331701+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:33:25.907598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.392757+0000","last_change":"2024-05-11T08:05:55.378586+0000","last_active":"2024-05-11T08:06:00.392757+0000","last_peered":"2024-05-11T08:06:00.392757+0000","last_clean":"2024-05-11T08:06:00.392757+0000","last_became_active":"2024-05-11T08:05:55.378323+0000","last_became_peered":"2024-05-11T08:05:55.378323+0000","last_unstale":"2024-05-11T08:06:00.392757+0000","last_undegraded":"2024-05-11T08:06:00.392757+0000","last_fullsized":"2024-05-11T08:06:00.392757+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:35:03.601599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.646291+0000","last_change":"2024-05-11T08:05:59.402315+0000","last_active":"2024-05-11T08:06:01.646291+0000","last_peered":"2024-05-11T08:06:01.646291+0000","last_clean":"2024-05-11T08:06:01.646291+0000","last_became_active":"2024-05-11T08:05:59.401886+0000","last_became_peered":"2024-05-11T08:05:59.401886+0000","last_unstale":"2024-05-11T08:06:01.646291+0000","last_undegraded":"2024-05-11T08:06:01.646291+0000","last_fullsized":"2024-05-11T08:06:01.646291+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_clean_scrub_stamp":"2024-05-11T08:05:58.374552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:33:42.953506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"22'1","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.646467+0000","last_change":"2024-05-11T08:05:54.370886+0000","last_active":"2024-05-11T08:06:01.646467+0000","last_peered":"2024-05-11T08:06:01.646467+0000","last_clean":"2024-05-11T08:06:01.646467+0000","last_became_active":"2024-05-11T08:05:54.370611+0000","last_became_peered":"2024-05-11T08:05:54.370611+0000","last_unstale":"2024-05-11T08:06:01.646467+0000","last_undegraded":"2024-05-11T08:06:01.646467+0000","last_fullsized":"2024-05-11T08:06:01.646467+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_clean_scrub_stamp":"2024-05-11T08:05:53.331701+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:36:34.523259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.823139+0000","last_change":"2024-05-11T08:05:55.373391+0000","last_active":"2024-05-11T08:06:00.823139+0000","last_peered":"2024-05-11T08:06:00.823139+0000","last_clean":"2024-05-11T08:06:00.823139+0000","last_became_active":"2024-05-11T08:05:55.373096+0000","last_became_peered":"2024-05-11T08:05:55.373096+0000","last_unstale":"2024-05-11T08:06:00.823139+0000","last_undegraded":"2024-05-11T08:06:00.823139+0000","last_fullsized":"2024-05-11T08:06:00.823139+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:53:11.622973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.647038+0000","last_change":"2024-05-11T08:05:59.403522+0000","last_active":"2024-05-11T08:06:01.647038+0000","last_peered":"2024-05-11T08:06:01.647038+0000","last_clean":"2024-05-11T08:06:01.647038+0000","last_became_active":"2024-05-11T08:05:59.403132+0000","last_became_peered":"2024-05-11T08:05:59.403132+0000","last_unstale":"2024-05-11T08:06:01.647038+0000","last_undegraded":"2024-05-11T08:06:01.647038+0000","last_fullsized":"2024-05-11T08:06:01.647038+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_clean_scrub_stamp":"2024-05-11T08:05:58.374552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:06:03.606038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.645399+0000","last_change":"2024-05-11T08:05:55.364367+0000","last_active":"2024-05-11T08:06:01.645399+0000","last_peered":"2024-05-11T08:06:01.645399+0000","last_clean":"2024-05-11T08:06:01.645399+0000","last_became_active":"2024-05-11T08:05:55.364089+0000","last_became_peered":"2024-05-11T08:05:55.364089+0000","last_unstale":"2024-05-11T08:06:01.645399+0000","last_undegraded":"2024-05-11T08:06:01.645399+0000","last_fullsized":"2024-05-11T08:06:01.645399+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:38:18.066779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.645226+0000","last_change":"2024-05-11T08:05:55.369401+0000","last_active":"2024-05-11T08:06:01.645226+0000","last_peered":"2024-05-11T08:06:01.645226+0000","last_clean":"2024-05-11T08:06:01.645226+0000","last_became_active":"2024-05-11T08:05:55.369154+0000","last_became_peered":"2024-05-11T08:05:55.369154+0000","last_unstale":"2024-05-11T08:06:01.645226+0000","last_undegraded":"2024-05-11T08:06:01.645226+0000","last_fullsized":"2024-05-11T08:06:01.645226+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:46:51.750892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.389323+0000","last_change":"2024-05-11T08:05:55.375787+0000","last_active":"2024-05-11T08:06:00.389323+0000","last_peered":"2024-05-11T08:06:00.389323+0000","last_clean":"2024-05-11T08:06:00.389323+0000","last_became_active":"2024-05-11T08:05:55.374199+0000","last_became_peered":"2024-05-11T08:05:55.374199+0000","last_unstale":"2024-05-11T08:06:00.389323+0000","last_undegraded":"2024-05-11T08:06:00.389323+0000","last_fullsized":"2024-05-11T08:06:00.389323+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:38:02.393559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.392641+0000","last_change":"2024-05-11T08:05:55.369439+0000","last_active":"2024-05-11T08:06:00.392641+0000","last_peered":"2024-05-11T08:06:00.392641+0000","last_clean":"2024-05-11T08:06:00.392641+0000","last_became_active":"2024-05-11T08:05:55.369182+0000","last_became_peered":"2024-05-11T08:05:55.369182+0000","last_unstale":"2024-05-11T08:06:00.392641+0000","last_undegraded":"2024-05-11T08:06:00.392641+0000","last_fullsized":"2024-05-11T08:06:00.392641+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:35:39.553476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.647343+0000","last_change":"2024-05-11T08:05:55.373779+0000","last_active":"2024-05-11T08:06:01.647343+0000","last_peered":"2024-05-11T08:06:01.647343+0000","last_clean":"2024-05-11T08:06:01.647343+0000","last_became_active":"2024-05-11T08:05:55.373523+0000","last_became_peered":"2024-05-11T08:05:55.373523+0000","last_unstale":"2024-05-11T08:06:01.647343+0000","last_undegraded":"2024-05-11T08:06:01.647343+0000","last_fullsized":"2024-05-11T08:06:01.647343+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:35:39.553476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.645671+0000","last_change":"2024-05-11T08:05:55.367765+0000","last_active":"2024-05-11T08:06:01.645671+0000","last_peered":"2024-05-11T08:06:01.645671+0000","last_clean":"2024-05-11T08:06:01.645671+0000","last_became_active":"2024-05-11T08:05:55.367521+0000","last_became_peered":"2024-05-11T08:05:55.367521+0000","last_unstale":"2024-05-11T08:06:01.645671+0000","last_undegraded":"2024-05-11T08:06:01.645671+0000","last_fullsized":"2024-05-11T08:06:01.645671+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:35:39.553476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.646145+0000","last_change":"2024-05-11T08:05:55.373963+0000","last_active":"2024-05-11T08:06:01.646145+0000","last_peered":"2024-05-11T08:06:01.646145+0000","last_clean":"2024-05-11T08:06:01.646145+0000","last_became_active":"2024-05-11T08:05:55.373700+0000","last_became_peered":"2024-05-11T08:05:55.373700+0000","last_unstale":"2024-05-11T08:06:01.646145+0000","last_undegraded":"2024-05-11T08:06:01.646145+0000","last_fullsized":"2024-05-11T08:06:01.646145+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:35:03.601599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.646584+0000","last_change":"2024-05-11T08:05:55.374020+0000","last_active":"2024-05-11T08:06:01.646584+0000","last_peered":"2024-05-11T08:06:01.646584+0000","last_clean":"2024-05-11T08:06:01.646584+0000","last_became_active":"2024-05-11T08:05:55.373697+0000","last_became_peered":"2024-05-11T08:05:55.373697+0000","last_unstale":"2024-05-11T08:06:01.646584+0000","last_undegraded":"2024-05-11T08:06:01.646584+0000","last_fullsized":"2024-05-11T08:06:01.646584+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:02:23.158539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.389077+0000","last_change":"2024-05-11T08:05:55.371797+0000","last_active":"2024-05-11T08:06:00.389077+0000","last_peered":"2024-05-11T08:06:00.389077+0000","last_clean":"2024-05-11T08:06:00.389077+0000","last_became_active":"2024-05-11T08:05:55.371535+0000","last_became_peered":"2024-05-11T08:05:55.371535+0000","last_unstale":"2024-05-11T08:06:00.389077+0000","last_undegraded":"2024-05-11T08:06:00.389077+0000","last_fullsized":"2024-05-11T08:06:00.389077+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:16:46.423362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.647177+0000","last_change":"2024-05-11T08:05:55.373536+0000","last_active":"2024-05-11T08:06:01.647177+0000","last_peered":"2024-05-11T08:06:01.647177+0000","last_clean":"2024-05-11T08:06:01.647177+0000","last_became_active":"2024-05-11T08:05:55.373273+0000","last_became_peered":"2024-05-11T08:05:55.373273+0000","last_unstale":"2024-05-11T08:06:01.647177+0000","last_undegraded":"2024-05-11T08:06:01.647177+0000","last_fullsized":"2024-05-11T08:06:01.647177+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:35:03.601599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.389188+0000","last_change":"2024-05-11T08:05:55.382077+0000","last_active":"2024-05-11T08:06:00.389188+0000","last_peered":"2024-05-11T08:06:00.389188+0000","last_clean":"2024-05-11T08:06:00.389188+0000","last_became_active":"2024-05-11T08:05:55.381767+0000","last_became_peered":"2024-05-11T08:05:55.381767+0000","last_unstale":"2024-05-11T08:06:00.389188+0000","last_undegraded":"2024-05-11T08:06:00.389188+0000","last_fullsized":"2024-05-11T08:06:00.389188+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:56:59.948641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.392492+0000","last_change":"2024-05-11T08:05:55.374245+0000","last_active":"2024-05-11T08:06:00.392492+0000","last_peered":"2024-05-11T08:06:00.392492+0000","last_clean":"2024-05-11T08:06:00.392492+0000","last_became_active":"2024-05-11T08:05:55.373871+0000","last_became_peered":"2024-05-11T08:05:55.373871+0000","last_unstale":"2024-05-11T08:06:00.392492+0000","last_undegraded":"2024-05-11T08:06:00.392492+0000","last_fullsized":"2024-05-11T08:06:00.392492+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:32:48.010809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.645850+0000","last_change":"2024-05-11T08:05:55.376534+0000","last_active":"2024-05-11T08:06:01.645850+0000","last_peered":"2024-05-11T08:06:01.645850+0000","last_clean":"2024-05-11T08:06:01.645850+0000","last_became_active":"2024-05-11T08:05:55.376265+0000","last_became_peered":"2024-05-11T08:05:55.376265+0000","last_unstale":"2024-05-11T08:06:01.645850+0000","last_undegraded":"2024-05-11T08:06:01.645850+0000","last_fullsized":"2024-05-11T08:06:01.645850+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:43:37.948904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.388549+0000","last_change":"2024-05-11T08:05:55.381048+0000","last_active":"2024-05-11T08:06:00.388549+0000","last_peered":"2024-05-11T08:06:00.388549+0000","last_clean":"2024-05-11T08:06:00.388549+0000","last_became_active":"2024-05-11T08:05:55.380781+0000","last_became_peered":"2024-05-11T08:05:55.380781+0000","last_unstale":"2024-05-11T08:06:00.388549+0000","last_undegraded":"2024-05-11T08:06:00.388549+0000","last_fullsized":"2024-05-11T08:06:00.388549+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:28:24.740747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.645586+0000","last_change":"2024-05-11T08:05:55.364750+0000","last_active":"2024-05-11T08:06:01.645586+0000","last_peered":"2024-05-11T08:06:01.645586+0000","last_clean":"2024-05-11T08:06:01.645586+0000","last_became_active":"2024-05-11T08:05:55.364495+0000","last_became_peered":"2024-05-11T08:05:55.364495+0000","last_unstale":"2024-05-11T08:06:01.645586+0000","last_undegraded":"2024-05-11T08:06:01.645586+0000","last_fullsized":"2024-05-11T08:06:01.645586+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:43:46.282095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.389971+0000","last_change":"2024-05-11T08:05:55.375569+0000","last_active":"2024-05-11T08:06:00.389971+0000","last_peered":"2024-05-11T08:06:00.389971+0000","last_clean":"2024-05-11T08:06:00.389971+0000","last_became_active":"2024-05-11T08:05:55.375303+0000","last_became_peered":"2024-05-11T08:05:55.375303+0000","last_unstale":"2024-05-11T08:06:00.389971+0000","last_undegraded":"2024-05-11T08:06:00.389971+0000","last_fullsized":"2024-05-11T08:06:00.389971+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:41:38.931447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.389063+0000","last_change":"2024-05-11T08:05:55.377664+0000","last_active":"2024-05-11T08:06:00.389063+0000","last_peered":"2024-05-11T08:06:00.389063+0000","last_clean":"2024-05-11T08:06:00.389063+0000","last_became_active":"2024-05-11T08:05:55.377373+0000","last_became_peered":"2024-05-11T08:05:55.377373+0000","last_unstale":"2024-05-11T08:06:00.389063+0000","last_undegraded":"2024-05-11T08:06:00.389063+0000","last_fullsized":"2024-05-11T08:06:00.389063+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:35:03.601599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.645312+0000","last_change":"2024-05-11T08:05:55.373802+0000","last_active":"2024-05-11T08:06:01.645312+0000","last_peered":"2024-05-11T08:06:01.645312+0000","last_clean":"2024-05-11T08:06:01.645312+0000","last_became_active":"2024-05-11T08:05:55.373558+0000","last_became_peered":"2024-05-11T08:05:55.373558+0000","last_unstale":"2024-05-11T08:06:01.645312+0000","last_undegraded":"2024-05-11T08:06:01.645312+0000","last_fullsized":"2024-05-11T08:06:01.645312+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:43:37.948904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.392536+0000","last_change":"2024-05-11T08:05:55.366027+0000","last_active":"2024-05-11T08:06:00.392536+0000","last_peered":"2024-05-11T08:06:00.392536+0000","last_clean":"2024-05-11T08:06:00.392536+0000","last_became_active":"2024-05-11T08:05:55.365360+0000","last_became_peered":"2024-05-11T08:05:55.365360+0000","last_unstale":"2024-05-11T08:06:00.392536+0000","last_undegraded":"2024-05-11T08:06:00.392536+0000","last_fullsized":"2024-05-11T08:06:00.392536+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:20:43.832246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.646413+0000","last_change":"2024-05-11T08:05:55.369665+0000","last_active":"2024-05-11T08:06:01.646413+0000","last_peered":"2024-05-11T08:06:01.646413+0000","last_clean":"2024-05-11T08:06:01.646413+0000","last_became_active":"2024-05-11T08:05:55.369420+0000","last_became_peered":"2024-05-11T08:05:55.369420+0000","last_unstale":"2024-05-11T08:06:01.646413+0000","last_undegraded":"2024-05-11T08:06:01.646413+0000","last_fullsized":"2024-05-11T08:06:01.646413+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:43:37.948904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.823465+0000","last_change":"2024-05-11T08:05:55.377949+0000","last_active":"2024-05-11T08:06:00.823465+0000","last_peered":"2024-05-11T08:06:00.823465+0000","last_clean":"2024-05-11T08:06:00.823465+0000","last_became_active":"2024-05-11T08:05:55.377677+0000","last_became_peered":"2024-05-11T08:05:55.377677+0000","last_unstale":"2024-05-11T08:06:00.823465+0000","last_undegraded":"2024-05-11T08:06:00.823465+0000","last_fullsized":"2024-05-11T08:06:00.823465+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:50:18.627516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.645928+0000","last_change":"2024-05-11T08:05:55.368388+0000","last_active":"2024-05-11T08:06:01.645928+0000","last_peered":"2024-05-11T08:06:01.645928+0000","last_clean":"2024-05-11T08:06:01.645928+0000","last_became_active":"2024-05-11T08:05:55.368103+0000","last_became_peered":"2024-05-11T08:05:55.368103+0000","last_unstale":"2024-05-11T08:06:01.645928+0000","last_undegraded":"2024-05-11T08:06:01.645928+0000","last_fullsized":"2024-05-11T08:06:01.645928+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:32:48.010809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.646145+0000","last_change":"2024-05-11T08:05:55.368708+0000","last_active":"2024-05-11T08:06:01.646145+0000","last_peered":"2024-05-11T08:06:01.646145+0000","last_clean":"2024-05-11T08:06:01.646145+0000","last_became_active":"2024-05-11T08:05:55.368177+0000","last_became_peered":"2024-05-11T08:05:55.368177+0000","last_unstale":"2024-05-11T08:06:01.646145+0000","last_undegraded":"2024-05-11T08:06:01.646145+0000","last_fullsized":"2024-05-11T08:06:01.646145+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:02:23.158539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.388971+0000","last_change":"2024-05-11T08:05:55.383485+0000","last_active":"2024-05-11T08:06:00.388971+0000","last_peered":"2024-05-11T08:06:00.388971+0000","last_clean":"2024-05-11T08:06:00.388971+0000","last_became_active":"2024-05-11T08:05:55.383016+0000","last_became_peered":"2024-05-11T08:05:55.383016+0000","last_unstale":"2024-05-11T08:06:00.388971+0000","last_undegraded":"2024-05-11T08:06:00.388971+0000","last_fullsized":"2024-05-11T08:06:00.388971+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:51:26.849946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.645483+0000","last_change":"2024-05-11T08:05:55.370103+0000","last_active":"2024-05-11T08:06:01.645483+0000","last_peered":"2024-05-11T08:06:01.645483+0000","last_clean":"2024-05-11T08:06:01.645483+0000","last_became_active":"2024-05-11T08:05:55.369838+0000","last_became_peered":"2024-05-11T08:05:55.369838+0000","last_unstale":"2024-05-11T08:06:01.645483+0000","last_undegraded":"2024-05-11T08:06:01.645483+0000","last_fullsized":"2024-05-11T08:06:01.645483+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:20:43.832246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.388264+0000","last_change":"2024-05-11T08:05:55.379192+0000","last_active":"2024-05-11T08:06:00.388264+0000","last_peered":"2024-05-11T08:06:00.388264+0000","last_clean":"2024-05-11T08:06:00.388264+0000","last_became_active":"2024-05-11T08:05:55.378919+0000","last_became_peered":"2024-05-11T08:05:55.378919+0000","last_unstale":"2024-05-11T08:06:00.388264+0000","last_undegraded":"2024-05-11T08:06:00.388264+0000","last_fullsized":"2024-05-11T08:06:00.388264+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:22:00.631022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.389205+0000","last_change":"2024-05-11T08:05:55.379230+0000","last_active":"2024-05-11T08:06:00.389205+0000","last_peered":"2024-05-11T08:06:00.389205+0000","last_clean":"2024-05-11T08:06:00.389205+0000","last_became_active":"2024-05-11T08:05:55.378960+0000","last_became_peered":"2024-05-11T08:05:55.378960+0000","last_unstale":"2024-05-11T08:06:00.389205+0000","last_undegraded":"2024-05-11T08:06:00.389205+0000","last_fullsized":"2024-05-11T08:06:00.389205+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:22:00.631022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.388309+0000","last_change":"2024-05-11T08:05:55.378492+0000","last_active":"2024-05-11T08:06:00.388309+0000","last_peered":"2024-05-11T08:06:00.388309+0000","last_clean":"2024-05-11T08:06:00.388309+0000","last_became_active":"2024-05-11T08:05:55.378200+0000","last_became_peered":"2024-05-11T08:05:55.378200+0000","last_unstale":"2024-05-11T08:06:00.388309+0000","last_undegraded":"2024-05-11T08:06:00.388309+0000","last_fullsized":"2024-05-11T08:06:00.388309+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:44:43.387850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.389503+0000","last_change":"2024-05-11T08:05:55.379296+0000","last_active":"2024-05-11T08:06:00.389503+0000","last_peered":"2024-05-11T08:06:00.389503+0000","last_clean":"2024-05-11T08:06:00.389503+0000","last_became_active":"2024-05-11T08:05:55.379047+0000","last_became_peered":"2024-05-11T08:05:55.379047+0000","last_unstale":"2024-05-11T08:06:00.389503+0000","last_undegraded":"2024-05-11T08:06:00.389503+0000","last_fullsized":"2024-05-11T08:06:00.389503+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:44:43.387850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.392612+0000","last_change":"2024-05-11T08:05:55.360980+0000","last_active":"2024-05-11T08:06:00.392612+0000","last_peered":"2024-05-11T08:06:00.392612+0000","last_clean":"2024-05-11T08:06:00.392612+0000","last_became_active":"2024-05-11T08:05:55.360626+0000","last_became_peered":"2024-05-11T08:05:55.360626+0000","last_unstale":"2024-05-11T08:06:00.392612+0000","last_undegraded":"2024-05-11T08:06:00.392612+0000","last_fullsized":"2024-05-11T08:06:00.392612+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:02:23.158539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.646210+0000","last_change":"2024-05-11T08:05:55.363639+0000","last_active":"2024-05-11T08:06:01.646210+0000","last_peered":"2024-05-11T08:06:01.646210+0000","last_clean":"2024-05-11T08:06:01.646210+0000","last_became_active":"2024-05-11T08:05:55.363320+0000","last_became_peered":"2024-05-11T08:05:55.363320+0000","last_unstale":"2024-05-11T08:06:01.646210+0000","last_undegraded":"2024-05-11T08:06:01.646210+0000","last_fullsized":"2024-05-11T08:06:01.646210+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:02:23.158539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.388821+0000","last_change":"2024-05-11T08:05:55.380035+0000","last_active":"2024-05-11T08:06:00.388821+0000","last_peered":"2024-05-11T08:06:00.388821+0000","last_clean":"2024-05-11T08:06:00.388821+0000","last_became_active":"2024-05-11T08:05:55.378781+0000","last_became_peered":"2024-05-11T08:05:55.378781+0000","last_unstale":"2024-05-11T08:06:00.388821+0000","last_undegraded":"2024-05-11T08:06:00.388821+0000","last_fullsized":"2024-05-11T08:06:00.388821+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:45:24.551499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509449,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27884,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343956,"statfs":{"total":96636764160,"available":96608210944,"internally_reserved":0,"allocated":880640,"data_stored":701507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509449,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":294912,"data_stored":114399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509449,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":286720,"data_stored":110647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509449,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":294912,"data_stored":110683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509450,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":294912,"data_stored":114399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509450,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":290816,"data_stored":110717,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509449,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27892,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343948,"statfs":{"total":96636764160,"available":96608202752,"internally_reserved":0,"allocated":888832,"data_stored":701563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574857,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":282624,"data_stored":110557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-11T08:06:15.429 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-11T08:06:15.564 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.563+0000 7f1fa7e47640 1 Processor -- start 2024-05-11T08:06:15.564 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.563+0000 7f1fa7e47640 1 -- start start 2024-05-11T08:06:15.565 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.564+0000 7f1fa7e47640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f1fa01573a0 0x7f1fa0157830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:15.565 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.564+0000 7f1fa7e47640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1fa00d0ec0 con 0x7f1fa0071cc0 2024-05-11T08:06:15.565 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.564+0000 7f1fa7e47640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1fa00d11a0 con 0x7f1fa00718e0 2024-05-11T08:06:15.566 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.564+0000 7f1fa7e47640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f1fa00d1d20 con 0x7f1fa01573a0 2024-05-11T08:06:15.566 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.564+0000 7f1fa53e2640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f1fa0071cc0 0x7f1fa00cd610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:60744/0 (socket says 172.21.15.167:60744) 2024-05-11T08:06:15.566 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.564+0000 7f1fa53e2640 1 -- 172.21.15.167:0/2491261174 learned_addr learned my addr 172.21.15.167:0/2491261174 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T08:06:15.566 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.564+0000 7f1fa63e4640 1 --2- 172.21.15.167:0/2491261174 >> v2:172.21.15.204:3300/0 conn(0x7f1fa01573a0 0x7f1fa0157830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:15.566 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.564+0000 7f1fa4be1640 1 -- 172.21.15.167:0/2491261174 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3197756009 0 0) 0x7f1fa00d0ec0 con 0x7f1fa0071cc0 2024-05-11T08:06:15.566 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.564+0000 7f1fa4be1640 1 -- 172.21.15.167:0/2491261174 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1f8c003520 con 0x7f1fa0071cc0 2024-05-11T08:06:15.566 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.564+0000 7f1fa4be1640 1 -- 172.21.15.167:0/2491261174 <== mon.0 v1:172.21.15.167:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 216364291 0 0) 0x7f1f8c003520 con 0x7f1fa0071cc0 2024-05-11T08:06:15.566 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.564+0000 7f1fa63e4640 1 -- 172.21.15.167:0/2491261174 >> v1:172.21.15.204:6789/0 conn(0x7f1fa00718e0 legacy=0x7f1fa0160e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:15.566 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.564+0000 7f1fa63e4640 1 -- 172.21.15.167:0/2491261174 >> v1:172.21.15.167:6789/0 conn(0x7f1fa0071cc0 legacy=0x7f1fa00cd610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:15.566 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.564+0000 7f1fa63e4640 1 -- 172.21.15.167:0/2491261174 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fa00d1480 con 0x7f1fa01573a0 2024-05-11T08:06:15.568 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.565+0000 7f1fa63e4640 1 --2- 172.21.15.167:0/2491261174 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1fa01573a0 0x7f1fa0157830 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1f98009dc0 tx=0x7f1f9802a630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=98398939f1dcc074 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:15.568 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.565+0000 7f1fa4be1640 1 -- 172.21.15.167:0/2491261174 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f900016f0 con 0x7f1fa01573a0 2024-05-11T08:06:15.568 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.565+0000 7f1fa4be1640 1 -- 172.21.15.167:0/2491261174 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f98007540 con 0x7f1fa01573a0 2024-05-11T08:06:15.568 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.565+0000 7f1fa4be1640 1 -- 172.21.15.167:0/2491261174 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f980078f0 con 0x7f1fa01573a0 2024-05-11T08:06:15.568 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.566+0000 7f1fa7e47640 1 -- 172.21.15.167:0/2491261174 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1fa01573a0 msgr2=0x7f1fa0157830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:15.568 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.566+0000 7f1fa7e47640 1 --2- 172.21.15.167:0/2491261174 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1fa01573a0 0x7f1fa0157830 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1f98009dc0 tx=0x7f1f9802a630 comp rx=0 tx=0).stop 2024-05-11T08:06:15.568 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.566+0000 7f1fa7e47640 1 -- 172.21.15.167:0/2491261174 shutdown_connections 2024-05-11T08:06:15.568 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.566+0000 7f1fa7e47640 1 --2- 172.21.15.167:0/2491261174 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1fa01573a0 0x7f1fa0157830 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:15.568 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.566+0000 7f1fa7e47640 1 -- 172.21.15.167:0/2491261174 >> 172.21.15.167:0/2491261174 conn(0x7f1fa0072730 msgr2=0x7f1fa00d2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:15.568 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.566+0000 7f1fa7e47640 1 -- 172.21.15.167:0/2491261174 shutdown_connections 2024-05-11T08:06:15.568 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.566+0000 7f1fa7e47640 1 -- 172.21.15.167:0/2491261174 wait complete. 2024-05-11T08:06:15.571 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.567+0000 7f1fa7e47640 1 Processor -- start 2024-05-11T08:06:15.571 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.567+0000 7f1fa7e47640 1 -- start start 2024-05-11T08:06:15.571 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.568+0000 7f1fa7e47640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1fa00718e0 0x7f1fa01525f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:15.571 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.568+0000 7f1fa7e47640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1fa0071cc0 0x7f1fa00cc260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:15.571 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.568+0000 7f1fa7e47640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1fa01573a0 0x7f1fa00cc7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:15.571 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.568+0000 7f1fa7e47640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1fa00d1a40 con 0x7f1fa00718e0 2024-05-11T08:06:15.571 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.568+0000 7f1fa7e47640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1fa0157d70 con 0x7f1fa01573a0 2024-05-11T08:06:15.571 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.568+0000 7f1fa7e47640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f1fa00cdd20 con 0x7f1fa0071cc0 2024-05-11T08:06:15.572 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.568+0000 7f1fa5be3640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1fa00718e0 0x7f1fa01525f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:15.572 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.568+0000 7f1fa5be3640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1fa00718e0 0x7f1fa01525f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:54162/0 (socket says 172.21.15.167:54162) 2024-05-11T08:06:15.572 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.568+0000 7f1fa5be3640 1 -- 172.21.15.167:0/1542251680 learned_addr learned my addr 172.21.15.167:0/1542251680 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:15.572 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.568+0000 7f1fa63e4640 1 --2- 172.21.15.167:0/1542251680 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1fa01573a0 0x7f1fa00cc7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:15.572 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.568+0000 7f1fa53e2640 1 --2- 172.21.15.167:0/1542251680 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1fa0071cc0 0x7f1fa00cc260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:15.572 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.568+0000 7f1fa5be3640 1 -- 172.21.15.167:0/1542251680 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1fa0071cc0 msgr2=0x7f1fa00cc260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:15.572 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.568+0000 7f1fa5be3640 1 --2- 172.21.15.167:0/1542251680 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1fa0071cc0 0x7f1fa00cc260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:15.572 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.568+0000 7f1fa5be3640 1 -- 172.21.15.167:0/1542251680 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1fa01573a0 msgr2=0x7f1fa00cc7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:15.572 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.568+0000 7f1fa5be3640 1 --2- 172.21.15.167:0/1542251680 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1fa01573a0 0x7f1fa00cc7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:15.572 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.568+0000 7f1fa5be3640 1 -- 172.21.15.167:0/1542251680 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fa01423f0 con 0x7f1fa00718e0 2024-05-11T08:06:15.572 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.568+0000 7f1fa53e2640 1 --2- 172.21.15.167:0/1542251680 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1fa0071cc0 0x7f1fa00cc260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T08:06:15.572 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.568+0000 7f1fa63e4640 1 --2- 172.21.15.167:0/1542251680 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1fa01573a0 0x7f1fa00cc7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T08:06:15.573 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.569+0000 7f1fa5be3640 1 --2- 172.21.15.167:0/1542251680 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1fa00718e0 0x7f1fa01525f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f1f9c00c530 tx=0x7f1f9c00c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:15.573 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.569+0000 7f1f967fc640 1 -- 172.21.15.167:0/1542251680 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f9c0052d0 con 0x7f1fa00718e0 2024-05-11T08:06:15.573 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.569+0000 7f1f967fc640 1 -- 172.21.15.167:0/1542251680 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f9c005470 con 0x7f1fa00718e0 2024-05-11T08:06:15.573 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.569+0000 7f1fa7e47640 1 -- 172.21.15.167:0/1542251680 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fa01f3b60 con 0x7f1fa00718e0 2024-05-11T08:06:15.573 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.569+0000 7f1f967fc640 1 -- 172.21.15.167:0/1542251680 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f9c012870 con 0x7f1fa00718e0 2024-05-11T08:06:15.573 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.569+0000 7f1fa7e47640 1 -- 172.21.15.167:0/1542251680 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fa0152b30 con 0x7f1fa00718e0 2024-05-11T08:06:15.573 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.570+0000 7f1fa7e47640 1 -- 172.21.15.167:0/1542251680 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fa0069590 con 0x7f1fa00718e0 2024-05-11T08:06:15.574 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.570+0000 7f1f967fc640 1 -- 172.21.15.167:0/1542251680 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f1f9c012a10 con 0x7f1fa00718e0 2024-05-11T08:06:15.574 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.571+0000 7f1f967fc640 1 --2- 172.21.15.167:0/1542251680 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f1f840762d0 0x7f1f84078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:15.574 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.571+0000 7f1fa53e2640 1 --2- 172.21.15.167:0/1542251680 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f1f840762d0 0x7f1f84078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:15.574 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.571+0000 7f1f967fc640 1 -- 172.21.15.167:0/1542251680 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f1f9c097550 con 0x7f1fa00718e0 2024-05-11T08:06:15.574 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.571+0000 7f1fa53e2640 1 --2- 172.21.15.167:0/1542251680 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f1f840762d0 0x7f1f84078790 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1f90001220 tx=0x7f1f90009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:15.576 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.575+0000 7f1f967fc640 1 -- 172.21.15.167:0/1542251680 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f1f9c060b70 con 0x7f1fa00718e0 2024-05-11T08:06:15.757 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.756+0000 7f1fa7e47640 1 -- 172.21.15.167:0/1542251680 --> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1fa00c5f50 con 0x7f1f840762d0 2024-05-11T08:06:15.765 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.764+0000 7f1f967fc640 1 -- 172.21.15.167:0/1542251680 <== mgr.4113 v2:172.21.15.167:6832/253569496 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332523 (secure 0 0 0) 0x7f1fa00c5f50 con 0x7f1f840762d0 2024-05-11T08:06:15.768 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T08:06:15.770 INFO:teuthology.orchestra.run.smithi167.stderr:dumped all 2024-05-11T08:06:15.771 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.767+0000 7f1fa7e47640 1 -- 172.21.15.167:0/1542251680 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f1f840762d0 msgr2=0x7f1f84078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:15.772 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.768+0000 7f1fa7e47640 1 --2- 172.21.15.167:0/1542251680 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f1f840762d0 0x7f1f84078790 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1f90001220 tx=0x7f1f90009040 comp rx=0 tx=0).stop 2024-05-11T08:06:15.772 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.768+0000 7f1fa7e47640 1 -- 172.21.15.167:0/1542251680 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1fa00718e0 msgr2=0x7f1fa01525f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:15.772 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.768+0000 7f1fa7e47640 1 --2- 172.21.15.167:0/1542251680 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1fa00718e0 0x7f1fa01525f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f1f9c00c530 tx=0x7f1f9c00c9f0 comp rx=0 tx=0).stop 2024-05-11T08:06:15.772 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.768+0000 7f1fa7e47640 1 -- 172.21.15.167:0/1542251680 shutdown_connections 2024-05-11T08:06:15.772 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.768+0000 7f1fa7e47640 1 --2- 172.21.15.167:0/1542251680 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f1f840762d0 0x7f1f84078790 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:15.772 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.768+0000 7f1fa7e47640 1 --2- 172.21.15.167:0/1542251680 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1fa01573a0 0x7f1fa00cc7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:15.772 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.768+0000 7f1fa7e47640 1 --2- 172.21.15.167:0/1542251680 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1fa0071cc0 0x7f1fa00cc260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:15.772 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.768+0000 7f1fa7e47640 1 --2- 172.21.15.167:0/1542251680 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1fa00718e0 0x7f1fa01525f0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:15.772 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.768+0000 7f1fa7e47640 1 -- 172.21.15.167:0/1542251680 >> 172.21.15.167:0/1542251680 conn(0x7f1fa0072730 msgr2=0x7f1fa015bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:15.772 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.768+0000 7f1fa7e47640 1 -- 172.21.15.167:0/1542251680 shutdown_connections 2024-05-11T08:06:15.772 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.768+0000 7f1fa7e47640 1 -- 172.21.15.167:0/1542251680 wait complete. 2024-05-11T08:06:15.780 INFO:teuthology.orchestra.run.smithi167.stdout:{"pg_ready":true,"pg_map":{"version":37,"stamp":"2024-05-11T08:06:13.921061+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":149,"num_write_kb":1646,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":114,"ondisk_log_size":114,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219604,"kb_used_data":3432,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755116,"statfs":{"total":773094113280,"available":772869238784,"internally_reserved":0,"allocated":3514368,"data_stored":2074472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003049"},"pg_stats":[{"pgid":"2.1f","version":"22'4","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.392985+0000","last_change":"2024-05-11T08:05:54.368188+0000","last_active":"2024-05-11T08:06:00.392985+0000","last_peered":"2024-05-11T08:06:00.392985+0000","last_clean":"2024-05-11T08:06:00.392985+0000","last_became_active":"2024-05-11T08:05:54.367881+0000","last_became_peered":"2024-05-11T08:05:54.367881+0000","last_unstale":"2024-05-11T08:06:00.392985+0000","last_undegraded":"2024-05-11T08:06:00.392985+0000","last_fullsized":"2024-05-11T08:06:00.392985+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_clean_scrub_stamp":"2024-05-11T08:05:53.331701+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:22:53.142453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.645941+0000","last_change":"2024-05-11T08:05:55.367955+0000","last_active":"2024-05-11T08:06:01.645941+0000","last_peered":"2024-05-11T08:06:01.645941+0000","last_clean":"2024-05-11T08:06:01.645941+0000","last_became_active":"2024-05-11T08:05:55.366260+0000","last_became_peered":"2024-05-11T08:05:55.366260+0000","last_unstale":"2024-05-11T08:06:01.645941+0000","last_undegraded":"2024-05-11T08:06:01.645941+0000","last_fullsized":"2024-05-11T08:06:01.645941+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:44:43.387850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.647113+0000","last_change":"2024-05-11T08:05:59.558909+0000","last_active":"2024-05-11T08:06:01.647113+0000","last_peered":"2024-05-11T08:06:01.647113+0000","last_clean":"2024-05-11T08:06:01.647113+0000","last_became_active":"2024-05-11T08:05:59.558344+0000","last_became_peered":"2024-05-11T08:05:59.558344+0000","last_unstale":"2024-05-11T08:06:01.647113+0000","last_undegraded":"2024-05-11T08:06:01.647113+0000","last_fullsized":"2024-05-11T08:06:01.647113+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_clean_scrub_stamp":"2024-05-11T08:05:58.374552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:50:45.024927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"22'1","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.646170+0000","last_change":"2024-05-11T08:05:54.371497+0000","last_active":"2024-05-11T08:06:01.646170+0000","last_peered":"2024-05-11T08:06:01.646170+0000","last_clean":"2024-05-11T08:06:01.646170+0000","last_became_active":"2024-05-11T08:05:54.371241+0000","last_became_peered":"2024-05-11T08:05:54.371241+0000","last_unstale":"2024-05-11T08:06:01.646170+0000","last_undegraded":"2024-05-11T08:06:01.646170+0000","last_fullsized":"2024-05-11T08:06:01.646170+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_clean_scrub_stamp":"2024-05-11T08:05:53.331701+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:33:25.907598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.392845+0000","last_change":"2024-05-11T08:05:55.369937+0000","last_active":"2024-05-11T08:06:00.392845+0000","last_peered":"2024-05-11T08:06:00.392845+0000","last_clean":"2024-05-11T08:06:00.392845+0000","last_became_active":"2024-05-11T08:05:55.369476+0000","last_became_peered":"2024-05-11T08:05:55.369476+0000","last_unstale":"2024-05-11T08:06:00.392845+0000","last_undegraded":"2024-05-11T08:06:00.392845+0000","last_fullsized":"2024-05-11T08:06:00.392845+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:16:46.423362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.823013+0000","last_change":"2024-05-11T08:05:59.407301+0000","last_active":"2024-05-11T08:06:00.823013+0000","last_peered":"2024-05-11T08:06:00.823013+0000","last_clean":"2024-05-11T08:06:00.823013+0000","last_became_active":"2024-05-11T08:05:59.406627+0000","last_became_peered":"2024-05-11T08:05:59.406627+0000","last_unstale":"2024-05-11T08:06:00.823013+0000","last_undegraded":"2024-05-11T08:06:00.823013+0000","last_fullsized":"2024-05-11T08:06:00.823013+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_clean_scrub_stamp":"2024-05-11T08:05:58.374552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:41:42.964013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"22'2","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.388522+0000","last_change":"2024-05-11T08:05:54.367998+0000","last_active":"2024-05-11T08:06:00.388522+0000","last_peered":"2024-05-11T08:06:00.388522+0000","last_clean":"2024-05-11T08:06:00.388522+0000","last_became_active":"2024-05-11T08:05:54.367681+0000","last_became_peered":"2024-05-11T08:05:54.367681+0000","last_unstale":"2024-05-11T08:06:00.388522+0000","last_undegraded":"2024-05-11T08:06:00.388522+0000","last_fullsized":"2024-05-11T08:06:00.388522+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_clean_scrub_stamp":"2024-05-11T08:05:53.331701+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:50:17.617231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.389537+0000","last_change":"2024-05-11T08:05:55.379931+0000","last_active":"2024-05-11T08:06:00.389537+0000","last_peered":"2024-05-11T08:06:00.389537+0000","last_clean":"2024-05-11T08:06:00.389537+0000","last_became_active":"2024-05-11T08:05:55.379656+0000","last_became_peered":"2024-05-11T08:05:55.379656+0000","last_unstale":"2024-05-11T08:06:00.389537+0000","last_undegraded":"2024-05-11T08:06:00.389537+0000","last_fullsized":"2024-05-11T08:06:00.389537+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:14:09.302889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.823238+0000","last_change":"2024-05-11T08:05:59.404405+0000","last_active":"2024-05-11T08:06:00.823238+0000","last_peered":"2024-05-11T08:06:00.823238+0000","last_clean":"2024-05-11T08:06:00.823238+0000","last_became_active":"2024-05-11T08:05:59.403407+0000","last_became_peered":"2024-05-11T08:05:59.403407+0000","last_unstale":"2024-05-11T08:06:00.823238+0000","last_undegraded":"2024-05-11T08:06:00.823238+0000","last_fullsized":"2024-05-11T08:06:00.823238+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_clean_scrub_stamp":"2024-05-11T08:05:58.374552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:57:03.981207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"22'2","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.388963+0000","last_change":"2024-05-11T08:05:54.369560+0000","last_active":"2024-05-11T08:06:00.388963+0000","last_peered":"2024-05-11T08:06:00.388963+0000","last_clean":"2024-05-11T08:06:00.388963+0000","last_became_active":"2024-05-11T08:05:54.369150+0000","last_became_peered":"2024-05-11T08:05:54.369150+0000","last_unstale":"2024-05-11T08:06:00.388963+0000","last_undegraded":"2024-05-11T08:06:00.388963+0000","last_fullsized":"2024-05-11T08:06:00.388963+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_clean_scrub_stamp":"2024-05-11T08:05:53.331701+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:17:22.080661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.389718+0000","last_change":"2024-05-11T08:05:55.379813+0000","last_active":"2024-05-11T08:06:00.389718+0000","last_peered":"2024-05-11T08:06:00.389718+0000","last_clean":"2024-05-11T08:06:00.389718+0000","last_became_active":"2024-05-11T08:05:55.379575+0000","last_became_peered":"2024-05-11T08:05:55.379575+0000","last_unstale":"2024-05-11T08:06:00.389718+0000","last_undegraded":"2024-05-11T08:06:00.389718+0000","last_fullsized":"2024-05-11T08:06:00.389718+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:28:24.740747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.823228+0000","last_change":"2024-05-11T08:05:59.412477+0000","last_active":"2024-05-11T08:06:00.823228+0000","last_peered":"2024-05-11T08:06:00.823228+0000","last_clean":"2024-05-11T08:06:00.823228+0000","last_became_active":"2024-05-11T08:05:59.411969+0000","last_became_peered":"2024-05-11T08:05:59.411969+0000","last_unstale":"2024-05-11T08:06:00.823228+0000","last_undegraded":"2024-05-11T08:06:00.823228+0000","last_fullsized":"2024-05-11T08:06:00.823228+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_clean_scrub_stamp":"2024-05-11T08:05:58.374552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:38:06.426124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.645499+0000","last_change":"2024-05-11T08:05:55.376503+0000","last_active":"2024-05-11T08:06:01.645499+0000","last_peered":"2024-05-11T08:06:01.645499+0000","last_clean":"2024-05-11T08:06:01.645499+0000","last_became_active":"2024-05-11T08:05:55.376198+0000","last_became_peered":"2024-05-11T08:05:55.376198+0000","last_unstale":"2024-05-11T08:06:01.645499+0000","last_undegraded":"2024-05-11T08:06:01.645499+0000","last_fullsized":"2024-05-11T08:06:01.645499+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:20:43.832246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"22'2","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.388669+0000","last_change":"2024-05-11T08:05:54.370164+0000","last_active":"2024-05-11T08:06:00.388669+0000","last_peered":"2024-05-11T08:06:00.388669+0000","last_clean":"2024-05-11T08:06:00.388669+0000","last_became_active":"2024-05-11T08:05:54.369893+0000","last_became_peered":"2024-05-11T08:05:54.369893+0000","last_unstale":"2024-05-11T08:06:00.388669+0000","last_undegraded":"2024-05-11T08:06:00.388669+0000","last_fullsized":"2024-05-11T08:06:00.388669+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_clean_scrub_stamp":"2024-05-11T08:05:53.331701+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:53:10.612688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.823618+0000","last_change":"2024-05-11T08:05:59.409212+0000","last_active":"2024-05-11T08:06:00.823618+0000","last_peered":"2024-05-11T08:06:00.823618+0000","last_clean":"2024-05-11T08:06:00.823618+0000","last_became_active":"2024-05-11T08:05:59.408832+0000","last_became_peered":"2024-05-11T08:05:59.408832+0000","last_unstale":"2024-05-11T08:06:00.823618+0000","last_undegraded":"2024-05-11T08:06:00.823618+0000","last_fullsized":"2024-05-11T08:06:00.823618+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_clean_scrub_stamp":"2024-05-11T08:05:58.374552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:45:28.584065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.646456+0000","last_change":"2024-05-11T08:05:55.362770+0000","last_active":"2024-05-11T08:06:01.646456+0000","last_peered":"2024-05-11T08:06:01.646456+0000","last_clean":"2024-05-11T08:06:01.646456+0000","last_became_active":"2024-05-11T08:05:55.362481+0000","last_became_peered":"2024-05-11T08:05:55.362481+0000","last_unstale":"2024-05-11T08:06:01.646456+0000","last_undegraded":"2024-05-11T08:06:01.646456+0000","last_fullsized":"2024-05-11T08:06:01.646456+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:22:00.631022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.645921+0000","last_change":"2024-05-11T08:05:54.369496+0000","last_active":"2024-05-11T08:06:01.645921+0000","last_peered":"2024-05-11T08:06:01.645921+0000","last_clean":"2024-05-11T08:06:01.645921+0000","last_became_active":"2024-05-11T08:05:54.369246+0000","last_became_peered":"2024-05-11T08:05:54.369246+0000","last_unstale":"2024-05-11T08:06:01.645921+0000","last_undegraded":"2024-05-11T08:06:01.645921+0000","last_fullsized":"2024-05-11T08:06:01.645921+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_clean_scrub_stamp":"2024-05-11T08:05:53.331701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:30:23.111497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.823390+0000","last_change":"2024-05-11T08:05:59.403693+0000","last_active":"2024-05-11T08:06:00.823390+0000","last_peered":"2024-05-11T08:06:00.823390+0000","last_clean":"2024-05-11T08:06:00.823390+0000","last_became_active":"2024-05-11T08:05:59.403195+0000","last_became_peered":"2024-05-11T08:05:59.403195+0000","last_unstale":"2024-05-11T08:06:00.823390+0000","last_undegraded":"2024-05-11T08:06:00.823390+0000","last_fullsized":"2024-05-11T08:06:00.823390+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_clean_scrub_stamp":"2024-05-11T08:05:58.374552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:28:28.773313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.645822+0000","last_change":"2024-05-11T08:05:55.366582+0000","last_active":"2024-05-11T08:06:01.645822+0000","last_peered":"2024-05-11T08:06:01.645822+0000","last_clean":"2024-05-11T08:06:01.645822+0000","last_became_active":"2024-05-11T08:05:55.366309+0000","last_became_peered":"2024-05-11T08:05:55.366309+0000","last_unstale":"2024-05-11T08:06:01.645822+0000","last_undegraded":"2024-05-11T08:06:01.645822+0000","last_fullsized":"2024-05-11T08:06:01.645822+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:32:48.010809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.645408+0000","last_change":"2024-05-11T08:05:54.368032+0000","last_active":"2024-05-11T08:06:01.645408+0000","last_peered":"2024-05-11T08:06:01.645408+0000","last_clean":"2024-05-11T08:06:01.645408+0000","last_became_active":"2024-05-11T08:05:54.367739+0000","last_became_peered":"2024-05-11T08:05:54.367739+0000","last_unstale":"2024-05-11T08:06:01.645408+0000","last_undegraded":"2024-05-11T08:06:01.645408+0000","last_fullsized":"2024-05-11T08:06:01.645408+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_clean_scrub_stamp":"2024-05-11T08:05:53.331701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:01:58.573898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.646280+0000","last_change":"2024-05-11T08:05:59.560219+0000","last_active":"2024-05-11T08:06:01.646280+0000","last_peered":"2024-05-11T08:06:01.646280+0000","last_clean":"2024-05-11T08:06:01.646280+0000","last_became_active":"2024-05-11T08:05:59.559821+0000","last_became_peered":"2024-05-11T08:05:59.559821+0000","last_unstale":"2024-05-11T08:06:01.646280+0000","last_undegraded":"2024-05-11T08:06:01.646280+0000","last_fullsized":"2024-05-11T08:06:01.646280+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_clean_scrub_stamp":"2024-05-11T08:05:58.374552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:51:30.882512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.823321+0000","last_change":"2024-05-11T08:05:55.370564+0000","last_active":"2024-05-11T08:06:00.823321+0000","last_peered":"2024-05-11T08:06:00.823321+0000","last_clean":"2024-05-11T08:06:00.823321+0000","last_became_active":"2024-05-11T08:05:55.370277+0000","last_became_peered":"2024-05-11T08:05:55.370277+0000","last_unstale":"2024-05-11T08:06:00.823321+0000","last_undegraded":"2024-05-11T08:06:00.823321+0000","last_fullsized":"2024-05-11T08:06:00.823321+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:30:24.121782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.387998+0000","last_change":"2024-05-11T08:05:54.371153+0000","last_active":"2024-05-11T08:06:00.387998+0000","last_peered":"2024-05-11T08:06:00.387998+0000","last_clean":"2024-05-11T08:06:00.387998+0000","last_became_active":"2024-05-11T08:05:54.369944+0000","last_became_peered":"2024-05-11T08:05:54.369944+0000","last_unstale":"2024-05-11T08:06:00.387998+0000","last_undegraded":"2024-05-11T08:06:00.387998+0000","last_fullsized":"2024-05-11T08:06:00.387998+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_clean_scrub_stamp":"2024-05-11T08:05:53.331701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:30:23.111497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.823584+0000","last_change":"2024-05-11T08:05:59.403650+0000","last_active":"2024-05-11T08:06:00.823584+0000","last_peered":"2024-05-11T08:06:00.823584+0000","last_clean":"2024-05-11T08:06:00.823584+0000","last_became_active":"2024-05-11T08:05:59.403181+0000","last_became_peered":"2024-05-11T08:05:59.403181+0000","last_unstale":"2024-05-11T08:06:00.823584+0000","last_undegraded":"2024-05-11T08:06:00.823584+0000","last_fullsized":"2024-05-11T08:06:00.823584+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_clean_scrub_stamp":"2024-05-11T08:05:58.374552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:51:30.882512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.392942+0000","last_change":"2024-05-11T08:05:55.365766+0000","last_active":"2024-05-11T08:06:00.392942+0000","last_peered":"2024-05-11T08:06:00.392942+0000","last_clean":"2024-05-11T08:06:00.392942+0000","last_became_active":"2024-05-11T08:05:55.365513+0000","last_became_peered":"2024-05-11T08:05:55.365513+0000","last_unstale":"2024-05-11T08:06:00.392942+0000","last_undegraded":"2024-05-11T08:06:00.392942+0000","last_fullsized":"2024-05-11T08:06:00.392942+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:43:37.948904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"22'2","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.646133+0000","last_change":"2024-05-11T08:05:54.368015+0000","last_active":"2024-05-11T08:06:01.646133+0000","last_peered":"2024-05-11T08:06:01.646133+0000","last_clean":"2024-05-11T08:06:01.646133+0000","last_became_active":"2024-05-11T08:05:54.367740+0000","last_became_peered":"2024-05-11T08:05:54.367740+0000","last_unstale":"2024-05-11T08:06:01.646133+0000","last_undegraded":"2024-05-11T08:06:01.646133+0000","last_fullsized":"2024-05-11T08:06:01.646133+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_clean_scrub_stamp":"2024-05-11T08:05:53.331701+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:30:23.111497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.823061+0000","last_change":"2024-05-11T08:05:59.559986+0000","last_active":"2024-05-11T08:06:00.823061+0000","last_peered":"2024-05-11T08:06:00.823061+0000","last_clean":"2024-05-11T08:06:00.823061+0000","last_became_active":"2024-05-11T08:05:59.559594+0000","last_became_peered":"2024-05-11T08:05:59.559594+0000","last_unstale":"2024-05-11T08:06:00.823061+0000","last_undegraded":"2024-05-11T08:06:00.823061+0000","last_fullsized":"2024-05-11T08:06:00.823061+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_clean_scrub_stamp":"2024-05-11T08:05:58.374552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:14:13.335454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.646670+0000","last_change":"2024-05-11T08:05:55.374876+0000","last_active":"2024-05-11T08:06:01.646670+0000","last_peered":"2024-05-11T08:06:01.646670+0000","last_clean":"2024-05-11T08:06:01.646670+0000","last_became_active":"2024-05-11T08:05:55.374617+0000","last_became_peered":"2024-05-11T08:05:55.374617+0000","last_unstale":"2024-05-11T08:06:01.646670+0000","last_undegraded":"2024-05-11T08:06:01.646670+0000","last_fullsized":"2024-05-11T08:06:01.646670+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:41:38.931447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"23'1","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.646291+0000","last_change":"2024-05-11T08:05:54.365008+0000","last_active":"2024-05-11T08:06:01.646291+0000","last_peered":"2024-05-11T08:06:01.646291+0000","last_clean":"2024-05-11T08:06:01.646291+0000","last_became_active":"2024-05-11T08:05:54.364698+0000","last_became_peered":"2024-05-11T08:05:54.364698+0000","last_unstale":"2024-05-11T08:06:01.646291+0000","last_undegraded":"2024-05-11T08:06:01.646291+0000","last_fullsized":"2024-05-11T08:06:01.646291+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_clean_scrub_stamp":"2024-05-11T08:05:53.331701+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:17:22.080661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.645075+0000","last_change":"2024-05-11T08:05:59.562139+0000","last_active":"2024-05-11T08:06:01.645075+0000","last_peered":"2024-05-11T08:06:01.645075+0000","last_clean":"2024-05-11T08:06:01.645075+0000","last_became_active":"2024-05-11T08:05:59.561081+0000","last_became_peered":"2024-05-11T08:05:59.561081+0000","last_unstale":"2024-05-11T08:06:01.645075+0000","last_undegraded":"2024-05-11T08:06:01.645075+0000","last_fullsized":"2024-05-11T08:06:01.645075+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_clean_scrub_stamp":"2024-05-11T08:05:58.374552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:41:33.826242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.645886+0000","last_change":"2024-05-11T08:05:55.371779+0000","last_active":"2024-05-11T08:06:01.645886+0000","last_peered":"2024-05-11T08:06:01.645886+0000","last_clean":"2024-05-11T08:06:01.645886+0000","last_became_active":"2024-05-11T08:05:55.368726+0000","last_became_peered":"2024-05-11T08:05:55.368726+0000","last_unstale":"2024-05-11T08:06:01.645886+0000","last_undegraded":"2024-05-11T08:06:01.645886+0000","last_fullsized":"2024-05-11T08:06:01.645886+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:32:48.010809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.388659+0000","last_change":"2024-05-11T08:05:54.370339+0000","last_active":"2024-05-11T08:06:00.388659+0000","last_peered":"2024-05-11T08:06:00.388659+0000","last_clean":"2024-05-11T08:06:00.388659+0000","last_became_active":"2024-05-11T08:05:54.370026+0000","last_became_peered":"2024-05-11T08:05:54.370026+0000","last_unstale":"2024-05-11T08:06:00.388659+0000","last_undegraded":"2024-05-11T08:06:00.388659+0000","last_fullsized":"2024-05-11T08:06:00.388659+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_clean_scrub_stamp":"2024-05-11T08:05:53.331701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:32:43.591947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.392567+0000","last_change":"2024-05-11T08:05:59.558387+0000","last_active":"2024-05-11T08:06:00.392567+0000","last_peered":"2024-05-11T08:06:00.392567+0000","last_clean":"2024-05-11T08:06:00.392567+0000","last_became_active":"2024-05-11T08:05:59.557980+0000","last_became_peered":"2024-05-11T08:05:59.557980+0000","last_unstale":"2024-05-11T08:06:00.392567+0000","last_undegraded":"2024-05-11T08:06:00.392567+0000","last_fullsized":"2024-05-11T08:06:00.392567+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_clean_scrub_stamp":"2024-05-11T08:05:58.374552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:12:42.864479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.389813+0000","last_change":"2024-05-11T08:05:55.373737+0000","last_active":"2024-05-11T08:06:00.389813+0000","last_peered":"2024-05-11T08:06:00.389813+0000","last_clean":"2024-05-11T08:06:00.389813+0000","last_became_active":"2024-05-11T08:05:55.373459+0000","last_became_peered":"2024-05-11T08:05:55.373459+0000","last_unstale":"2024-05-11T08:06:00.389813+0000","last_undegraded":"2024-05-11T08:06:00.389813+0000","last_fullsized":"2024-05-11T08:06:00.389813+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:51:26.849946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.645985+0000","last_change":"2024-05-11T08:05:54.367679+0000","last_active":"2024-05-11T08:06:01.645985+0000","last_peered":"2024-05-11T08:06:01.645985+0000","last_clean":"2024-05-11T08:06:01.645985+0000","last_became_active":"2024-05-11T08:05:54.367393+0000","last_became_peered":"2024-05-11T08:05:54.367393+0000","last_unstale":"2024-05-11T08:06:01.645985+0000","last_undegraded":"2024-05-11T08:06:01.645985+0000","last_fullsized":"2024-05-11T08:06:01.645985+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_clean_scrub_stamp":"2024-05-11T08:05:53.331701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:50:17.617231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.645628+0000","last_change":"2024-05-11T08:05:59.413104+0000","last_active":"2024-05-11T08:06:01.645628+0000","last_peered":"2024-05-11T08:06:01.645628+0000","last_clean":"2024-05-11T08:06:01.645628+0000","last_became_active":"2024-05-11T08:05:59.412442+0000","last_became_peered":"2024-05-11T08:05:59.412442+0000","last_unstale":"2024-05-11T08:06:01.645628+0000","last_undegraded":"2024-05-11T08:06:01.645628+0000","last_fullsized":"2024-05-11T08:06:01.645628+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_clean_scrub_stamp":"2024-05-11T08:05:58.374552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:57:03.981207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.823506+0000","last_change":"2024-05-11T08:05:55.377393+0000","last_active":"2024-05-11T08:06:00.823506+0000","last_peered":"2024-05-11T08:06:00.823506+0000","last_clean":"2024-05-11T08:06:00.823506+0000","last_became_active":"2024-05-11T08:05:55.377129+0000","last_became_peered":"2024-05-11T08:05:55.377129+0000","last_unstale":"2024-05-11T08:06:00.823506+0000","last_undegraded":"2024-05-11T08:06:00.823506+0000","last_fullsized":"2024-05-11T08:06:00.823506+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:05:55.494912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.389461+0000","last_change":"2024-05-11T08:05:54.361751+0000","last_active":"2024-05-11T08:06:00.389461+0000","last_peered":"2024-05-11T08:06:00.389461+0000","last_clean":"2024-05-11T08:06:00.389461+0000","last_became_active":"2024-05-11T08:05:54.361484+0000","last_became_peered":"2024-05-11T08:05:54.361484+0000","last_unstale":"2024-05-11T08:06:00.389461+0000","last_undegraded":"2024-05-11T08:06:00.389461+0000","last_fullsized":"2024-05-11T08:06:00.389461+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_clean_scrub_stamp":"2024-05-11T08:05:53.331701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:53:10.612688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.645837+0000","last_change":"2024-05-11T08:05:59.409532+0000","last_active":"2024-05-11T08:06:01.645837+0000","last_peered":"2024-05-11T08:06:01.645837+0000","last_clean":"2024-05-11T08:06:01.645837+0000","last_became_active":"2024-05-11T08:05:59.409096+0000","last_became_peered":"2024-05-11T08:05:59.409096+0000","last_unstale":"2024-05-11T08:06:01.645837+0000","last_undegraded":"2024-05-11T08:06:01.645837+0000","last_fullsized":"2024-05-11T08:06:01.645837+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_clean_scrub_stamp":"2024-05-11T08:05:58.374552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:12:42.864479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.645313+0000","last_change":"2024-05-11T08:05:55.368600+0000","last_active":"2024-05-11T08:06:01.645313+0000","last_peered":"2024-05-11T08:06:01.645313+0000","last_clean":"2024-05-11T08:06:01.645313+0000","last_became_active":"2024-05-11T08:05:55.368322+0000","last_became_peered":"2024-05-11T08:05:55.368322+0000","last_unstale":"2024-05-11T08:06:01.645313+0000","last_undegraded":"2024-05-11T08:06:01.645313+0000","last_fullsized":"2024-05-11T08:06:01.645313+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:38:18.066779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.645325+0000","last_change":"2024-05-11T08:05:54.369698+0000","last_active":"2024-05-11T08:06:01.645325+0000","last_peered":"2024-05-11T08:06:01.645325+0000","last_clean":"2024-05-11T08:06:01.645325+0000","last_became_active":"2024-05-11T08:05:54.369387+0000","last_became_peered":"2024-05-11T08:05:54.369387+0000","last_unstale":"2024-05-11T08:06:01.645325+0000","last_undegraded":"2024-05-11T08:06:01.645325+0000","last_fullsized":"2024-05-11T08:06:01.645325+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_clean_scrub_stamp":"2024-05-11T08:05:53.331701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:14:18.956461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.646283+0000","last_change":"2024-05-11T08:05:59.413264+0000","last_active":"2024-05-11T08:06:01.646283+0000","last_peered":"2024-05-11T08:06:01.646283+0000","last_clean":"2024-05-11T08:06:01.646283+0000","last_became_active":"2024-05-11T08:05:59.412704+0000","last_became_peered":"2024-05-11T08:05:59.412704+0000","last_unstale":"2024-05-11T08:06:01.646283+0000","last_undegraded":"2024-05-11T08:06:01.646283+0000","last_fullsized":"2024-05-11T08:06:01.646283+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_clean_scrub_stamp":"2024-05-11T08:05:58.374552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:45:28.584065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.392303+0000","last_change":"2024-05-11T08:05:55.373518+0000","last_active":"2024-05-11T08:06:00.392303+0000","last_peered":"2024-05-11T08:06:00.392303+0000","last_clean":"2024-05-11T08:06:00.392303+0000","last_became_active":"2024-05-11T08:05:55.373067+0000","last_became_peered":"2024-05-11T08:05:55.373067+0000","last_unstale":"2024-05-11T08:06:00.392303+0000","last_undegraded":"2024-05-11T08:06:00.392303+0000","last_fullsized":"2024-05-11T08:06:00.392303+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:35:55.478239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.388011+0000","last_change":"2024-05-11T08:05:54.369563+0000","last_active":"2024-05-11T08:06:00.388011+0000","last_peered":"2024-05-11T08:06:00.388011+0000","last_clean":"2024-05-11T08:06:00.388011+0000","last_became_active":"2024-05-11T08:05:54.369257+0000","last_became_peered":"2024-05-11T08:05:54.369257+0000","last_unstale":"2024-05-11T08:06:00.388011+0000","last_undegraded":"2024-05-11T08:06:00.388011+0000","last_fullsized":"2024-05-11T08:06:00.388011+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_clean_scrub_stamp":"2024-05-11T08:05:53.331701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:05:54.484627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.392895+0000","last_change":"2024-05-11T08:05:59.560554+0000","last_active":"2024-05-11T08:06:00.392895+0000","last_peered":"2024-05-11T08:06:00.392895+0000","last_clean":"2024-05-11T08:06:00.392895+0000","last_became_active":"2024-05-11T08:05:59.560163+0000","last_became_peered":"2024-05-11T08:05:59.560163+0000","last_unstale":"2024-05-11T08:06:00.392895+0000","last_undegraded":"2024-05-11T08:06:00.392895+0000","last_fullsized":"2024-05-11T08:06:00.392895+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_clean_scrub_stamp":"2024-05-11T08:05:58.374552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:19:03.458410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.645474+0000","last_change":"2024-05-11T08:05:55.371707+0000","last_active":"2024-05-11T08:06:01.645474+0000","last_peered":"2024-05-11T08:06:01.645474+0000","last_clean":"2024-05-11T08:06:01.645474+0000","last_became_active":"2024-05-11T08:05:55.371443+0000","last_became_peered":"2024-05-11T08:05:55.371443+0000","last_unstale":"2024-05-11T08:06:01.645474+0000","last_undegraded":"2024-05-11T08:06:01.645474+0000","last_fullsized":"2024-05-11T08:06:01.645474+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:16:46.423362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.389333+0000","last_change":"2024-05-11T08:05:54.361406+0000","last_active":"2024-05-11T08:06:00.389333+0000","last_peered":"2024-05-11T08:06:00.389333+0000","last_clean":"2024-05-11T08:06:00.389333+0000","last_became_active":"2024-05-11T08:05:54.361065+0000","last_became_peered":"2024-05-11T08:05:54.361065+0000","last_unstale":"2024-05-11T08:06:00.389333+0000","last_undegraded":"2024-05-11T08:06:00.389333+0000","last_fullsized":"2024-05-11T08:06:00.389333+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_clean_scrub_stamp":"2024-05-11T08:05:53.331701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:33:25.907598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":118,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.388646+0000","last_change":"2024-05-11T08:05:45.772368+0000","last_active":"2024-05-11T08:06:00.388646+0000","last_peered":"2024-05-11T08:06:00.388646+0000","last_clean":"2024-05-11T08:06:00.388646+0000","last_became_active":"2024-05-11T08:05:45.771373+0000","last_became_peered":"2024-05-11T08:05:45.771373+0000","last_unstale":"2024-05-11T08:06:00.388646+0000","last_undegraded":"2024-05-11T08:06:00.388646+0000","last_fullsized":"2024-05-11T08:06:00.388646+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:44.616617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:44.616617+0000","last_clean_scrub_stamp":"2024-05-11T08:05:44.616617+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:02:08.078748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.645881+0000","last_change":"2024-05-11T08:05:59.558739+0000","last_active":"2024-05-11T08:06:01.645881+0000","last_peered":"2024-05-11T08:06:01.645881+0000","last_clean":"2024-05-11T08:06:01.645881+0000","last_became_active":"2024-05-11T08:05:59.558307+0000","last_became_peered":"2024-05-11T08:05:59.558307+0000","last_unstale":"2024-05-11T08:06:01.645881+0000","last_undegraded":"2024-05-11T08:06:01.645881+0000","last_fullsized":"2024-05-11T08:06:01.645881+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_clean_scrub_stamp":"2024-05-11T08:05:58.374552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:28:28.773313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.388334+0000","last_change":"2024-05-11T08:05:55.379286+0000","last_active":"2024-05-11T08:06:00.388334+0000","last_peered":"2024-05-11T08:06:00.388334+0000","last_clean":"2024-05-11T08:06:00.388334+0000","last_became_active":"2024-05-11T08:05:55.378804+0000","last_became_peered":"2024-05-11T08:05:55.378804+0000","last_unstale":"2024-05-11T08:06:00.388334+0000","last_undegraded":"2024-05-11T08:06:00.388334+0000","last_fullsized":"2024-05-11T08:06:00.388334+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:56:59.948641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.646359+0000","last_change":"2024-05-11T08:05:54.364902+0000","last_active":"2024-05-11T08:06:01.646359+0000","last_peered":"2024-05-11T08:06:01.646359+0000","last_clean":"2024-05-11T08:06:01.646359+0000","last_became_active":"2024-05-11T08:05:54.364619+0000","last_became_peered":"2024-05-11T08:05:54.364619+0000","last_unstale":"2024-05-11T08:06:01.646359+0000","last_undegraded":"2024-05-11T08:06:01.646359+0000","last_fullsized":"2024-05-11T08:06:01.646359+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_clean_scrub_stamp":"2024-05-11T08:05:53.331701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:10:56.295408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.645635+0000","last_change":"2024-05-11T08:05:59.558409+0000","last_active":"2024-05-11T08:06:01.645635+0000","last_peered":"2024-05-11T08:06:01.645635+0000","last_clean":"2024-05-11T08:06:01.645635+0000","last_became_active":"2024-05-11T08:05:59.557939+0000","last_became_peered":"2024-05-11T08:05:59.557939+0000","last_unstale":"2024-05-11T08:06:01.645635+0000","last_undegraded":"2024-05-11T08:06:01.645635+0000","last_fullsized":"2024-05-11T08:06:01.645635+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_clean_scrub_stamp":"2024-05-11T08:05:58.374552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:57:03.981207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.389620+0000","last_change":"2024-05-11T08:05:55.376614+0000","last_active":"2024-05-11T08:06:00.389620+0000","last_peered":"2024-05-11T08:06:00.389620+0000","last_clean":"2024-05-11T08:06:00.389620+0000","last_became_active":"2024-05-11T08:05:55.376350+0000","last_became_peered":"2024-05-11T08:05:55.376350+0000","last_unstale":"2024-05-11T08:06:00.389620+0000","last_undegraded":"2024-05-11T08:06:00.389620+0000","last_fullsized":"2024-05-11T08:06:00.389620+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:12:38.831913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"22'1","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.645871+0000","last_change":"2024-05-11T08:05:54.369427+0000","last_active":"2024-05-11T08:06:01.645871+0000","last_peered":"2024-05-11T08:06:01.645871+0000","last_clean":"2024-05-11T08:06:01.645871+0000","last_became_active":"2024-05-11T08:05:54.369181+0000","last_became_peered":"2024-05-11T08:05:54.369181+0000","last_unstale":"2024-05-11T08:06:01.645871+0000","last_undegraded":"2024-05-11T08:06:01.645871+0000","last_fullsized":"2024-05-11T08:06:01.645871+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_clean_scrub_stamp":"2024-05-11T08:05:53.331701+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:05:54.484627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.392744+0000","last_change":"2024-05-11T08:05:59.411982+0000","last_active":"2024-05-11T08:06:00.392744+0000","last_peered":"2024-05-11T08:06:00.392744+0000","last_clean":"2024-05-11T08:06:00.392744+0000","last_became_active":"2024-05-11T08:05:59.411561+0000","last_became_peered":"2024-05-11T08:05:59.411561+0000","last_unstale":"2024-05-11T08:06:00.392744+0000","last_undegraded":"2024-05-11T08:06:00.392744+0000","last_fullsized":"2024-05-11T08:06:00.392744+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_clean_scrub_stamp":"2024-05-11T08:05:58.374552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:45:28.584065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.389445+0000","last_change":"2024-05-11T08:05:55.386175+0000","last_active":"2024-05-11T08:06:00.389445+0000","last_peered":"2024-05-11T08:06:00.389445+0000","last_clean":"2024-05-11T08:06:00.389445+0000","last_became_active":"2024-05-11T08:05:55.385883+0000","last_became_peered":"2024-05-11T08:05:55.385883+0000","last_unstale":"2024-05-11T08:06:00.389445+0000","last_undegraded":"2024-05-11T08:06:00.389445+0000","last_fullsized":"2024-05-11T08:06:00.389445+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:43:20.023451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"22'2","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.645669+0000","last_change":"2024-05-11T08:05:54.369810+0000","last_active":"2024-05-11T08:06:01.645669+0000","last_peered":"2024-05-11T08:06:01.645669+0000","last_clean":"2024-05-11T08:06:01.645669+0000","last_became_active":"2024-05-11T08:05:54.369512+0000","last_became_peered":"2024-05-11T08:05:54.369512+0000","last_unstale":"2024-05-11T08:06:01.645669+0000","last_undegraded":"2024-05-11T08:06:01.645669+0000","last_fullsized":"2024-05-11T08:06:01.645669+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_clean_scrub_stamp":"2024-05-11T08:05:53.331701+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:39:04.019231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.388134+0000","last_change":"2024-05-11T08:05:59.562447+0000","last_active":"2024-05-11T08:06:00.388134+0000","last_peered":"2024-05-11T08:06:00.388134+0000","last_clean":"2024-05-11T08:06:00.388134+0000","last_became_active":"2024-05-11T08:05:59.562033+0000","last_became_peered":"2024-05-11T08:05:59.562033+0000","last_unstale":"2024-05-11T08:06:00.388134+0000","last_undegraded":"2024-05-11T08:06:00.388134+0000","last_fullsized":"2024-05-11T08:06:00.388134+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_clean_scrub_stamp":"2024-05-11T08:05:58.374552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:41:33.826242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.388740+0000","last_change":"2024-05-11T08:05:55.386091+0000","last_active":"2024-05-11T08:06:00.388740+0000","last_peered":"2024-05-11T08:06:00.388740+0000","last_clean":"2024-05-11T08:06:00.388740+0000","last_became_active":"2024-05-11T08:05:55.385751+0000","last_became_peered":"2024-05-11T08:05:55.385751+0000","last_unstale":"2024-05-11T08:06:00.388740+0000","last_undegraded":"2024-05-11T08:06:00.388740+0000","last_fullsized":"2024-05-11T08:06:00.388740+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:12:38.831913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"22'1","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.823802+0000","last_change":"2024-05-11T08:05:54.372054+0000","last_active":"2024-05-11T08:06:00.823802+0000","last_peered":"2024-05-11T08:06:00.823802+0000","last_clean":"2024-05-11T08:06:00.823802+0000","last_became_active":"2024-05-11T08:05:54.371721+0000","last_became_peered":"2024-05-11T08:05:54.371721+0000","last_unstale":"2024-05-11T08:06:00.823802+0000","last_undegraded":"2024-05-11T08:06:00.823802+0000","last_fullsized":"2024-05-11T08:06:00.823802+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_clean_scrub_stamp":"2024-05-11T08:05:53.331701+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:40:29.770055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.823074+0000","last_change":"2024-05-11T08:05:59.404048+0000","last_active":"2024-05-11T08:06:00.823074+0000","last_peered":"2024-05-11T08:06:00.823074+0000","last_clean":"2024-05-11T08:06:00.823074+0000","last_became_active":"2024-05-11T08:05:59.403010+0000","last_became_peered":"2024-05-11T08:05:59.403010+0000","last_unstale":"2024-05-11T08:06:00.823074+0000","last_undegraded":"2024-05-11T08:06:00.823074+0000","last_fullsized":"2024-05-11T08:06:00.823074+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_clean_scrub_stamp":"2024-05-11T08:05:58.374552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:44:47.420416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.388312+0000","last_change":"2024-05-11T08:05:55.382224+0000","last_active":"2024-05-11T08:06:00.388312+0000","last_peered":"2024-05-11T08:06:00.388312+0000","last_clean":"2024-05-11T08:06:00.388312+0000","last_became_active":"2024-05-11T08:05:55.381614+0000","last_became_peered":"2024-05-11T08:05:55.381614+0000","last_unstale":"2024-05-11T08:06:00.388312+0000","last_undegraded":"2024-05-11T08:06:00.388312+0000","last_fullsized":"2024-05-11T08:06:00.388312+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:41:38.931447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"22'8","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.645775+0000","last_change":"2024-05-11T08:05:54.365900+0000","last_active":"2024-05-11T08:06:01.645775+0000","last_peered":"2024-05-11T08:06:01.645775+0000","last_clean":"2024-05-11T08:06:01.645775+0000","last_became_active":"2024-05-11T08:05:54.365638+0000","last_became_peered":"2024-05-11T08:05:54.365638+0000","last_unstale":"2024-05-11T08:06:01.645775+0000","last_undegraded":"2024-05-11T08:06:01.645775+0000","last_fullsized":"2024-05-11T08:06:01.645775+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_clean_scrub_stamp":"2024-05-11T08:05:53.331701+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:23:10.014766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.392726+0000","last_change":"2024-05-11T08:05:59.412069+0000","last_active":"2024-05-11T08:06:00.392726+0000","last_peered":"2024-05-11T08:06:00.392726+0000","last_clean":"2024-05-11T08:06:00.392726+0000","last_became_active":"2024-05-11T08:05:59.411674+0000","last_became_peered":"2024-05-11T08:05:59.411674+0000","last_unstale":"2024-05-11T08:06:00.392726+0000","last_undegraded":"2024-05-11T08:06:00.392726+0000","last_fullsized":"2024-05-11T08:06:00.392726+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_clean_scrub_stamp":"2024-05-11T08:05:58.374552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:51:30.882512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.645506+0000","last_change":"2024-05-11T08:05:55.364883+0000","last_active":"2024-05-11T08:06:01.645506+0000","last_peered":"2024-05-11T08:06:01.645506+0000","last_clean":"2024-05-11T08:06:01.645506+0000","last_became_active":"2024-05-11T08:05:55.364622+0000","last_became_peered":"2024-05-11T08:05:55.364622+0000","last_unstale":"2024-05-11T08:06:01.645506+0000","last_undegraded":"2024-05-11T08:06:01.645506+0000","last_fullsized":"2024-05-11T08:06:01.645506+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:35:39.553476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.645631+0000","last_change":"2024-05-11T08:05:54.368776+0000","last_active":"2024-05-11T08:06:01.645631+0000","last_peered":"2024-05-11T08:06:01.645631+0000","last_clean":"2024-05-11T08:06:01.645631+0000","last_became_active":"2024-05-11T08:05:54.368514+0000","last_became_peered":"2024-05-11T08:05:54.368514+0000","last_unstale":"2024-05-11T08:06:01.645631+0000","last_undegraded":"2024-05-11T08:06:01.645631+0000","last_fullsized":"2024-05-11T08:06:01.645631+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_clean_scrub_stamp":"2024-05-11T08:05:53.331701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:01:58.573898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.646534+0000","last_change":"2024-05-11T08:05:59.561579+0000","last_active":"2024-05-11T08:06:01.646534+0000","last_peered":"2024-05-11T08:06:01.646534+0000","last_clean":"2024-05-11T08:06:01.646534+0000","last_became_active":"2024-05-11T08:05:59.561027+0000","last_became_peered":"2024-05-11T08:05:59.561027+0000","last_unstale":"2024-05-11T08:06:01.646534+0000","last_undegraded":"2024-05-11T08:06:01.646534+0000","last_fullsized":"2024-05-11T08:06:01.646534+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_clean_scrub_stamp":"2024-05-11T08:05:58.374552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:06:20.739397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.388603+0000","last_change":"2024-05-11T08:05:55.387188+0000","last_active":"2024-05-11T08:06:00.388603+0000","last_peered":"2024-05-11T08:06:00.388603+0000","last_clean":"2024-05-11T08:06:00.388603+0000","last_became_active":"2024-05-11T08:05:55.386935+0000","last_became_peered":"2024-05-11T08:05:55.386935+0000","last_unstale":"2024-05-11T08:06:00.388603+0000","last_undegraded":"2024-05-11T08:06:00.388603+0000","last_fullsized":"2024-05-11T08:06:00.388603+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:14:09.302889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.389390+0000","last_change":"2024-05-11T08:05:54.370172+0000","last_active":"2024-05-11T08:06:00.389390+0000","last_peered":"2024-05-11T08:06:00.389390+0000","last_clean":"2024-05-11T08:06:00.389390+0000","last_became_active":"2024-05-11T08:05:54.369635+0000","last_became_peered":"2024-05-11T08:05:54.369635+0000","last_unstale":"2024-05-11T08:06:00.389390+0000","last_undegraded":"2024-05-11T08:06:00.389390+0000","last_fullsized":"2024-05-11T08:06:00.389390+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_clean_scrub_stamp":"2024-05-11T08:05:53.331701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:17:22.080661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.647316+0000","last_change":"2024-05-11T08:05:59.559006+0000","last_active":"2024-05-11T08:06:01.647316+0000","last_peered":"2024-05-11T08:06:01.647316+0000","last_clean":"2024-05-11T08:06:01.647316+0000","last_became_active":"2024-05-11T08:05:59.558255+0000","last_became_peered":"2024-05-11T08:05:59.558255+0000","last_unstale":"2024-05-11T08:06:01.647316+0000","last_undegraded":"2024-05-11T08:06:01.647316+0000","last_fullsized":"2024-05-11T08:06:01.647316+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_clean_scrub_stamp":"2024-05-11T08:05:58.374552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:32:53.866284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.392560+0000","last_change":"2024-05-11T08:05:55.375477+0000","last_active":"2024-05-11T08:06:00.392560+0000","last_peered":"2024-05-11T08:06:00.392560+0000","last_clean":"2024-05-11T08:06:00.392560+0000","last_became_active":"2024-05-11T08:05:55.375215+0000","last_became_peered":"2024-05-11T08:05:55.375215+0000","last_unstale":"2024-05-11T08:06:00.392560+0000","last_undegraded":"2024-05-11T08:06:00.392560+0000","last_fullsized":"2024-05-11T08:06:00.392560+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:38:18.066779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"22'2","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.392809+0000","last_change":"2024-05-11T08:05:54.368097+0000","last_active":"2024-05-11T08:06:00.392809+0000","last_peered":"2024-05-11T08:06:00.392809+0000","last_clean":"2024-05-11T08:06:00.392809+0000","last_became_active":"2024-05-11T08:05:54.367812+0000","last_became_peered":"2024-05-11T08:05:54.367812+0000","last_unstale":"2024-05-11T08:06:00.392809+0000","last_undegraded":"2024-05-11T08:06:00.392809+0000","last_fullsized":"2024-05-11T08:06:00.392809+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_clean_scrub_stamp":"2024-05-11T08:05:53.331701+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:00:20.233006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.389793+0000","last_change":"2024-05-11T08:05:59.558753+0000","last_active":"2024-05-11T08:06:00.389793+0000","last_peered":"2024-05-11T08:06:00.389793+0000","last_clean":"2024-05-11T08:06:00.389793+0000","last_became_active":"2024-05-11T08:05:59.558140+0000","last_became_peered":"2024-05-11T08:05:59.558140+0000","last_unstale":"2024-05-11T08:06:00.389793+0000","last_undegraded":"2024-05-11T08:06:00.389793+0000","last_fullsized":"2024-05-11T08:06:00.389793+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_clean_scrub_stamp":"2024-05-11T08:05:58.374552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:54:52.804997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.388229+0000","last_change":"2024-05-11T08:05:55.379691+0000","last_active":"2024-05-11T08:06:00.388229+0000","last_peered":"2024-05-11T08:06:00.388229+0000","last_clean":"2024-05-11T08:06:00.388229+0000","last_became_active":"2024-05-11T08:05:55.378519+0000","last_became_peered":"2024-05-11T08:05:55.378519+0000","last_unstale":"2024-05-11T08:06:00.388229+0000","last_undegraded":"2024-05-11T08:06:00.388229+0000","last_fullsized":"2024-05-11T08:06:00.388229+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:38:02.393559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"27'9","reported_seq":31,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:04.814504+0000","last_change":"2024-05-11T08:05:54.371057+0000","last_active":"2024-05-11T08:06:04.814504+0000","last_peered":"2024-05-11T08:06:04.814504+0000","last_clean":"2024-05-11T08:06:04.814504+0000","last_became_active":"2024-05-11T08:05:54.370746+0000","last_became_peered":"2024-05-11T08:05:54.370746+0000","last_unstale":"2024-05-11T08:06:04.814504+0000","last_undegraded":"2024-05-11T08:06:04.814504+0000","last_fullsized":"2024-05-11T08:06:04.814504+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_clean_scrub_stamp":"2024-05-11T08:05:53.331701+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:17:22.080661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.645768+0000","last_change":"2024-05-11T08:05:59.414118+0000","last_active":"2024-05-11T08:06:01.645768+0000","last_peered":"2024-05-11T08:06:01.645768+0000","last_clean":"2024-05-11T08:06:01.645768+0000","last_became_active":"2024-05-11T08:05:59.413183+0000","last_became_peered":"2024-05-11T08:05:59.413183+0000","last_unstale":"2024-05-11T08:06:01.645768+0000","last_undegraded":"2024-05-11T08:06:01.645768+0000","last_fullsized":"2024-05-11T08:06:01.645768+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_clean_scrub_stamp":"2024-05-11T08:05:58.374552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:28:28.773313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.823505+0000","last_change":"2024-05-11T08:05:55.373641+0000","last_active":"2024-05-11T08:06:00.823505+0000","last_peered":"2024-05-11T08:06:00.823505+0000","last_clean":"2024-05-11T08:06:00.823505+0000","last_became_active":"2024-05-11T08:05:55.373369+0000","last_became_peered":"2024-05-11T08:05:55.373369+0000","last_unstale":"2024-05-11T08:06:00.823505+0000","last_undegraded":"2024-05-11T08:06:00.823505+0000","last_fullsized":"2024-05-11T08:06:00.823505+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:32:44.602233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"22'3","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.645781+0000","last_change":"2024-05-11T08:05:54.366529+0000","last_active":"2024-05-11T08:06:01.645781+0000","last_peered":"2024-05-11T08:06:01.645781+0000","last_clean":"2024-05-11T08:06:01.645781+0000","last_became_active":"2024-05-11T08:05:54.366215+0000","last_became_peered":"2024-05-11T08:05:54.366215+0000","last_unstale":"2024-05-11T08:06:01.645781+0000","last_undegraded":"2024-05-11T08:06:01.645781+0000","last_fullsized":"2024-05-11T08:06:01.645781+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_clean_scrub_stamp":"2024-05-11T08:05:53.331701+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:53:10.612688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.823463+0000","last_change":"2024-05-11T08:05:59.407003+0000","last_active":"2024-05-11T08:06:00.823463+0000","last_peered":"2024-05-11T08:06:00.823463+0000","last_clean":"2024-05-11T08:06:00.823463+0000","last_became_active":"2024-05-11T08:05:59.406437+0000","last_became_peered":"2024-05-11T08:05:59.406437+0000","last_unstale":"2024-05-11T08:06:00.823463+0000","last_undegraded":"2024-05-11T08:06:00.823463+0000","last_fullsized":"2024-05-11T08:06:00.823463+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_clean_scrub_stamp":"2024-05-11T08:05:58.374552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:12:42.864479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.392952+0000","last_change":"2024-05-11T08:05:55.370107+0000","last_active":"2024-05-11T08:06:00.392952+0000","last_peered":"2024-05-11T08:06:00.392952+0000","last_clean":"2024-05-11T08:06:00.392952+0000","last_became_active":"2024-05-11T08:05:55.369823+0000","last_became_peered":"2024-05-11T08:05:55.369823+0000","last_unstale":"2024-05-11T08:06:00.392952+0000","last_undegraded":"2024-05-11T08:06:00.392952+0000","last_fullsized":"2024-05-11T08:06:00.392952+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:22:00.631022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.389731+0000","last_change":"2024-05-11T08:05:54.363934+0000","last_active":"2024-05-11T08:06:00.389731+0000","last_peered":"2024-05-11T08:06:00.389731+0000","last_clean":"2024-05-11T08:06:00.389731+0000","last_became_active":"2024-05-11T08:05:54.363428+0000","last_became_peered":"2024-05-11T08:05:54.363428+0000","last_unstale":"2024-05-11T08:06:00.389731+0000","last_undegraded":"2024-05-11T08:06:00.389731+0000","last_fullsized":"2024-05-11T08:06:00.389731+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_clean_scrub_stamp":"2024-05-11T08:05:53.331701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:30:23.111497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.645683+0000","last_change":"2024-05-11T08:05:59.403691+0000","last_active":"2024-05-11T08:06:01.645683+0000","last_peered":"2024-05-11T08:06:01.645683+0000","last_clean":"2024-05-11T08:06:01.645683+0000","last_became_active":"2024-05-11T08:05:59.403311+0000","last_became_peered":"2024-05-11T08:05:59.403311+0000","last_unstale":"2024-05-11T08:06:01.645683+0000","last_undegraded":"2024-05-11T08:06:01.645683+0000","last_fullsized":"2024-05-11T08:06:01.645683+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_clean_scrub_stamp":"2024-05-11T08:05:58.374552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:54:52.804997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.389689+0000","last_change":"2024-05-11T08:05:55.374886+0000","last_active":"2024-05-11T08:06:00.389689+0000","last_peered":"2024-05-11T08:06:00.389689+0000","last_clean":"2024-05-11T08:06:00.389689+0000","last_became_active":"2024-05-11T08:05:55.374622+0000","last_became_peered":"2024-05-11T08:05:55.374622+0000","last_unstale":"2024-05-11T08:06:00.389689+0000","last_undegraded":"2024-05-11T08:06:00.389689+0000","last_fullsized":"2024-05-11T08:06:00.389689+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:43:37.948904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.646049+0000","last_change":"2024-05-11T08:05:54.364291+0000","last_active":"2024-05-11T08:06:01.646049+0000","last_peered":"2024-05-11T08:06:01.646049+0000","last_clean":"2024-05-11T08:06:01.646049+0000","last_became_active":"2024-05-11T08:05:54.363676+0000","last_became_peered":"2024-05-11T08:05:54.363676+0000","last_unstale":"2024-05-11T08:06:01.646049+0000","last_undegraded":"2024-05-11T08:06:01.646049+0000","last_fullsized":"2024-05-11T08:06:01.646049+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_clean_scrub_stamp":"2024-05-11T08:05:53.331701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:53:10.612688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.647107+0000","last_change":"2024-05-11T08:05:59.558985+0000","last_active":"2024-05-11T08:06:01.647107+0000","last_peered":"2024-05-11T08:06:01.647107+0000","last_clean":"2024-05-11T08:06:01.647107+0000","last_became_active":"2024-05-11T08:05:59.558561+0000","last_became_peered":"2024-05-11T08:05:59.558561+0000","last_unstale":"2024-05-11T08:06:01.647107+0000","last_undegraded":"2024-05-11T08:06:01.647107+0000","last_fullsized":"2024-05-11T08:06:01.647107+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_clean_scrub_stamp":"2024-05-11T08:05:58.374552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:54:53.957903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.392778+0000","last_change":"2024-05-11T08:05:59.561482+0000","last_active":"2024-05-11T08:06:00.392778+0000","last_peered":"2024-05-11T08:06:00.392778+0000","last_clean":"2024-05-11T08:06:00.392778+0000","last_became_active":"2024-05-11T08:05:59.560743+0000","last_became_peered":"2024-05-11T08:05:59.560743+0000","last_unstale":"2024-05-11T08:06:00.392778+0000","last_undegraded":"2024-05-11T08:06:00.392778+0000","last_fullsized":"2024-05-11T08:06:00.392778+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_clean_scrub_stamp":"2024-05-11T08:05:58.374552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:43:24.056017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.645261+0000","last_change":"2024-05-11T08:05:55.367915+0000","last_active":"2024-05-11T08:06:01.645261+0000","last_peered":"2024-05-11T08:06:01.645261+0000","last_clean":"2024-05-11T08:06:01.645261+0000","last_became_active":"2024-05-11T08:05:55.367652+0000","last_became_peered":"2024-05-11T08:05:55.367652+0000","last_unstale":"2024-05-11T08:06:01.645261+0000","last_undegraded":"2024-05-11T08:06:01.645261+0000","last_fullsized":"2024-05-11T08:06:01.645261+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:35:03.601599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.389207+0000","last_change":"2024-05-11T08:05:54.368060+0000","last_active":"2024-05-11T08:06:00.389207+0000","last_peered":"2024-05-11T08:06:00.389207+0000","last_clean":"2024-05-11T08:06:00.389207+0000","last_became_active":"2024-05-11T08:05:54.367770+0000","last_became_peered":"2024-05-11T08:05:54.367770+0000","last_unstale":"2024-05-11T08:06:00.389207+0000","last_undegraded":"2024-05-11T08:06:00.389207+0000","last_fullsized":"2024-05-11T08:06:00.389207+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_clean_scrub_stamp":"2024-05-11T08:05:53.331701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:01:58.573898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.646058+0000","last_change":"2024-05-11T08:05:59.412426+0000","last_active":"2024-05-11T08:06:01.646058+0000","last_peered":"2024-05-11T08:06:01.646058+0000","last_clean":"2024-05-11T08:06:01.646058+0000","last_became_active":"2024-05-11T08:05:59.412011+0000","last_became_peered":"2024-05-11T08:05:59.412011+0000","last_unstale":"2024-05-11T08:06:01.646058+0000","last_undegraded":"2024-05-11T08:06:01.646058+0000","last_fullsized":"2024-05-11T08:06:01.646058+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_clean_scrub_stamp":"2024-05-11T08:05:58.374552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:12:42.864479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.645734+0000","last_change":"2024-05-11T08:05:55.363149+0000","last_active":"2024-05-11T08:06:01.645734+0000","last_peered":"2024-05-11T08:06:01.645734+0000","last_clean":"2024-05-11T08:06:01.645734+0000","last_became_active":"2024-05-11T08:05:55.362892+0000","last_became_peered":"2024-05-11T08:05:55.362892+0000","last_unstale":"2024-05-11T08:06:01.645734+0000","last_undegraded":"2024-05-11T08:06:01.645734+0000","last_fullsized":"2024-05-11T08:06:01.645734+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:38:18.066779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.647209+0000","last_change":"2024-05-11T08:05:54.368847+0000","last_active":"2024-05-11T08:06:01.647209+0000","last_peered":"2024-05-11T08:06:01.647209+0000","last_clean":"2024-05-11T08:06:01.647209+0000","last_became_active":"2024-05-11T08:05:54.368529+0000","last_became_peered":"2024-05-11T08:05:54.368529+0000","last_unstale":"2024-05-11T08:06:01.647209+0000","last_undegraded":"2024-05-11T08:06:01.647209+0000","last_fullsized":"2024-05-11T08:06:01.647209+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_clean_scrub_stamp":"2024-05-11T08:05:53.331701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:22:53.142453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"22'1","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.645710+0000","last_change":"2024-05-11T08:05:54.369435+0000","last_active":"2024-05-11T08:06:01.645710+0000","last_peered":"2024-05-11T08:06:01.645710+0000","last_clean":"2024-05-11T08:06:01.645710+0000","last_became_active":"2024-05-11T08:05:54.369163+0000","last_became_peered":"2024-05-11T08:05:54.369163+0000","last_unstale":"2024-05-11T08:06:01.645710+0000","last_undegraded":"2024-05-11T08:06:01.645710+0000","last_fullsized":"2024-05-11T08:06:01.645710+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_clean_scrub_stamp":"2024-05-11T08:05:53.331701+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:33:25.907598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.392757+0000","last_change":"2024-05-11T08:05:55.378586+0000","last_active":"2024-05-11T08:06:00.392757+0000","last_peered":"2024-05-11T08:06:00.392757+0000","last_clean":"2024-05-11T08:06:00.392757+0000","last_became_active":"2024-05-11T08:05:55.378323+0000","last_became_peered":"2024-05-11T08:05:55.378323+0000","last_unstale":"2024-05-11T08:06:00.392757+0000","last_undegraded":"2024-05-11T08:06:00.392757+0000","last_fullsized":"2024-05-11T08:06:00.392757+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:35:03.601599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.646291+0000","last_change":"2024-05-11T08:05:59.402315+0000","last_active":"2024-05-11T08:06:01.646291+0000","last_peered":"2024-05-11T08:06:01.646291+0000","last_clean":"2024-05-11T08:06:01.646291+0000","last_became_active":"2024-05-11T08:05:59.401886+0000","last_became_peered":"2024-05-11T08:05:59.401886+0000","last_unstale":"2024-05-11T08:06:01.646291+0000","last_undegraded":"2024-05-11T08:06:01.646291+0000","last_fullsized":"2024-05-11T08:06:01.646291+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_clean_scrub_stamp":"2024-05-11T08:05:58.374552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:33:42.953506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"22'1","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.646467+0000","last_change":"2024-05-11T08:05:54.370886+0000","last_active":"2024-05-11T08:06:01.646467+0000","last_peered":"2024-05-11T08:06:01.646467+0000","last_clean":"2024-05-11T08:06:01.646467+0000","last_became_active":"2024-05-11T08:05:54.370611+0000","last_became_peered":"2024-05-11T08:05:54.370611+0000","last_unstale":"2024-05-11T08:06:01.646467+0000","last_undegraded":"2024-05-11T08:06:01.646467+0000","last_fullsized":"2024-05-11T08:06:01.646467+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_clean_scrub_stamp":"2024-05-11T08:05:53.331701+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:36:34.523259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.823139+0000","last_change":"2024-05-11T08:05:55.373391+0000","last_active":"2024-05-11T08:06:00.823139+0000","last_peered":"2024-05-11T08:06:00.823139+0000","last_clean":"2024-05-11T08:06:00.823139+0000","last_became_active":"2024-05-11T08:05:55.373096+0000","last_became_peered":"2024-05-11T08:05:55.373096+0000","last_unstale":"2024-05-11T08:06:00.823139+0000","last_undegraded":"2024-05-11T08:06:00.823139+0000","last_fullsized":"2024-05-11T08:06:00.823139+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:53:11.622973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.647038+0000","last_change":"2024-05-11T08:05:59.403522+0000","last_active":"2024-05-11T08:06:01.647038+0000","last_peered":"2024-05-11T08:06:01.647038+0000","last_clean":"2024-05-11T08:06:01.647038+0000","last_became_active":"2024-05-11T08:05:59.403132+0000","last_became_peered":"2024-05-11T08:05:59.403132+0000","last_unstale":"2024-05-11T08:06:01.647038+0000","last_undegraded":"2024-05-11T08:06:01.647038+0000","last_fullsized":"2024-05-11T08:06:01.647038+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_clean_scrub_stamp":"2024-05-11T08:05:58.374552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:06:03.606038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.645399+0000","last_change":"2024-05-11T08:05:55.364367+0000","last_active":"2024-05-11T08:06:01.645399+0000","last_peered":"2024-05-11T08:06:01.645399+0000","last_clean":"2024-05-11T08:06:01.645399+0000","last_became_active":"2024-05-11T08:05:55.364089+0000","last_became_peered":"2024-05-11T08:05:55.364089+0000","last_unstale":"2024-05-11T08:06:01.645399+0000","last_undegraded":"2024-05-11T08:06:01.645399+0000","last_fullsized":"2024-05-11T08:06:01.645399+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:38:18.066779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.645226+0000","last_change":"2024-05-11T08:05:55.369401+0000","last_active":"2024-05-11T08:06:01.645226+0000","last_peered":"2024-05-11T08:06:01.645226+0000","last_clean":"2024-05-11T08:06:01.645226+0000","last_became_active":"2024-05-11T08:05:55.369154+0000","last_became_peered":"2024-05-11T08:05:55.369154+0000","last_unstale":"2024-05-11T08:06:01.645226+0000","last_undegraded":"2024-05-11T08:06:01.645226+0000","last_fullsized":"2024-05-11T08:06:01.645226+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:46:51.750892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.389323+0000","last_change":"2024-05-11T08:05:55.375787+0000","last_active":"2024-05-11T08:06:00.389323+0000","last_peered":"2024-05-11T08:06:00.389323+0000","last_clean":"2024-05-11T08:06:00.389323+0000","last_became_active":"2024-05-11T08:05:55.374199+0000","last_became_peered":"2024-05-11T08:05:55.374199+0000","last_unstale":"2024-05-11T08:06:00.389323+0000","last_undegraded":"2024-05-11T08:06:00.389323+0000","last_fullsized":"2024-05-11T08:06:00.389323+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:38:02.393559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.392641+0000","last_change":"2024-05-11T08:05:55.369439+0000","last_active":"2024-05-11T08:06:00.392641+0000","last_peered":"2024-05-11T08:06:00.392641+0000","last_clean":"2024-05-11T08:06:00.392641+0000","last_became_active":"2024-05-11T08:05:55.369182+0000","last_became_peered":"2024-05-11T08:05:55.369182+0000","last_unstale":"2024-05-11T08:06:00.392641+0000","last_undegraded":"2024-05-11T08:06:00.392641+0000","last_fullsized":"2024-05-11T08:06:00.392641+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:35:39.553476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.647343+0000","last_change":"2024-05-11T08:05:55.373779+0000","last_active":"2024-05-11T08:06:01.647343+0000","last_peered":"2024-05-11T08:06:01.647343+0000","last_clean":"2024-05-11T08:06:01.647343+0000","last_became_active":"2024-05-11T08:05:55.373523+0000","last_became_peered":"2024-05-11T08:05:55.373523+0000","last_unstale":"2024-05-11T08:06:01.647343+0000","last_undegraded":"2024-05-11T08:06:01.647343+0000","last_fullsized":"2024-05-11T08:06:01.647343+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:35:39.553476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.645671+0000","last_change":"2024-05-11T08:05:55.367765+0000","last_active":"2024-05-11T08:06:01.645671+0000","last_peered":"2024-05-11T08:06:01.645671+0000","last_clean":"2024-05-11T08:06:01.645671+0000","last_became_active":"2024-05-11T08:05:55.367521+0000","last_became_peered":"2024-05-11T08:05:55.367521+0000","last_unstale":"2024-05-11T08:06:01.645671+0000","last_undegraded":"2024-05-11T08:06:01.645671+0000","last_fullsized":"2024-05-11T08:06:01.645671+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:35:39.553476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.646145+0000","last_change":"2024-05-11T08:05:55.373963+0000","last_active":"2024-05-11T08:06:01.646145+0000","last_peered":"2024-05-11T08:06:01.646145+0000","last_clean":"2024-05-11T08:06:01.646145+0000","last_became_active":"2024-05-11T08:05:55.373700+0000","last_became_peered":"2024-05-11T08:05:55.373700+0000","last_unstale":"2024-05-11T08:06:01.646145+0000","last_undegraded":"2024-05-11T08:06:01.646145+0000","last_fullsized":"2024-05-11T08:06:01.646145+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:35:03.601599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.646584+0000","last_change":"2024-05-11T08:05:55.374020+0000","last_active":"2024-05-11T08:06:01.646584+0000","last_peered":"2024-05-11T08:06:01.646584+0000","last_clean":"2024-05-11T08:06:01.646584+0000","last_became_active":"2024-05-11T08:05:55.373697+0000","last_became_peered":"2024-05-11T08:05:55.373697+0000","last_unstale":"2024-05-11T08:06:01.646584+0000","last_undegraded":"2024-05-11T08:06:01.646584+0000","last_fullsized":"2024-05-11T08:06:01.646584+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:02:23.158539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.389077+0000","last_change":"2024-05-11T08:05:55.371797+0000","last_active":"2024-05-11T08:06:00.389077+0000","last_peered":"2024-05-11T08:06:00.389077+0000","last_clean":"2024-05-11T08:06:00.389077+0000","last_became_active":"2024-05-11T08:05:55.371535+0000","last_became_peered":"2024-05-11T08:05:55.371535+0000","last_unstale":"2024-05-11T08:06:00.389077+0000","last_undegraded":"2024-05-11T08:06:00.389077+0000","last_fullsized":"2024-05-11T08:06:00.389077+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:16:46.423362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.647177+0000","last_change":"2024-05-11T08:05:55.373536+0000","last_active":"2024-05-11T08:06:01.647177+0000","last_peered":"2024-05-11T08:06:01.647177+0000","last_clean":"2024-05-11T08:06:01.647177+0000","last_became_active":"2024-05-11T08:05:55.373273+0000","last_became_peered":"2024-05-11T08:05:55.373273+0000","last_unstale":"2024-05-11T08:06:01.647177+0000","last_undegraded":"2024-05-11T08:06:01.647177+0000","last_fullsized":"2024-05-11T08:06:01.647177+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:35:03.601599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.389188+0000","last_change":"2024-05-11T08:05:55.382077+0000","last_active":"2024-05-11T08:06:00.389188+0000","last_peered":"2024-05-11T08:06:00.389188+0000","last_clean":"2024-05-11T08:06:00.389188+0000","last_became_active":"2024-05-11T08:05:55.381767+0000","last_became_peered":"2024-05-11T08:05:55.381767+0000","last_unstale":"2024-05-11T08:06:00.389188+0000","last_undegraded":"2024-05-11T08:06:00.389188+0000","last_fullsized":"2024-05-11T08:06:00.389188+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:56:59.948641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.392492+0000","last_change":"2024-05-11T08:05:55.374245+0000","last_active":"2024-05-11T08:06:00.392492+0000","last_peered":"2024-05-11T08:06:00.392492+0000","last_clean":"2024-05-11T08:06:00.392492+0000","last_became_active":"2024-05-11T08:05:55.373871+0000","last_became_peered":"2024-05-11T08:05:55.373871+0000","last_unstale":"2024-05-11T08:06:00.392492+0000","last_undegraded":"2024-05-11T08:06:00.392492+0000","last_fullsized":"2024-05-11T08:06:00.392492+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:32:48.010809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.645850+0000","last_change":"2024-05-11T08:05:55.376534+0000","last_active":"2024-05-11T08:06:01.645850+0000","last_peered":"2024-05-11T08:06:01.645850+0000","last_clean":"2024-05-11T08:06:01.645850+0000","last_became_active":"2024-05-11T08:05:55.376265+0000","last_became_peered":"2024-05-11T08:05:55.376265+0000","last_unstale":"2024-05-11T08:06:01.645850+0000","last_undegraded":"2024-05-11T08:06:01.645850+0000","last_fullsized":"2024-05-11T08:06:01.645850+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:43:37.948904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.388549+0000","last_change":"2024-05-11T08:05:55.381048+0000","last_active":"2024-05-11T08:06:00.388549+0000","last_peered":"2024-05-11T08:06:00.388549+0000","last_clean":"2024-05-11T08:06:00.388549+0000","last_became_active":"2024-05-11T08:05:55.380781+0000","last_became_peered":"2024-05-11T08:05:55.380781+0000","last_unstale":"2024-05-11T08:06:00.388549+0000","last_undegraded":"2024-05-11T08:06:00.388549+0000","last_fullsized":"2024-05-11T08:06:00.388549+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:28:24.740747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.645586+0000","last_change":"2024-05-11T08:05:55.364750+0000","last_active":"2024-05-11T08:06:01.645586+0000","last_peered":"2024-05-11T08:06:01.645586+0000","last_clean":"2024-05-11T08:06:01.645586+0000","last_became_active":"2024-05-11T08:05:55.364495+0000","last_became_peered":"2024-05-11T08:05:55.364495+0000","last_unstale":"2024-05-11T08:06:01.645586+0000","last_undegraded":"2024-05-11T08:06:01.645586+0000","last_fullsized":"2024-05-11T08:06:01.645586+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:43:46.282095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.389971+0000","last_change":"2024-05-11T08:05:55.375569+0000","last_active":"2024-05-11T08:06:00.389971+0000","last_peered":"2024-05-11T08:06:00.389971+0000","last_clean":"2024-05-11T08:06:00.389971+0000","last_became_active":"2024-05-11T08:05:55.375303+0000","last_became_peered":"2024-05-11T08:05:55.375303+0000","last_unstale":"2024-05-11T08:06:00.389971+0000","last_undegraded":"2024-05-11T08:06:00.389971+0000","last_fullsized":"2024-05-11T08:06:00.389971+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:41:38.931447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.389063+0000","last_change":"2024-05-11T08:05:55.377664+0000","last_active":"2024-05-11T08:06:00.389063+0000","last_peered":"2024-05-11T08:06:00.389063+0000","last_clean":"2024-05-11T08:06:00.389063+0000","last_became_active":"2024-05-11T08:05:55.377373+0000","last_became_peered":"2024-05-11T08:05:55.377373+0000","last_unstale":"2024-05-11T08:06:00.389063+0000","last_undegraded":"2024-05-11T08:06:00.389063+0000","last_fullsized":"2024-05-11T08:06:00.389063+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:35:03.601599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.645312+0000","last_change":"2024-05-11T08:05:55.373802+0000","last_active":"2024-05-11T08:06:01.645312+0000","last_peered":"2024-05-11T08:06:01.645312+0000","last_clean":"2024-05-11T08:06:01.645312+0000","last_became_active":"2024-05-11T08:05:55.373558+0000","last_became_peered":"2024-05-11T08:05:55.373558+0000","last_unstale":"2024-05-11T08:06:01.645312+0000","last_undegraded":"2024-05-11T08:06:01.645312+0000","last_fullsized":"2024-05-11T08:06:01.645312+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:43:37.948904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.392536+0000","last_change":"2024-05-11T08:05:55.366027+0000","last_active":"2024-05-11T08:06:00.392536+0000","last_peered":"2024-05-11T08:06:00.392536+0000","last_clean":"2024-05-11T08:06:00.392536+0000","last_became_active":"2024-05-11T08:05:55.365360+0000","last_became_peered":"2024-05-11T08:05:55.365360+0000","last_unstale":"2024-05-11T08:06:00.392536+0000","last_undegraded":"2024-05-11T08:06:00.392536+0000","last_fullsized":"2024-05-11T08:06:00.392536+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:20:43.832246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.646413+0000","last_change":"2024-05-11T08:05:55.369665+0000","last_active":"2024-05-11T08:06:01.646413+0000","last_peered":"2024-05-11T08:06:01.646413+0000","last_clean":"2024-05-11T08:06:01.646413+0000","last_became_active":"2024-05-11T08:05:55.369420+0000","last_became_peered":"2024-05-11T08:05:55.369420+0000","last_unstale":"2024-05-11T08:06:01.646413+0000","last_undegraded":"2024-05-11T08:06:01.646413+0000","last_fullsized":"2024-05-11T08:06:01.646413+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:43:37.948904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.823465+0000","last_change":"2024-05-11T08:05:55.377949+0000","last_active":"2024-05-11T08:06:00.823465+0000","last_peered":"2024-05-11T08:06:00.823465+0000","last_clean":"2024-05-11T08:06:00.823465+0000","last_became_active":"2024-05-11T08:05:55.377677+0000","last_became_peered":"2024-05-11T08:05:55.377677+0000","last_unstale":"2024-05-11T08:06:00.823465+0000","last_undegraded":"2024-05-11T08:06:00.823465+0000","last_fullsized":"2024-05-11T08:06:00.823465+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:50:18.627516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.645928+0000","last_change":"2024-05-11T08:05:55.368388+0000","last_active":"2024-05-11T08:06:01.645928+0000","last_peered":"2024-05-11T08:06:01.645928+0000","last_clean":"2024-05-11T08:06:01.645928+0000","last_became_active":"2024-05-11T08:05:55.368103+0000","last_became_peered":"2024-05-11T08:05:55.368103+0000","last_unstale":"2024-05-11T08:06:01.645928+0000","last_undegraded":"2024-05-11T08:06:01.645928+0000","last_fullsized":"2024-05-11T08:06:01.645928+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:32:48.010809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.646145+0000","last_change":"2024-05-11T08:05:55.368708+0000","last_active":"2024-05-11T08:06:01.646145+0000","last_peered":"2024-05-11T08:06:01.646145+0000","last_clean":"2024-05-11T08:06:01.646145+0000","last_became_active":"2024-05-11T08:05:55.368177+0000","last_became_peered":"2024-05-11T08:05:55.368177+0000","last_unstale":"2024-05-11T08:06:01.646145+0000","last_undegraded":"2024-05-11T08:06:01.646145+0000","last_fullsized":"2024-05-11T08:06:01.646145+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:02:23.158539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.388971+0000","last_change":"2024-05-11T08:05:55.383485+0000","last_active":"2024-05-11T08:06:00.388971+0000","last_peered":"2024-05-11T08:06:00.388971+0000","last_clean":"2024-05-11T08:06:00.388971+0000","last_became_active":"2024-05-11T08:05:55.383016+0000","last_became_peered":"2024-05-11T08:05:55.383016+0000","last_unstale":"2024-05-11T08:06:00.388971+0000","last_undegraded":"2024-05-11T08:06:00.388971+0000","last_fullsized":"2024-05-11T08:06:00.388971+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:51:26.849946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.645483+0000","last_change":"2024-05-11T08:05:55.370103+0000","last_active":"2024-05-11T08:06:01.645483+0000","last_peered":"2024-05-11T08:06:01.645483+0000","last_clean":"2024-05-11T08:06:01.645483+0000","last_became_active":"2024-05-11T08:05:55.369838+0000","last_became_peered":"2024-05-11T08:05:55.369838+0000","last_unstale":"2024-05-11T08:06:01.645483+0000","last_undegraded":"2024-05-11T08:06:01.645483+0000","last_fullsized":"2024-05-11T08:06:01.645483+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:20:43.832246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.388264+0000","last_change":"2024-05-11T08:05:55.379192+0000","last_active":"2024-05-11T08:06:00.388264+0000","last_peered":"2024-05-11T08:06:00.388264+0000","last_clean":"2024-05-11T08:06:00.388264+0000","last_became_active":"2024-05-11T08:05:55.378919+0000","last_became_peered":"2024-05-11T08:05:55.378919+0000","last_unstale":"2024-05-11T08:06:00.388264+0000","last_undegraded":"2024-05-11T08:06:00.388264+0000","last_fullsized":"2024-05-11T08:06:00.388264+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:22:00.631022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.389205+0000","last_change":"2024-05-11T08:05:55.379230+0000","last_active":"2024-05-11T08:06:00.389205+0000","last_peered":"2024-05-11T08:06:00.389205+0000","last_clean":"2024-05-11T08:06:00.389205+0000","last_became_active":"2024-05-11T08:05:55.378960+0000","last_became_peered":"2024-05-11T08:05:55.378960+0000","last_unstale":"2024-05-11T08:06:00.389205+0000","last_undegraded":"2024-05-11T08:06:00.389205+0000","last_fullsized":"2024-05-11T08:06:00.389205+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:22:00.631022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.388309+0000","last_change":"2024-05-11T08:05:55.378492+0000","last_active":"2024-05-11T08:06:00.388309+0000","last_peered":"2024-05-11T08:06:00.388309+0000","last_clean":"2024-05-11T08:06:00.388309+0000","last_became_active":"2024-05-11T08:05:55.378200+0000","last_became_peered":"2024-05-11T08:05:55.378200+0000","last_unstale":"2024-05-11T08:06:00.388309+0000","last_undegraded":"2024-05-11T08:06:00.388309+0000","last_fullsized":"2024-05-11T08:06:00.388309+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:44:43.387850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.389503+0000","last_change":"2024-05-11T08:05:55.379296+0000","last_active":"2024-05-11T08:06:00.389503+0000","last_peered":"2024-05-11T08:06:00.389503+0000","last_clean":"2024-05-11T08:06:00.389503+0000","last_became_active":"2024-05-11T08:05:55.379047+0000","last_became_peered":"2024-05-11T08:05:55.379047+0000","last_unstale":"2024-05-11T08:06:00.389503+0000","last_undegraded":"2024-05-11T08:06:00.389503+0000","last_fullsized":"2024-05-11T08:06:00.389503+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:44:43.387850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.392612+0000","last_change":"2024-05-11T08:05:55.360980+0000","last_active":"2024-05-11T08:06:00.392612+0000","last_peered":"2024-05-11T08:06:00.392612+0000","last_clean":"2024-05-11T08:06:00.392612+0000","last_became_active":"2024-05-11T08:05:55.360626+0000","last_became_peered":"2024-05-11T08:05:55.360626+0000","last_unstale":"2024-05-11T08:06:00.392612+0000","last_undegraded":"2024-05-11T08:06:00.392612+0000","last_fullsized":"2024-05-11T08:06:00.392612+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:02:23.158539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.646210+0000","last_change":"2024-05-11T08:05:55.363639+0000","last_active":"2024-05-11T08:06:01.646210+0000","last_peered":"2024-05-11T08:06:01.646210+0000","last_clean":"2024-05-11T08:06:01.646210+0000","last_became_active":"2024-05-11T08:05:55.363320+0000","last_became_peered":"2024-05-11T08:05:55.363320+0000","last_unstale":"2024-05-11T08:06:01.646210+0000","last_undegraded":"2024-05-11T08:06:01.646210+0000","last_fullsized":"2024-05-11T08:06:01.646210+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:02:23.158539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.388821+0000","last_change":"2024-05-11T08:05:55.380035+0000","last_active":"2024-05-11T08:06:00.388821+0000","last_peered":"2024-05-11T08:06:00.388821+0000","last_clean":"2024-05-11T08:06:00.388821+0000","last_became_active":"2024-05-11T08:05:55.378781+0000","last_became_peered":"2024-05-11T08:05:55.378781+0000","last_unstale":"2024-05-11T08:06:00.388821+0000","last_undegraded":"2024-05-11T08:06:00.388821+0000","last_fullsized":"2024-05-11T08:06:00.388821+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:45:24.551499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509449,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27884,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343956,"statfs":{"total":96636764160,"available":96608210944,"internally_reserved":0,"allocated":880640,"data_stored":701507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509449,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":294912,"data_stored":114399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509449,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":286720,"data_stored":110647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509449,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":294912,"data_stored":110683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509450,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":294912,"data_stored":114399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509450,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":290816,"data_stored":110717,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509449,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27892,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343948,"statfs":{"total":96636764160,"available":96608202752,"internally_reserved":0,"allocated":888832,"data_stored":701563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574857,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":282624,"data_stored":110557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-11T08:06:15.784 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-11T08:06:15.785 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-05-11T08:06:15.785 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-05-11T08:06:15.785 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-05-11T08:06:15.786 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-05-11T08:06:15.919 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.918+0000 7f2f7c26d640 1 Processor -- start 2024-05-11T08:06:15.919 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.918+0000 7f2f7c26d640 1 -- start start 2024-05-11T08:06:15.921 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.919+0000 7f2f7c26d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2f741598d0 0x7f2f740d2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:15.921 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.919+0000 7f2f7c26d640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f2f740cb6c0 0x7f2f740d0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:15.921 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.919+0000 7f2f7c26d640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f740d1190 con 0x7f2f740d0640 2024-05-11T08:06:15.921 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.919+0000 7f2f7c26d640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f2f7406d500 con 0x7f2f740cb6c0 2024-05-11T08:06:15.921 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.919+0000 7f2f7c26d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f2f7406d7e0 con 0x7f2f741598d0 2024-05-11T08:06:15.921 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.919+0000 7f2f79808640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f2f740d0640 0x7f2f740cafb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:60750/0 (socket says 172.21.15.167:60750) 2024-05-11T08:06:15.921 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.919+0000 7f2f79808640 1 -- 172.21.15.167:0/3949452557 learned_addr learned my addr 172.21.15.167:0/3949452557 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T08:06:15.921 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.919+0000 7f2f7a009640 1 --2- 172.21.15.167:0/3949452557 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2f741598d0 0x7f2f740d2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:15.921 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.919+0000 7f2f7a80a640 1 --2- 172.21.15.167:0/3949452557 >> v2:172.21.15.204:3300/0 conn(0x7f2f740cb6c0 0x7f2f740d0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:15.921 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.920+0000 7f2f7a80a640 1 -- 172.21.15.167:0/3949452557 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2f741598d0 msgr2=0x7f2f740d2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:15.921 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.920+0000 7f2f7a80a640 1 --2- 172.21.15.167:0/3949452557 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2f741598d0 0x7f2f740d2cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:15.921 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.920+0000 7f2f7a80a640 1 -- 172.21.15.167:0/3949452557 >> v1:172.21.15.167:6789/0 conn(0x7f2f740d0640 legacy=0x7f2f740cafb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:15.922 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.920+0000 7f2f7a80a640 1 -- 172.21.15.167:0/3949452557 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f7406dac0 con 0x7f2f740cb6c0 2024-05-11T08:06:15.922 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.920+0000 7f2f7a009640 1 --2- 172.21.15.167:0/3949452557 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2f741598d0 0x7f2f740d2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T08:06:15.923 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.920+0000 7f2f7a80a640 1 --2- 172.21.15.167:0/3949452557 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2f740cb6c0 0x7f2f740d0aa0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c009d90 tx=0x7f2f6c02a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2eea7fd78f38c70 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:15.923 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.921+0000 7f2f79007640 1 -- 172.21.15.167:0/3949452557 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f6c007740 con 0x7f2f740cb6c0 2024-05-11T08:06:15.923 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.921+0000 7f2f79007640 1 -- 172.21.15.167:0/3949452557 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f6c0078e0 con 0x7f2f740cb6c0 2024-05-11T08:06:15.923 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.921+0000 7f2f79007640 1 -- 172.21.15.167:0/3949452557 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f6c007cb0 con 0x7f2f740cb6c0 2024-05-11T08:06:15.923 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.921+0000 7f2f7c26d640 1 -- 172.21.15.167:0/3949452557 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2f740cb6c0 msgr2=0x7f2f740d0aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:15.923 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.921+0000 7f2f7c26d640 1 --2- 172.21.15.167:0/3949452557 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2f740cb6c0 0x7f2f740d0aa0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c009d90 tx=0x7f2f6c02a660 comp rx=0 tx=0).stop 2024-05-11T08:06:15.923 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.921+0000 7f2f7c26d640 1 -- 172.21.15.167:0/3949452557 shutdown_connections 2024-05-11T08:06:15.923 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.921+0000 7f2f7c26d640 1 --2- 172.21.15.167:0/3949452557 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2f740cb6c0 0x7f2f740d0aa0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:15.923 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.921+0000 7f2f7c26d640 1 --2- 172.21.15.167:0/3949452557 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2f741598d0 0x7f2f740d2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:15.923 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.921+0000 7f2f7c26d640 1 -- 172.21.15.167:0/3949452557 >> 172.21.15.167:0/3949452557 conn(0x7f2f740d1ba0 msgr2=0x7f2f740d1fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:15.923 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.921+0000 7f2f7c26d640 1 -- 172.21.15.167:0/3949452557 shutdown_connections 2024-05-11T08:06:15.924 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.921+0000 7f2f7c26d640 1 -- 172.21.15.167:0/3949452557 wait complete. 2024-05-11T08:06:15.925 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.923+0000 7f2f7c26d640 1 Processor -- start 2024-05-11T08:06:15.925 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.923+0000 7f2f7c26d640 1 -- start start 2024-05-11T08:06:15.926 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.923+0000 7f2f7c26d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2f740cb6c0 0x7f2f740cece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:15.926 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.923+0000 7f2f7c26d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2f740d0640 0x7f2f740b7e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:15.926 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.923+0000 7f2f7c26d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2f741598d0 0x7f2f74146b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:15.926 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.923+0000 7f2f7c26d640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2f7406d1b0 con 0x7f2f741598d0 2024-05-11T08:06:15.926 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.923+0000 7f2f7c26d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f2f74159cb0 con 0x7f2f740d0640 2024-05-11T08:06:15.926 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.923+0000 7f2f7c26d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f2f7406dfe0 con 0x7f2f740cb6c0 2024-05-11T08:06:15.926 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.923+0000 7f2f7a80a640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2f741598d0 0x7f2f74146b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:15.926 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.923+0000 7f2f7a80a640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2f741598d0 0x7f2f74146b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:54174/0 (socket says 172.21.15.167:54174) 2024-05-11T08:06:15.926 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.923+0000 7f2f7a80a640 1 -- 172.21.15.167:0/1629336581 learned_addr learned my addr 172.21.15.167:0/1629336581 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:15.926 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.924+0000 7f2f79808640 1 --2- 172.21.15.167:0/1629336581 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2f740d0640 0x7f2f740b7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:15.926 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.924+0000 7f2f7a009640 1 --2- 172.21.15.167:0/1629336581 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2f740cb6c0 0x7f2f740cece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:15.926 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.924+0000 7f2f7a80a640 1 -- 172.21.15.167:0/1629336581 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2f740cb6c0 msgr2=0x7f2f740cece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:15.926 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.924+0000 7f2f7a80a640 1 --2- 172.21.15.167:0/1629336581 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2f740cb6c0 0x7f2f740cece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:15.926 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.924+0000 7f2f7a80a640 1 -- 172.21.15.167:0/1629336581 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2f740d0640 msgr2=0x7f2f740b7e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:15.927 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.924+0000 7f2f7a80a640 1 --2- 172.21.15.167:0/1629336581 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2f740d0640 0x7f2f740b7e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:15.927 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.924+0000 7f2f7a80a640 1 -- 172.21.15.167:0/1629336581 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f740b8650 con 0x7f2f741598d0 2024-05-11T08:06:15.927 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.924+0000 7f2f7a80a640 1 --2- 172.21.15.167:0/1629336581 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2f741598d0 0x7f2f74146b40 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c02ac50 tx=0x7f2f6c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:15.927 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.924+0000 7f2f6affd640 1 -- 172.21.15.167:0/1629336581 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f6c040070 con 0x7f2f741598d0 2024-05-11T08:06:15.927 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.924+0000 7f2f6affd640 1 -- 172.21.15.167:0/1629336581 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f6c031bc0 con 0x7f2f741598d0 2024-05-11T08:06:15.927 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.924+0000 7f2f7c26d640 1 -- 172.21.15.167:0/1629336581 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f740b8930 con 0x7f2f741598d0 2024-05-11T08:06:15.927 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.924+0000 7f2f6affd640 1 -- 172.21.15.167:0/1629336581 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f6c009450 con 0x7f2f741598d0 2024-05-11T08:06:15.927 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.924+0000 7f2f7c26d640 1 -- 172.21.15.167:0/1629336581 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f740b8ef0 con 0x7f2f741598d0 2024-05-11T08:06:15.928 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.925+0000 7f2f7c26d640 1 -- 172.21.15.167:0/1629336581 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f74069590 con 0x7f2f741598d0 2024-05-11T08:06:15.928 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.926+0000 7f2f6affd640 1 -- 172.21.15.167:0/1629336581 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f2f6c039040 con 0x7f2f741598d0 2024-05-11T08:06:15.928 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.926+0000 7f2f6affd640 1 --2- 172.21.15.167:0/1629336581 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f2f580762d0 0x7f2f58078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:15.928 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.926+0000 7f2f6affd640 1 -- 172.21.15.167:0/1629336581 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f2f6c0b8fb0 con 0x7f2f741598d0 2024-05-11T08:06:15.928 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.926+0000 7f2f7a009640 1 --2- 172.21.15.167:0/1629336581 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f2f580762d0 0x7f2f58078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:15.928 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.927+0000 7f2f7a009640 1 --2- 172.21.15.167:0/1629336581 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f2f580762d0 0x7f2f58078790 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2f70009560 tx=0x7f2f70009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:15.931 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:15.931+0000 7f2f6affd640 1 -- 172.21.15.167:0/1629336581 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f2f6c082010 con 0x7f2f741598d0 2024-05-11T08:06:16.158 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.157+0000 7f2f7c26d640 1 -- 172.21.15.167:0/1629336581 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f2f740d0aa0 con 0x7f2f741598d0 2024-05-11T08:06:16.161 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.160+0000 7f2f6affd640 1 -- 172.21.15.167:0/1629336581 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+189524 (secure 0 0 0) 0x7f2f6c086310 con 0x7f2f741598d0 2024-05-11T08:06:16.163 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T08:06:16.164 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.162+0000 7f2f7c26d640 1 -- 172.21.15.167:0/1629336581 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f2f580762d0 msgr2=0x7f2f58078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:16.164 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.162+0000 7f2f7c26d640 1 --2- 172.21.15.167:0/1629336581 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f2f580762d0 0x7f2f58078790 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2f70009560 tx=0x7f2f70009190 comp rx=0 tx=0).stop 2024-05-11T08:06:16.164 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.162+0000 7f2f7c26d640 1 -- 172.21.15.167:0/1629336581 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2f741598d0 msgr2=0x7f2f74146b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:16.165 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.162+0000 7f2f7c26d640 1 --2- 172.21.15.167:0/1629336581 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2f741598d0 0x7f2f74146b40 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c02ac50 tx=0x7f2f6c004a40 comp rx=0 tx=0).stop 2024-05-11T08:06:16.165 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.162+0000 7f2f7c26d640 1 -- 172.21.15.167:0/1629336581 shutdown_connections 2024-05-11T08:06:16.165 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.162+0000 7f2f7c26d640 1 --2- 172.21.15.167:0/1629336581 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f2f580762d0 0x7f2f58078790 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:16.165 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.162+0000 7f2f7c26d640 1 --2- 172.21.15.167:0/1629336581 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2f741598d0 0x7f2f74146b40 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:16.165 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.162+0000 7f2f7c26d640 1 --2- 172.21.15.167:0/1629336581 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2f740d0640 0x7f2f740b7e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:16.165 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.162+0000 7f2f7c26d640 1 --2- 172.21.15.167:0/1629336581 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2f740cb6c0 0x7f2f740cece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:16.165 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.162+0000 7f2f7c26d640 1 -- 172.21.15.167:0/1629336581 >> 172.21.15.167:0/1629336581 conn(0x7f2f740d1ba0 msgr2=0x7f2f740ca480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:16.165 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.163+0000 7f2f7c26d640 1 -- 172.21.15.167:0/1629336581 shutdown_connections 2024-05-11T08:06:16.165 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.163+0000 7f2f7c26d640 1 -- 172.21.15.167:0/1629336581 wait complete. 2024-05-11T08:06:16.174 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":5,"flags":0,"active_gid":4113,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":253569496},{"type":"v1","addr":"172.21.15.167:6833","nonce":253569496}]},"active_addr":"172.21.15.167:6833/253569496","active_change":"2024-05-11T08:05:35.907825+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4102,"name":"x","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.167:0","nonce":2252409242}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.167:0","nonce":1183588375}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.167:0","nonce":2157517488}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.167:0","nonce":275717854}]}]} 2024-05-11T08:06:16.177 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-05-11T08:06:16.178 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-05-11T08:06:16.178 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-11T08:06:16.277 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.276+0000 7f9a225ac640 1 Processor -- start 2024-05-11T08:06:16.277 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.276+0000 7f9a225ac640 1 -- start start 2024-05-11T08:06:16.278 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.277+0000 7f9a225ac640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f9a1c0cb6c0 0x7f9a1c06dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:16.278 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.277+0000 7f9a225ac640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a1c0d1190 con 0x7f9a1c1598d0 2024-05-11T08:06:16.278 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.277+0000 7f9a225ac640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a1c06d500 con 0x7f9a1c0d2cf0 2024-05-11T08:06:16.278 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.277+0000 7f9a225ac640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f9a1c06d7e0 con 0x7f9a1c0cb6c0 2024-05-11T08:06:16.278 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.277+0000 7f9a20b49640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f9a1c0cb6c0 0x7f9a1c06dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:16.278 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.277+0000 7f9a1bfff640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f9a1c1598d0 0x7f9a1c0d0640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:60754/0 (socket says 172.21.15.167:60754) 2024-05-11T08:06:16.278 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.277+0000 7f9a1bfff640 1 -- 172.21.15.167:0/2033449798 learned_addr learned my addr 172.21.15.167:0/2033449798 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T08:06:16.278 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.277+0000 7f9a20b49640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f9a1c0cb6c0 0x7f9a1c06dfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:54182/0 (socket says 172.21.15.167:54182) 2024-05-11T08:06:16.278 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.277+0000 7f9a20b49640 1 -- 172.21.15.167:0/2033449798 >> v1:172.21.15.204:6789/0 conn(0x7f9a1c0d2cf0 legacy=0x7f9a1c0cafb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:16.279 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.277+0000 7f9a20b49640 1 -- 172.21.15.167:0/2033449798 >> v1:172.21.15.167:6789/0 conn(0x7f9a1c1598d0 legacy=0x7f9a1c0d0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:16.279 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.277+0000 7f9a20b49640 1 -- 172.21.15.167:0/2033449798 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a1c06dac0 con 0x7f9a1c0cb6c0 2024-05-11T08:06:16.279 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.277+0000 7f9a1affd640 1 -- 172.21.15.167:0/2033449798 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4063793354 0 0) 0x7f9a1c0d1190 con 0x7f9a1c1598d0 2024-05-11T08:06:16.280 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.278+0000 7f9a20b49640 1 --2- 172.21.15.167:0/2033449798 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9a1c0cb6c0 0x7f9a1c06dfe0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9a14009d90 tx=0x7f9a1402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d190116c4c92f650 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:16.280 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.278+0000 7f9a1affd640 1 -- 172.21.15.167:0/2033449798 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a14007740 con 0x7f9a1c0cb6c0 2024-05-11T08:06:16.280 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.278+0000 7f9a1affd640 1 -- 172.21.15.167:0/2033449798 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a140078e0 con 0x7f9a1c0cb6c0 2024-05-11T08:06:16.280 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.278+0000 7f9a1affd640 1 -- 172.21.15.167:0/2033449798 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a14007cb0 con 0x7f9a1c0cb6c0 2024-05-11T08:06:16.280 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.278+0000 7f9a225ac640 1 -- 172.21.15.167:0/2033449798 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9a1c0cb6c0 msgr2=0x7f9a1c06dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:16.280 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.278+0000 7f9a225ac640 1 --2- 172.21.15.167:0/2033449798 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9a1c0cb6c0 0x7f9a1c06dfe0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9a14009d90 tx=0x7f9a1402a6f0 comp rx=0 tx=0).stop 2024-05-11T08:06:16.280 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.278+0000 7f9a225ac640 1 -- 172.21.15.167:0/2033449798 shutdown_connections 2024-05-11T08:06:16.280 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.278+0000 7f9a225ac640 1 --2- 172.21.15.167:0/2033449798 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9a1c0cb6c0 0x7f9a1c06dfe0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:16.280 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.278+0000 7f9a225ac640 1 -- 172.21.15.167:0/2033449798 >> 172.21.15.167:0/2033449798 conn(0x7f9a1c0d1ba0 msgr2=0x7f9a1c0d1fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:16.280 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.278+0000 7f9a225ac640 1 -- 172.21.15.167:0/2033449798 shutdown_connections 2024-05-11T08:06:16.281 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.278+0000 7f9a225ac640 1 -- 172.21.15.167:0/2033449798 wait complete. 2024-05-11T08:06:16.283 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.280+0000 7f9a225ac640 1 Processor -- start 2024-05-11T08:06:16.283 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.280+0000 7f9a225ac640 1 -- start start 2024-05-11T08:06:16.283 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.280+0000 7f9a225ac640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9a1c0cb6c0 0x7f9a1c0bad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:16.283 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.280+0000 7f9a225ac640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9a1c0d2cf0 0x7f9a1c0ce9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:16.283 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.280+0000 7f9a225ac640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9a1c1598d0 0x7f9a1c152830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:16.283 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.280+0000 7f9a225ac640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9a1c06d1b0 con 0x7f9a1c0cb6c0 2024-05-11T08:06:16.283 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.280+0000 7f9a225ac640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f9a1c159cb0 con 0x7f9a1c1598d0 2024-05-11T08:06:16.283 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.280+0000 7f9a225ac640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f9a1c0d0d50 con 0x7f9a1c0d2cf0 2024-05-11T08:06:16.283 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.281+0000 7f9a1bfff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9a1c0cb6c0 0x7f9a1c0bad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:16.283 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.281+0000 7f9a1bfff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9a1c0cb6c0 0x7f9a1c0bad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:54192/0 (socket says 172.21.15.167:54192) 2024-05-11T08:06:16.283 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.281+0000 7f9a1bfff640 1 -- 172.21.15.167:0/2114771754 learned_addr learned my addr 172.21.15.167:0/2114771754 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:16.283 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.281+0000 7f9a20b49640 1 --2- 172.21.15.167:0/2114771754 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9a1c1598d0 0x7f9a1c152830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:16.283 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.281+0000 7f9a1b7fe640 1 --2- 172.21.15.167:0/2114771754 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9a1c0d2cf0 0x7f9a1c0ce9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:16.283 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.281+0000 7f9a1bfff640 1 -- 172.21.15.167:0/2114771754 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9a1c0d2cf0 msgr2=0x7f9a1c0ce9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:16.284 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.281+0000 7f9a1bfff640 1 --2- 172.21.15.167:0/2114771754 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9a1c0d2cf0 0x7f9a1c0ce9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:16.284 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.281+0000 7f9a1bfff640 1 -- 172.21.15.167:0/2114771754 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9a1c1598d0 msgr2=0x7f9a1c152830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:16.284 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.281+0000 7f9a1bfff640 1 --2- 172.21.15.167:0/2114771754 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9a1c1598d0 0x7f9a1c152830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:16.284 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.281+0000 7f9a1bfff640 1 -- 172.21.15.167:0/2114771754 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a1c0bb240 con 0x7f9a1c0cb6c0 2024-05-11T08:06:16.284 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.281+0000 7f9a1b7fe640 1 --2- 172.21.15.167:0/2114771754 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9a1c0d2cf0 0x7f9a1c0ce9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T08:06:16.284 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.281+0000 7f9a1bfff640 1 --2- 172.21.15.167:0/2114771754 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9a1c0cb6c0 0x7f9a1c0bad00 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c00c3d0 tx=0x7f9a0c00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:16.284 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.281+0000 7f9a18ff9640 1 -- 172.21.15.167:0/2114771754 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a0c0074e0 con 0x7f9a1c0cb6c0 2024-05-11T08:06:16.284 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.281+0000 7f9a18ff9640 1 -- 172.21.15.167:0/2114771754 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a0c0047e0 con 0x7f9a1c0cb6c0 2024-05-11T08:06:16.284 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.281+0000 7f9a18ff9640 1 -- 172.21.15.167:0/2114771754 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a0c004c10 con 0x7f9a1c0cb6c0 2024-05-11T08:06:16.284 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.281+0000 7f9a225ac640 1 -- 172.21.15.167:0/2114771754 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a1c147250 con 0x7f9a1c0cb6c0 2024-05-11T08:06:16.284 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.281+0000 7f9a225ac640 1 -- 172.21.15.167:0/2114771754 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a1c151790 con 0x7f9a1c0cb6c0 2024-05-11T08:06:16.285 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.282+0000 7f9a225ac640 1 -- 172.21.15.167:0/2114771754 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a1c069590 con 0x7f9a1c0cb6c0 2024-05-11T08:06:16.285 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.283+0000 7f9a18ff9640 1 -- 172.21.15.167:0/2114771754 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f9a0c011070 con 0x7f9a1c0cb6c0 2024-05-11T08:06:16.285 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.283+0000 7f9a18ff9640 1 --2- 172.21.15.167:0/2114771754 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f99fc0762d0 0x7f99fc078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:16.285 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.283+0000 7f9a1b7fe640 1 --2- 172.21.15.167:0/2114771754 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f99fc0762d0 0x7f99fc078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:16.285 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.283+0000 7f9a18ff9640 1 -- 172.21.15.167:0/2114771754 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f9a0c0973d0 con 0x7f9a1c0cb6c0 2024-05-11T08:06:16.285 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.284+0000 7f9a1b7fe640 1 --2- 172.21.15.167:0/2114771754 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f99fc0762d0 0x7f99fc078790 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9a04007fd0 tx=0x7f9a04009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:16.288 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.288+0000 7f9a18ff9640 1 -- 172.21.15.167:0/2114771754 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f9a0c060a80 con 0x7f9a1c0cb6c0 2024-05-11T08:06:16.470 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.469+0000 7f9a225ac640 1 -- 172.21.15.167:0/2114771754 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9a1c0cf7b0 con 0x7f9a1c0cb6c0 2024-05-11T08:06:16.471 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.470+0000 7f9a18ff9640 1 -- 172.21.15.167:0/2114771754 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18562 (secure 0 0 0) 0x7f9a0c060380 con 0x7f9a1c0cb6c0 2024-05-11T08:06:16.471 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T08:06:16.472 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":27,"fsid":"2034778d-8ff1-4823-aa44-ded32b518241","created":"2024-05-11T08:05:31.612421+0000","modified":"2024-05-11T08:05:59.756806+0000","last_up_change":"2024-05-11T08:05:42.557382+0000","last_in_change":"2024-05-11T08:05:33.757311+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-11T08:05:43.931263+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-11T08:05:52.573726+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-11T08:05:53.670746+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-11T08:05:57.380272+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"587eaced-5ded-406d-884b-30c1caa8686b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":3863307356},{"type":"v1","addr":"172.21.15.167:6806","nonce":3863307356}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":3863307356},{"type":"v1","addr":"172.21.15.167:6810","nonce":3863307356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":3863307356},{"type":"v1","addr":"172.21.15.167:6815","nonce":3863307356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":3863307356},{"type":"v1","addr":"172.21.15.167:6813","nonce":3863307356}]},"public_addr":"172.21.15.167:6806/3863307356","cluster_addr":"172.21.15.167:6810/3863307356","heartbeat_back_addr":"172.21.15.167:6815/3863307356","heartbeat_front_addr":"172.21.15.167:6813/3863307356","state":["exists","up"]},{"osd":1,"uuid":"37c59104-2dd0-4cf9-85ea-56636f6b02a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":1585024048},{"type":"v1","addr":"172.21.15.167:6817","nonce":1585024048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1585024048},{"type":"v1","addr":"172.21.15.167:6819","nonce":1585024048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1585024048},{"type":"v1","addr":"172.21.15.167:6823","nonce":1585024048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":1585024048},{"type":"v1","addr":"172.21.15.167:6821","nonce":1585024048}]},"public_addr":"172.21.15.167:6817/1585024048","cluster_addr":"172.21.15.167:6819/1585024048","heartbeat_back_addr":"172.21.15.167:6823/1585024048","heartbeat_front_addr":"172.21.15.167:6821/1585024048","state":["exists","up"]},{"osd":2,"uuid":"2bb4f7f5-c698-451b-b4bb-78e716921291","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":3242166900},{"type":"v1","addr":"172.21.15.167:6801","nonce":3242166900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":3242166900},{"type":"v1","addr":"172.21.15.167:6803","nonce":3242166900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6809","nonce":3242166900},{"type":"v1","addr":"172.21.15.167:6811","nonce":3242166900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6805","nonce":3242166900},{"type":"v1","addr":"172.21.15.167:6807","nonce":3242166900}]},"public_addr":"172.21.15.167:6801/3242166900","cluster_addr":"172.21.15.167:6803/3242166900","heartbeat_back_addr":"172.21.15.167:6811/3242166900","heartbeat_front_addr":"172.21.15.167:6807/3242166900","state":["exists","up"]},{"osd":3,"uuid":"9dd26759-a338-49d2-8103-45d068b9c000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":453689293},{"type":"v1","addr":"172.21.15.167:6825","nonce":453689293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":453689293},{"type":"v1","addr":"172.21.15.167:6827","nonce":453689293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":453689293},{"type":"v1","addr":"172.21.15.167:6831","nonce":453689293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":453689293},{"type":"v1","addr":"172.21.15.167:6829","nonce":453689293}]},"public_addr":"172.21.15.167:6825/453689293","cluster_addr":"172.21.15.167:6827/453689293","heartbeat_back_addr":"172.21.15.167:6831/453689293","heartbeat_front_addr":"172.21.15.167:6829/453689293","state":["exists","up"]},{"osd":4,"uuid":"cb27250d-2a80-4c7b-8cef-3a7af05c8a16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1311831500},{"type":"v1","addr":"172.21.15.204:6801","nonce":1311831500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1311831500},{"type":"v1","addr":"172.21.15.204:6803","nonce":1311831500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1311831500},{"type":"v1","addr":"172.21.15.204:6807","nonce":1311831500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1311831500},{"type":"v1","addr":"172.21.15.204:6805","nonce":1311831500}]},"public_addr":"172.21.15.204:6801/1311831500","cluster_addr":"172.21.15.204:6803/1311831500","heartbeat_back_addr":"172.21.15.204:6807/1311831500","heartbeat_front_addr":"172.21.15.204:6805/1311831500","state":["exists","up"]},{"osd":5,"uuid":"92998541-ffec-430b-ab76-8c3cb7c2752d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2921847521},{"type":"v1","addr":"172.21.15.204:6825","nonce":2921847521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2921847521},{"type":"v1","addr":"172.21.15.204:6827","nonce":2921847521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2921847521},{"type":"v1","addr":"172.21.15.204:6831","nonce":2921847521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2921847521},{"type":"v1","addr":"172.21.15.204:6829","nonce":2921847521}]},"public_addr":"172.21.15.204:6825/2921847521","cluster_addr":"172.21.15.204:6827/2921847521","heartbeat_back_addr":"172.21.15.204:6831/2921847521","heartbeat_front_addr":"172.21.15.204:6829/2921847521","state":["exists","up"]},{"osd":6,"uuid":"9543479c-2ec5-42a8-8933-707bfed2e048","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":2889851077},{"type":"v1","addr":"172.21.15.204:6817","nonce":2889851077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":2889851077},{"type":"v1","addr":"172.21.15.204:6819","nonce":2889851077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":2889851077},{"type":"v1","addr":"172.21.15.204:6823","nonce":2889851077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":2889851077},{"type":"v1","addr":"172.21.15.204:6821","nonce":2889851077}]},"public_addr":"172.21.15.204:6817/2889851077","cluster_addr":"172.21.15.204:6819/2889851077","heartbeat_back_addr":"172.21.15.204:6823/2889851077","heartbeat_front_addr":"172.21.15.204:6821/2889851077","state":["exists","up"]},{"osd":7,"uuid":"c45a5647-8358-409e-af91-c05dda65f0b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":1949534184},{"type":"v1","addr":"172.21.15.204:6809","nonce":1949534184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":1949534184},{"type":"v1","addr":"172.21.15.204:6811","nonce":1949534184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1949534184},{"type":"v1","addr":"172.21.15.204:6815","nonce":1949534184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1949534184},{"type":"v1","addr":"172.21.15.204:6813","nonce":1949534184}]},"public_addr":"172.21.15.204:6809/1949534184","cluster_addr":"172.21.15.204:6811/1949534184","heartbeat_back_addr":"172.21.15.204:6815/1949534184","heartbeat_front_addr":"172.21.15.204:6813/1949534184","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.952253+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.940689+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.860682+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.929350+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.901627+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.957451+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.987249+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.951197+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-11T08:06:16.474 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.472+0000 7f9a225ac640 1 -- 172.21.15.167:0/2114771754 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f99fc0762d0 msgr2=0x7f99fc078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:16.474 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.472+0000 7f9a225ac640 1 --2- 172.21.15.167:0/2114771754 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f99fc0762d0 0x7f99fc078790 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9a04007fd0 tx=0x7f9a04009040 comp rx=0 tx=0).stop 2024-05-11T08:06:16.474 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.472+0000 7f9a225ac640 1 -- 172.21.15.167:0/2114771754 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9a1c0cb6c0 msgr2=0x7f9a1c0bad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:16.474 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.472+0000 7f9a225ac640 1 --2- 172.21.15.167:0/2114771754 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9a1c0cb6c0 0x7f9a1c0bad00 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c00c3d0 tx=0x7f9a0c00c890 comp rx=0 tx=0).stop 2024-05-11T08:06:16.474 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.472+0000 7f9a225ac640 1 -- 172.21.15.167:0/2114771754 shutdown_connections 2024-05-11T08:06:16.474 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.472+0000 7f9a225ac640 1 --2- 172.21.15.167:0/2114771754 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f99fc0762d0 0x7f99fc078790 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:16.474 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.472+0000 7f9a225ac640 1 --2- 172.21.15.167:0/2114771754 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9a1c1598d0 0x7f9a1c152830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:16.474 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.472+0000 7f9a225ac640 1 --2- 172.21.15.167:0/2114771754 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9a1c0d2cf0 0x7f9a1c0ce9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:16.474 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.472+0000 7f9a225ac640 1 --2- 172.21.15.167:0/2114771754 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9a1c0cb6c0 0x7f9a1c0bad00 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:16.474 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.472+0000 7f9a225ac640 1 -- 172.21.15.167:0/2114771754 >> 172.21.15.167:0/2114771754 conn(0x7f9a1c0d1ba0 msgr2=0x7f9a1c0cad20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:16.475 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.472+0000 7f9a225ac640 1 -- 172.21.15.167:0/2114771754 shutdown_connections 2024-05-11T08:06:16.475 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.473+0000 7f9a225ac640 1 -- 172.21.15.167:0/2114771754 wait complete. 2024-05-11T08:06:16.485 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-05-11T08:06:16.485 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-11T08:06:16.585 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.583+0000 7f329191c640 1 Processor -- start 2024-05-11T08:06:16.585 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.583+0000 7f329191c640 1 -- start start 2024-05-11T08:06:16.586 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.584+0000 7f329191c640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f328c0d0b50 0x7f328c162e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:16.586 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.584+0000 7f329191c640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f328c0d2aa0 0x7f328c163380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:16.586 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.584+0000 7f329191c640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f328c0cd670 con 0x7f328c155b60 2024-05-11T08:06:16.586 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.584+0000 7f329191c640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f328c0ce210 con 0x7f328c0d0b50 2024-05-11T08:06:16.586 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.584+0000 7f329191c640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f328c1566e0 con 0x7f328c0d2aa0 2024-05-11T08:06:16.586 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.584+0000 7f328affd640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f328c0d0b50 0x7f328c162e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:16.586 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.584+0000 7f328affd640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f328c0d0b50 0x7f328c162e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:54208/0 (socket says 172.21.15.167:54208) 2024-05-11T08:06:16.586 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.584+0000 7f328affd640 1 -- 172.21.15.167:0/3614171309 learned_addr learned my addr 172.21.15.167:0/3614171309 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:16.587 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.585+0000 7f32827fc640 1 --2- 172.21.15.167:0/3614171309 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f328c0d2aa0 0x7f328c163380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:16.587 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.585+0000 7f328affd640 1 -- 172.21.15.167:0/3614171309 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f328c0d2aa0 msgr2=0x7f328c163380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:16.587 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.585+0000 7f328affd640 1 --2- 172.21.15.167:0/3614171309 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f328c0d2aa0 0x7f328c163380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:16.587 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.585+0000 7f328affd640 1 -- 172.21.15.167:0/3614171309 >> v1:172.21.15.204:6789/0 conn(0x7f328c155b60 legacy=0x7f328c155fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:16.587 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.585+0000 7f328affd640 1 -- 172.21.15.167:0/3614171309 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f328c0cd8e0 con 0x7f328c0d0b50 2024-05-11T08:06:16.587 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.585+0000 7f32827fc640 1 --2- 172.21.15.167:0/3614171309 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f328c0d2aa0 0x7f328c163380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T08:06:16.588 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.585+0000 7f328affd640 1 --2- 172.21.15.167:0/3614171309 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f328c0d0b50 0x7f328c162e40 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f328c0d1d60 tx=0x7f327c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7497a9334f6e033a server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:16.588 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.586+0000 7f328a7fc640 1 -- 172.21.15.167:0/3614171309 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f327c037070 con 0x7f328c0d0b50 2024-05-11T08:06:16.588 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.586+0000 7f328a7fc640 1 -- 172.21.15.167:0/3614171309 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f327c032440 con 0x7f328c0d0b50 2024-05-11T08:06:16.588 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.586+0000 7f328a7fc640 1 -- 172.21.15.167:0/3614171309 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f327c032810 con 0x7f328c0d0b50 2024-05-11T08:06:16.588 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.586+0000 7f329191c640 1 -- 172.21.15.167:0/3614171309 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f328c0d0b50 msgr2=0x7f328c162e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:16.588 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.586+0000 7f329191c640 1 --2- 172.21.15.167:0/3614171309 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f328c0d0b50 0x7f328c162e40 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f328c0d1d60 tx=0x7f327c02a180 comp rx=0 tx=0).stop 2024-05-11T08:06:16.589 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.586+0000 7f329191c640 1 -- 172.21.15.167:0/3614171309 shutdown_connections 2024-05-11T08:06:16.589 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.586+0000 7f329191c640 1 --2- 172.21.15.167:0/3614171309 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f328c0d2aa0 0x7f328c163380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:16.589 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.586+0000 7f329191c640 1 --2- 172.21.15.167:0/3614171309 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f328c0d0b50 0x7f328c162e40 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:16.589 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.586+0000 7f329191c640 1 -- 172.21.15.167:0/3614171309 >> 172.21.15.167:0/3614171309 conn(0x7f328c0d2f80 msgr2=0x7f328c0d3360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:16.589 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.586+0000 7f329191c640 1 -- 172.21.15.167:0/3614171309 shutdown_connections 2024-05-11T08:06:16.589 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.586+0000 7f329191c640 1 -- 172.21.15.167:0/3614171309 wait complete. 2024-05-11T08:06:16.589 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.587+0000 7f329191c640 1 Processor -- start 2024-05-11T08:06:16.589 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.588+0000 7f329191c640 1 -- start start 2024-05-11T08:06:16.591 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.588+0000 7f329191c640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f328c0d0b50 0x7f328c0ca2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:16.591 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.588+0000 7f329191c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f328c0d2aa0 0x7f328c0cc200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:16.591 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.588+0000 7f329191c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f328c155b60 0x7f328c0cc740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:16.592 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.588+0000 7f329191c640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f328c0d0f30 con 0x7f328c0d2aa0 2024-05-11T08:06:16.592 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.588+0000 7f329191c640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f328c0cdea0 con 0x7f328c155b60 2024-05-11T08:06:16.592 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.588+0000 7f329191c640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f328c0d10b0 con 0x7f328c0d0b50 2024-05-11T08:06:16.592 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.588+0000 7f32827fc640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f328c0d2aa0 0x7f328c0cc200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:16.592 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.588+0000 7f32827fc640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f328c0d2aa0 0x7f328c0cc200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:54218/0 (socket says 172.21.15.167:54218) 2024-05-11T08:06:16.592 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.588+0000 7f32827fc640 1 -- 172.21.15.167:0/2804565560 learned_addr learned my addr 172.21.15.167:0/2804565560 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:16.592 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.589+0000 7f328affd640 1 --2- 172.21.15.167:0/2804565560 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f328c0d0b50 0x7f328c0ca2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:16.592 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.589+0000 7f328b7fe640 1 --2- 172.21.15.167:0/2804565560 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f328c155b60 0x7f328c0cc740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:16.592 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.589+0000 7f32827fc640 1 -- 172.21.15.167:0/2804565560 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f328c0d0b50 msgr2=0x7f328c0ca2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:16.592 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.589+0000 7f32827fc640 1 --2- 172.21.15.167:0/2804565560 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f328c0d0b50 0x7f328c0ca2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:16.592 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.589+0000 7f32827fc640 1 -- 172.21.15.167:0/2804565560 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f328c155b60 msgr2=0x7f328c0cc740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:16.592 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.589+0000 7f32827fc640 1 --2- 172.21.15.167:0/2804565560 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f328c155b60 0x7f328c0cc740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:16.592 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.589+0000 7f32827fc640 1 -- 172.21.15.167:0/2804565560 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f328c1f8760 con 0x7f328c0d2aa0 2024-05-11T08:06:16.592 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.589+0000 7f328b7fe640 1 --2- 172.21.15.167:0/2804565560 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f328c155b60 0x7f328c0cc740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T08:06:16.592 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.589+0000 7f32827fc640 1 --2- 172.21.15.167:0/2804565560 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f328c0d2aa0 0x7f328c0cc200 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f327400c7f0 tx=0x7f327400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:16.592 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.589+0000 7f3288ff9640 1 -- 172.21.15.167:0/2804565560 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3274015070 con 0x7f328c0d2aa0 2024-05-11T08:06:16.593 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.589+0000 7f3288ff9640 1 -- 172.21.15.167:0/2804565560 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3274007a30 con 0x7f328c0d2aa0 2024-05-11T08:06:16.593 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.589+0000 7f329191c640 1 -- 172.21.15.167:0/2804565560 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f328c0cce50 con 0x7f328c0d2aa0 2024-05-11T08:06:16.593 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.589+0000 7f3288ff9640 1 -- 172.21.15.167:0/2804565560 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3274007e00 con 0x7f328c0d2aa0 2024-05-11T08:06:16.593 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.589+0000 7f329191c640 1 -- 172.21.15.167:0/2804565560 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f328c157ec0 con 0x7f328c0d2aa0 2024-05-11T08:06:16.593 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.590+0000 7f329191c640 1 -- 172.21.15.167:0/2804565560 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f328c069bc0 con 0x7f328c0d2aa0 2024-05-11T08:06:16.593 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.590+0000 7f3288ff9640 1 -- 172.21.15.167:0/2804565560 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f3274018070 con 0x7f328c0d2aa0 2024-05-11T08:06:16.594 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.591+0000 7f3288ff9640 1 --2- 172.21.15.167:0/2804565560 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f326c0762d0 0x7f326c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:16.594 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.591+0000 7f3288ff9640 1 -- 172.21.15.167:0/2804565560 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f3274097890 con 0x7f328c0d2aa0 2024-05-11T08:06:16.594 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.591+0000 7f328affd640 1 --2- 172.21.15.167:0/2804565560 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f326c0762d0 0x7f326c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:16.594 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.592+0000 7f328affd640 1 --2- 172.21.15.167:0/2804565560 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f326c0762d0 0x7f326c078790 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f327c034000 tx=0x7f327c02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:16.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.596+0000 7f3288ff9640 1 -- 172.21.15.167:0/2804565560 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f3274060eb0 con 0x7f328c0d2aa0 2024-05-11T08:06:16.778 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.777+0000 7f329191c640 1 -- 172.21.15.167:0/2804565560 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f328c0d2200 con 0x7f328c0d2aa0 2024-05-11T08:06:16.779 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.778+0000 7f3288ff9640 1 -- 172.21.15.167:0/2804565560 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18562 (secure 0 0 0) 0x7f32740607b0 con 0x7f328c0d2aa0 2024-05-11T08:06:16.780 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T08:06:16.780 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":27,"fsid":"2034778d-8ff1-4823-aa44-ded32b518241","created":"2024-05-11T08:05:31.612421+0000","modified":"2024-05-11T08:05:59.756806+0000","last_up_change":"2024-05-11T08:05:42.557382+0000","last_in_change":"2024-05-11T08:05:33.757311+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-11T08:05:43.931263+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-11T08:05:52.573726+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-11T08:05:53.670746+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-11T08:05:57.380272+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"587eaced-5ded-406d-884b-30c1caa8686b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":3863307356},{"type":"v1","addr":"172.21.15.167:6806","nonce":3863307356}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":3863307356},{"type":"v1","addr":"172.21.15.167:6810","nonce":3863307356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":3863307356},{"type":"v1","addr":"172.21.15.167:6815","nonce":3863307356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":3863307356},{"type":"v1","addr":"172.21.15.167:6813","nonce":3863307356}]},"public_addr":"172.21.15.167:6806/3863307356","cluster_addr":"172.21.15.167:6810/3863307356","heartbeat_back_addr":"172.21.15.167:6815/3863307356","heartbeat_front_addr":"172.21.15.167:6813/3863307356","state":["exists","up"]},{"osd":1,"uuid":"37c59104-2dd0-4cf9-85ea-56636f6b02a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":1585024048},{"type":"v1","addr":"172.21.15.167:6817","nonce":1585024048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1585024048},{"type":"v1","addr":"172.21.15.167:6819","nonce":1585024048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1585024048},{"type":"v1","addr":"172.21.15.167:6823","nonce":1585024048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":1585024048},{"type":"v1","addr":"172.21.15.167:6821","nonce":1585024048}]},"public_addr":"172.21.15.167:6817/1585024048","cluster_addr":"172.21.15.167:6819/1585024048","heartbeat_back_addr":"172.21.15.167:6823/1585024048","heartbeat_front_addr":"172.21.15.167:6821/1585024048","state":["exists","up"]},{"osd":2,"uuid":"2bb4f7f5-c698-451b-b4bb-78e716921291","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":3242166900},{"type":"v1","addr":"172.21.15.167:6801","nonce":3242166900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":3242166900},{"type":"v1","addr":"172.21.15.167:6803","nonce":3242166900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6809","nonce":3242166900},{"type":"v1","addr":"172.21.15.167:6811","nonce":3242166900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6805","nonce":3242166900},{"type":"v1","addr":"172.21.15.167:6807","nonce":3242166900}]},"public_addr":"172.21.15.167:6801/3242166900","cluster_addr":"172.21.15.167:6803/3242166900","heartbeat_back_addr":"172.21.15.167:6811/3242166900","heartbeat_front_addr":"172.21.15.167:6807/3242166900","state":["exists","up"]},{"osd":3,"uuid":"9dd26759-a338-49d2-8103-45d068b9c000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":453689293},{"type":"v1","addr":"172.21.15.167:6825","nonce":453689293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":453689293},{"type":"v1","addr":"172.21.15.167:6827","nonce":453689293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":453689293},{"type":"v1","addr":"172.21.15.167:6831","nonce":453689293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":453689293},{"type":"v1","addr":"172.21.15.167:6829","nonce":453689293}]},"public_addr":"172.21.15.167:6825/453689293","cluster_addr":"172.21.15.167:6827/453689293","heartbeat_back_addr":"172.21.15.167:6831/453689293","heartbeat_front_addr":"172.21.15.167:6829/453689293","state":["exists","up"]},{"osd":4,"uuid":"cb27250d-2a80-4c7b-8cef-3a7af05c8a16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1311831500},{"type":"v1","addr":"172.21.15.204:6801","nonce":1311831500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1311831500},{"type":"v1","addr":"172.21.15.204:6803","nonce":1311831500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1311831500},{"type":"v1","addr":"172.21.15.204:6807","nonce":1311831500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1311831500},{"type":"v1","addr":"172.21.15.204:6805","nonce":1311831500}]},"public_addr":"172.21.15.204:6801/1311831500","cluster_addr":"172.21.15.204:6803/1311831500","heartbeat_back_addr":"172.21.15.204:6807/1311831500","heartbeat_front_addr":"172.21.15.204:6805/1311831500","state":["exists","up"]},{"osd":5,"uuid":"92998541-ffec-430b-ab76-8c3cb7c2752d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2921847521},{"type":"v1","addr":"172.21.15.204:6825","nonce":2921847521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2921847521},{"type":"v1","addr":"172.21.15.204:6827","nonce":2921847521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2921847521},{"type":"v1","addr":"172.21.15.204:6831","nonce":2921847521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2921847521},{"type":"v1","addr":"172.21.15.204:6829","nonce":2921847521}]},"public_addr":"172.21.15.204:6825/2921847521","cluster_addr":"172.21.15.204:6827/2921847521","heartbeat_back_addr":"172.21.15.204:6831/2921847521","heartbeat_front_addr":"172.21.15.204:6829/2921847521","state":["exists","up"]},{"osd":6,"uuid":"9543479c-2ec5-42a8-8933-707bfed2e048","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":2889851077},{"type":"v1","addr":"172.21.15.204:6817","nonce":2889851077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":2889851077},{"type":"v1","addr":"172.21.15.204:6819","nonce":2889851077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":2889851077},{"type":"v1","addr":"172.21.15.204:6823","nonce":2889851077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":2889851077},{"type":"v1","addr":"172.21.15.204:6821","nonce":2889851077}]},"public_addr":"172.21.15.204:6817/2889851077","cluster_addr":"172.21.15.204:6819/2889851077","heartbeat_back_addr":"172.21.15.204:6823/2889851077","heartbeat_front_addr":"172.21.15.204:6821/2889851077","state":["exists","up"]},{"osd":7,"uuid":"c45a5647-8358-409e-af91-c05dda65f0b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":1949534184},{"type":"v1","addr":"172.21.15.204:6809","nonce":1949534184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":1949534184},{"type":"v1","addr":"172.21.15.204:6811","nonce":1949534184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1949534184},{"type":"v1","addr":"172.21.15.204:6815","nonce":1949534184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1949534184},{"type":"v1","addr":"172.21.15.204:6813","nonce":1949534184}]},"public_addr":"172.21.15.204:6809/1949534184","cluster_addr":"172.21.15.204:6811/1949534184","heartbeat_back_addr":"172.21.15.204:6815/1949534184","heartbeat_front_addr":"172.21.15.204:6813/1949534184","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.952253+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.940689+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.860682+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.929350+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.901627+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.957451+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.987249+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.951197+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-11T08:06:16.782 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.781+0000 7f329191c640 1 -- 172.21.15.167:0/2804565560 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f326c0762d0 msgr2=0x7f326c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:16.782 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.781+0000 7f329191c640 1 --2- 172.21.15.167:0/2804565560 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f326c0762d0 0x7f326c078790 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f327c034000 tx=0x7f327c02e000 comp rx=0 tx=0).stop 2024-05-11T08:06:16.782 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.781+0000 7f329191c640 1 -- 172.21.15.167:0/2804565560 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f328c0d2aa0 msgr2=0x7f328c0cc200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:16.782 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.781+0000 7f329191c640 1 --2- 172.21.15.167:0/2804565560 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f328c0d2aa0 0x7f328c0cc200 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f327400c7f0 tx=0x7f327400cbf0 comp rx=0 tx=0).stop 2024-05-11T08:06:16.782 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.781+0000 7f329191c640 1 -- 172.21.15.167:0/2804565560 shutdown_connections 2024-05-11T08:06:16.782 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.781+0000 7f329191c640 1 --2- 172.21.15.167:0/2804565560 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f326c0762d0 0x7f326c078790 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:16.782 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.781+0000 7f329191c640 1 --2- 172.21.15.167:0/2804565560 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f328c155b60 0x7f328c0cc740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:16.782 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.781+0000 7f329191c640 1 --2- 172.21.15.167:0/2804565560 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f328c0d2aa0 0x7f328c0cc200 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:16.783 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.781+0000 7f329191c640 1 --2- 172.21.15.167:0/2804565560 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f328c0d0b50 0x7f328c0ca2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:16.783 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.781+0000 7f329191c640 1 -- 172.21.15.167:0/2804565560 >> 172.21.15.167:0/2804565560 conn(0x7f328c0d2f80 msgr2=0x7f328c15f630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:16.783 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.781+0000 7f329191c640 1 -- 172.21.15.167:0/2804565560 shutdown_connections 2024-05-11T08:06:16.783 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.781+0000 7f329191c640 1 -- 172.21.15.167:0/2804565560 wait complete. 2024-05-11T08:06:16.793 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-05-11T08:06:16.794 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-05-11T08:06:16.794 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-05-11T08:06:16.794 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-05-11T08:06:16.795 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-05-11T08:06:16.795 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-05-11T08:06:16.796 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-05-11T08:06:16.796 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-05-11T08:06:16.969 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.968+0000 7f220cd04640 1 Processor -- start 2024-05-11T08:06:16.969 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.968+0000 7f80bbfff640 1 Processor -- start 2024-05-11T08:06:16.970 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.968+0000 7f80bbfff640 1 -- start start 2024-05-11T08:06:16.970 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.968+0000 7f220cd04640 1 -- start start 2024-05-11T08:06:16.970 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.968+0000 7fe6aded1640 1 Processor -- start 2024-05-11T08:06:16.970 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.968+0000 7fae14558640 1 Processor -- start 2024-05-11T08:06:16.971 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.967+0000 7f821db51640 1 Processor -- start 2024-05-11T08:06:16.971 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.968+0000 7f821db51640 1 -- start start 2024-05-11T08:06:16.971 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.969+0000 7f5527f91640 1 Processor -- start 2024-05-11T08:06:16.971 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.969+0000 7fae14558640 1 -- start start 2024-05-11T08:06:16.972 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.969+0000 7f124ffff640 1 Processor -- start 2024-05-11T08:06:16.972 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.969+0000 7f124ffff640 1 -- start start 2024-05-11T08:06:16.972 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.969+0000 7f821db51640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f82180b6550 0x7f82180cf7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:16.972 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.969+0000 7f821db51640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f821806ebf0 0x7f821806f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:16.972 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.969+0000 7f80bbfff640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f80b404b3b0 0x7f80b4034010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:16.973 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.969+0000 7f80bbfff640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f80b404f4c0 0x7f80b404f940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:16.973 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.969+0000 7f8217fff640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f821806ebf0 0x7f821806f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:16.973 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.969+0000 7f80bbfff640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f80b404fe80 0x7f80b40eb250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:16.974 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.969+0000 7f8217fff640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f821806ebf0 0x7f821806f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:54226/0 (socket says 172.21.15.167:54226) 2024-05-11T08:06:16.974 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.969+0000 7f8217fff640 1 -- 172.21.15.167:0/3052437780 learned_addr learned my addr 172.21.15.167:0/3052437780 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:16.974 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.969+0000 7f80bbfff640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f80b40062d0 con 0x7f80b404fe80 2024-05-11T08:06:16.974 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.969+0000 7f80bbfff640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f80b4054c00 con 0x7f80b404b3b0 2024-05-11T08:06:16.974 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.969+0000 7f80bbfff640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f80b4053a40 con 0x7f80b404f4c0 2024-05-11T08:06:16.974 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.970+0000 7f821db51640 1 -- 172.21.15.167:0/3052437780 --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f821806e460 con 0x7f821815b410 2024-05-11T08:06:16.975 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.970+0000 7f8216ffd640 1 --2- 172.21.15.167:0/3052437780 >> v2:172.21.15.204:3300/0 conn(0x7f82180b6550 0x7f82180cf7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:16.975 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.970+0000 7f80bb7fe640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f80b404fe80 0x7f80b40eb250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:16.975 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.970+0000 7f80bb7fe640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f80b404fe80 0x7f80b40eb250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:54230/0 (socket says 172.21.15.167:54230) 2024-05-11T08:06:16.975 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.970+0000 7f80bb7fe640 1 -- 172.21.15.167:0/2940300717 learned_addr learned my addr 172.21.15.167:0/2940300717 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:16.975 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.970+0000 7f80ba7fc640 1 --2- 172.21.15.167:0/2940300717 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f80b404f4c0 0x7f80b404f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:16.975 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.969+0000 7f220cd04640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f22080d08a0 0x7f22080d4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:16.976 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.969+0000 7f220cd04640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2208163bd0 con 0x7f22080b4490 2024-05-11T08:06:16.976 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.969+0000 7f220cd04640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2208163e90 con 0x7f220806ce60 2024-05-11T08:06:16.976 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.969+0000 7f220cd04640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f2208164170 con 0x7f22080d08a0 2024-05-11T08:06:16.976 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.970+0000 7f4eba8e1640 1 Processor -- start 2024-05-11T08:06:16.976 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.970+0000 7f821db51640 1 -- 172.21.15.167:0/3052437780 --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f8218072b00 con 0x7f821806ebf0 2024-05-11T08:06:16.976 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.970+0000 7f821db51640 1 -- 172.21.15.167:0/3052437780 --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f8218072da0 con 0x7f82180b6550 2024-05-11T08:06:16.976 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.970+0000 7f82167fc640 1 -- 172.21.15.167:0/3052437780 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 450958079 0 0) 0x7f821806e460 con 0x7f821815b410 2024-05-11T08:06:16.976 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.970+0000 7f82167fc640 1 -- 172.21.15.167:0/3052437780 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8200003520 con 0x7f821815b410 2024-05-11T08:06:16.977 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.970+0000 7f8217fff640 1 -- 172.21.15.167:0/3052437780 >> v2:172.21.15.204:3300/0 conn(0x7f82180b6550 msgr2=0x7f82180cf7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:16.977 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.970+0000 7f8217fff640 1 --2- 172.21.15.167:0/3052437780 >> v2:172.21.15.204:3300/0 conn(0x7f82180b6550 0x7f82180cf7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:16.977 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.970+0000 7f8217fff640 1 -- 172.21.15.167:0/3052437780 >> v1:172.21.15.204:6789/0 conn(0x7f821815b410 legacy=0x7f82180cf0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:16.977 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.971+0000 7f8217fff640 1 -- 172.21.15.167:0/3052437780 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f821806e6d0 con 0x7f821806ebf0 2024-05-11T08:06:16.977 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.970+0000 7f124ffff640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f12500cf790 0x7f12500cfc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:16.977 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.970+0000 7f124ffff640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f125006cc50 0x7f125006d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:16.977 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.970+0000 7f124ffff640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12501402d0 con 0x7f1250160ac0 2024-05-11T08:06:16.977 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.970+0000 7f124ffff640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f12501405a0 con 0x7f125006cc50 2024-05-11T08:06:16.977 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.971+0000 7f124ffff640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f12500d04c0 con 0x7f12500cf790 2024-05-11T08:06:16.978 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.970+0000 7f80bb7fe640 1 -- 172.21.15.167:0/2940300717 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f80b404f4c0 msgr2=0x7f80b404f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:16.978 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.970+0000 7f80bb7fe640 1 --2- 172.21.15.167:0/2940300717 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f80b404f4c0 0x7f80b404f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:16.978 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.970+0000 7f80bb7fe640 1 -- 172.21.15.167:0/2940300717 >> v2:172.21.15.204:3300/0 conn(0x7f80b404b3b0 msgr2=0x7f80b4034010 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-11T08:06:16.978 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.970+0000 7f80bb7fe640 1 --2- 172.21.15.167:0/2940300717 >> v2:172.21.15.204:3300/0 conn(0x7f80b404b3b0 0x7f80b4034010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:16.978 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.970+0000 7f80bb7fe640 1 -- 172.21.15.167:0/2940300717 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80b4053d20 con 0x7f80b404fe80 2024-05-11T08:06:16.978 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.970+0000 7f220659c640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f220806ce60 0x7f220806d240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.167:45632/0 (socket says 172.21.15.167:45632) 2024-05-11T08:06:16.978 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.970+0000 7f220659c640 1 -- 172.21.15.167:0/3672438570 learned_addr learned my addr 172.21.15.167:0/3672438570 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T08:06:16.978 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.970+0000 7f220559a640 1 -- 172.21.15.167:0/3672438570 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3997384137 0 0) 0x7f2208163bd0 con 0x7f22080b4490 2024-05-11T08:06:16.978 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.971+0000 7f220559a640 1 -- 172.21.15.167:0/3672438570 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f21f0003520 con 0x7f22080b4490 2024-05-11T08:06:16.979 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.971+0000 7f2205d9b640 1 --2- 172.21.15.167:0/3672438570 >> v2:172.21.15.204:3300/0 conn(0x7f22080d08a0 0x7f22080d4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:16.979 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.971+0000 7f220559a640 1 -- 172.21.15.167:0/3672438570 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2044874484 0 0) 0x7f2208163e90 con 0x7f220806ce60 2024-05-11T08:06:16.979 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.971+0000 7f220559a640 1 -- 172.21.15.167:0/3672438570 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2208163bd0 con 0x7f220806ce60 2024-05-11T08:06:16.979 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.968+0000 7fe6aded1640 1 -- start start 2024-05-11T08:06:16.979 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.970+0000 7fe6aded1640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fe6a815d120 0x7fe6a8070000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:16.979 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.970+0000 7fe6aded1640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe6a80d0100 0x7fe6a8070540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:16.979 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.970+0000 7fe6aded1640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fe6a80cd620 0x7fe6a8070a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:16.979 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.970+0000 7fe6aded1640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7fe6a80cdbe0 con 0x7fe6a80cd620 2024-05-11T08:06:16.979 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.970+0000 7fe6aded1640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fe6a80cf340 con 0x7fe6a815d120 2024-05-11T08:06:16.979 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.970+0000 7fe6aded1640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fe6a80cf620 con 0x7fe6a80d0100 2024-05-11T08:06:16.979 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.970+0000 7fe6a77fe640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fe6a815d120 0x7fe6a8070000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:16.980 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.970+0000 7fe6a7fff640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fe6a80cd620 0x7fe6a8070a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:16.980 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.970+0000 7fe6a7fff640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fe6a80cd620 0x7fe6a8070a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:54240/0 (socket says 172.21.15.167:54240) 2024-05-11T08:06:16.980 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.970+0000 7fe6a77fe640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fe6a815d120 0x7fe6a8070000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.167:37048/0 (socket says 172.21.15.167:37048) 2024-05-11T08:06:16.980 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.970+0000 7fe6a7fff640 1 -- 172.21.15.167:0/2058635215 learned_addr learned my addr 172.21.15.167:0/2058635215 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:16.980 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.970+0000 7fe6a6ffd640 1 --2- 172.21.15.167:0/2058635215 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe6a80d0100 0x7fe6a8070540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:16.980 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.971+0000 7fe6a77fe640 1 -- 172.21.15.167:0/2058635215 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe6a80d0100 msgr2=0x7fe6a8070540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:16.980 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.971+0000 7fe6a77fe640 1 --2- 172.21.15.167:0/2058635215 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe6a80d0100 0x7fe6a8070540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:16.980 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.971+0000 7fe6a77fe640 1 -- 172.21.15.167:0/2058635215 >> v2:172.21.15.167:3300/0 conn(0x7fe6a80cd620 msgr2=0x7fe6a8070a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:16.980 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.971+0000 7fe6a77fe640 1 --2- 172.21.15.167:0/2058635215 >> v2:172.21.15.167:3300/0 conn(0x7fe6a80cd620 0x7fe6a8070a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:16.980 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.971+0000 7fe6a77fe640 1 -- 172.21.15.167:0/2058635215 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6a80cf900 con 0x7fe6a815d120 2024-05-11T08:06:16.980 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.971+0000 7f220559a640 1 -- 172.21.15.167:0/3672438570 <== mon.0 v1:172.21.15.167:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 872006514 0 0) 0x7f21f0003520 con 0x7f22080b4490 2024-05-11T08:06:16.980 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.971+0000 7f220559a640 1 -- 172.21.15.167:0/3672438570 >> v2:172.21.15.204:3300/0 conn(0x7f22080d08a0 msgr2=0x7f22080d4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:16.981 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.971+0000 7f220559a640 1 --2- 172.21.15.167:0/3672438570 >> v2:172.21.15.204:3300/0 conn(0x7f22080d08a0 0x7f22080d4580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:16.981 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.971+0000 7f220559a640 1 -- 172.21.15.167:0/3672438570 >> v1:172.21.15.204:6789/0 conn(0x7f220806ce60 legacy=0x7f220806d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:16.981 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.971+0000 7f80bb7fe640 1 --2- 172.21.15.167:0/2940300717 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f80b404fe80 0x7f80b40eb250 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f80bc06b120 tx=0x7f80bc06a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2ee2da5fa37d84d server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:16.981 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.971+0000 7f220559a640 1 -- 172.21.15.167:0/3672438570 --> v1:172.21.15.167:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2208164450 con 0x7f22080b4490 2024-05-11T08:06:16.981 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.970+0000 7f4eba8e1640 1 -- start start 2024-05-11T08:06:16.982 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.971+0000 7f80b9ffb640 1 -- 172.21.15.167:0/2940300717 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80bc0b2070 con 0x7f80b404fe80 2024-05-11T08:06:16.982 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.971+0000 7f220559a640 1 -- 172.21.15.167:0/3672438570 <== mon.0 v1:172.21.15.167:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3602277846 0 0) 0x7f2200001a70 con 0x7f22080b4490 2024-05-11T08:06:16.982 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.971+0000 7f2205d9b640 1 --2- 172.21.15.167:0/3672438570 >> v2:172.21.15.204:3300/0 conn(0x7f22080d08a0 0x7f22080d4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T08:06:16.982 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.970+0000 7fae14558640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fae0c0ce6a0 0x7fae0c1672a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:16.983 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.972+0000 7f4eba8e1640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f4eac0e30b0 0x7f4eac057d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:16.983 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.972+0000 7f4eba8e1640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4eac03e830 0x7f4eac053be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:16.983 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.972+0000 7f220559a640 1 -- 172.21.15.167:0/3672438570 >> v1:172.21.15.167:6789/0 conn(0x7f22080b4490 legacy=0x7f220816d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:16.983 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.970+0000 7fae122f4640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7fae0c0ce2c0 0x7fae0c0cf620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:60766/0 (socket says 172.21.15.167:60766) 2024-05-11T08:06:16.983 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.972+0000 7f220559a640 1 --2- 172.21.15.167:0/3672438570 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f21f0003cb0 0x7f21f0004090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:16.984 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.970+0000 7fae122f4640 1 -- 172.21.15.167:0/3585353483 learned_addr learned my addr 172.21.15.167:0/3585353483 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T08:06:16.984 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.972+0000 7f220559a640 1 --2- 172.21.15.167:0/3672438570 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f21f0004800 0x7f21f0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:16.984 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.972+0000 7f4eba8e1640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4eac0582c0 con 0x7f4eac054120 2024-05-11T08:06:16.985 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.970+0000 7fae14558640 1 -- 172.21.15.167:0/3585353483 --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae0c168080 con 0x7fae0c0ce2c0 2024-05-11T08:06:16.985 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.972+0000 7f220559a640 1 --2- 172.21.15.167:0/3672438570 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f21f0005280 0x7f21f000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:16.985 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.972+0000 7f4eb3fff640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f4eac0e30b0 0x7f4eac057d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:16.985 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.972+0000 7f4eb3fff640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f4eac0e30b0 0x7f4eac057d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.167:37082/0 (socket says 172.21.15.167:37082) 2024-05-11T08:06:16.985 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.972+0000 7f4eb3fff640 1 -- 172.21.15.167:0/2171148724 learned_addr learned my addr 172.21.15.167:0/2171148724 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:16.986 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.972+0000 7f220659c640 1 --2- 172.21.15.167:0/3672438570 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f21f0003cb0 0x7f21f0004090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:16.986 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.972+0000 7f4eba8e1640 1 -- 172.21.15.167:0/2171148724 --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f4eac058560 con 0x7f4eac0e30b0 2024-05-11T08:06:16.986 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.972+0000 7f4eba8e1640 1 -- 172.21.15.167:0/2171148724 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f4eac058840 con 0x7f4eac03e830 2024-05-11T08:06:16.986 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.972+0000 7f2205d9b640 1 --2- 172.21.15.167:0/3672438570 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f21f0004800 0x7f21f0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:16.987 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.971+0000 7fae14558640 1 -- 172.21.15.167:0/3585353483 --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae0c0d0190 con 0x7fae0c1677e0 2024-05-11T08:06:16.987 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.971+0000 7fae14558640 1 -- 172.21.15.167:0/3585353483 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fae0c0b4ed0 con 0x7fae0c0ce6a0 2024-05-11T08:06:16.987 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.971+0000 7fae112f2640 1 -- 172.21.15.167:0/3585353483 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1596514215 0 0) 0x7fae0c168080 con 0x7fae0c0ce2c0 2024-05-11T08:06:16.987 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.972+0000 7f8217fff640 1 --2- 172.21.15.167:0/3052437780 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f821806ebf0 0x7f821806f0a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f82180701f0 tx=0x7f821002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a086012af957c7a1 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:16.987 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.972+0000 7f4eb37fe640 1 --2- 172.21.15.167:0/2171148724 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4eac03e830 0x7f4eac053be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:16.988 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.971+0000 7fae112f2640 1 -- 172.21.15.167:0/3585353483 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fadf8003520 con 0x7fae0c0ce2c0 2024-05-11T08:06:16.988 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.972+0000 7fae112f2640 1 -- 172.21.15.167:0/3585353483 <== mon.0 v1:172.21.15.167:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1010077699 0 0) 0x7fadf8003520 con 0x7fae0c0ce2c0 2024-05-11T08:06:16.988 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.972+0000 7fae112f2640 1 -- 172.21.15.167:0/3585353483 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fae0c0ce6a0 msgr2=0x7fae0c1672a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-11T08:06:16.988 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.972+0000 7fae112f2640 1 --2- 172.21.15.167:0/3585353483 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fae0c0ce6a0 0x7fae0c1672a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:16.988 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.972+0000 7fae112f2640 1 -- 172.21.15.167:0/3585353483 >> v1:172.21.15.204:6789/0 conn(0x7fae0c1677e0 legacy=0x7fae0c0b5360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:16.988 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.972+0000 7fae112f2640 1 -- 172.21.15.167:0/3585353483 --> v1:172.21.15.167:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae0c0d0470 con 0x7fae0c0ce2c0 2024-05-11T08:06:16.988 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.972+0000 7fae112f2640 1 -- 172.21.15.167:0/3585353483 <== mon.0 v1:172.21.15.167:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3602277846 0 0) 0x7fae080016f0 con 0x7fae0c0ce2c0 2024-05-11T08:06:16.988 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.972+0000 7fae112f2640 1 -- 172.21.15.167:0/3585353483 >> v1:172.21.15.167:6789/0 conn(0x7fae0c0ce2c0 legacy=0x7fae0c0cf620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:16.988 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.972+0000 7fae112f2640 1 --2- 172.21.15.167:0/3585353483 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fadf8003a30 0x7fadf8003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:16.988 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.972+0000 7fae112f2640 1 --2- 172.21.15.167:0/3585353483 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fadf8004800 0x7fadf8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:16.989 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.972+0000 7fae112f2640 1 --2- 172.21.15.167:0/3585353483 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fadf80053c0 0x7fadf800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:16.989 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.972+0000 7fae112f2640 1 -- 172.21.15.167:0/3585353483 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fae0c0b4ed0 con 0x7fadf8004800 2024-05-11T08:06:16.989 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.972+0000 7fae112f2640 1 -- 172.21.15.167:0/3585353483 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fadf800c230 con 0x7fadf8003a30 2024-05-11T08:06:16.989 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.972+0000 7fae112f2640 1 -- 172.21.15.167:0/3585353483 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fadf800c430 con 0x7fadf80053c0 2024-05-11T08:06:16.989 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.972+0000 7f80b9ffb640 1 -- 172.21.15.167:0/2940300717 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80bc0ab7d0 con 0x7f80b404fe80 2024-05-11T08:06:16.989 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.972+0000 7f80b9ffb640 1 -- 172.21.15.167:0/2940300717 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80bc0a6e60 con 0x7f80b404fe80 2024-05-11T08:06:16.989 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.972+0000 7f220559a640 1 -- 172.21.15.167:0/3672438570 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2208164170 con 0x7f21f0005280 2024-05-11T08:06:16.990 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.972+0000 7f220559a640 1 -- 172.21.15.167:0/3672438570 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f21f000c190 con 0x7f21f0004800 2024-05-11T08:06:16.990 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.972+0000 7f220559a640 1 -- 172.21.15.167:0/3672438570 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f21f000c390 con 0x7f21f0003cb0 2024-05-11T08:06:16.990 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.972+0000 7fe6a77fe640 1 --2- 172.21.15.167:0/2058635215 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe6a815d120 0x7fe6a8070000 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fe6a8074080 tx=0x7fe6980328b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d38eb85aaf9b0327 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:16.990 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.972+0000 7f2206d9d640 1 --2- 172.21.15.167:0/3672438570 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f21f0005280 0x7f21f000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:16.991 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.972+0000 7fe6a67fc640 1 -- 172.21.15.167:0/2058635215 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe698039640 con 0x7fe6a815d120 2024-05-11T08:06:16.991 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.971+0000 7f5527f91640 1 -- start start 2024-05-11T08:06:16.991 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.972+0000 7f5527f91640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f55201462f0 0x7f55200b6d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:16.991 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.972+0000 7f5527f91640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f55200d3590 0x7f55200d1790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:16.991 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.972+0000 7f5527f91640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55200d1290 con 0x7f5520161930 2024-05-11T08:06:16.991 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.972+0000 7f5527f91640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f55200b7740 con 0x7f55201462f0 2024-05-11T08:06:16.991 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.972+0000 7f5527f91640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f55200b7a20 con 0x7f55200d3590 2024-05-11T08:06:16.992 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.973+0000 7fe6a67fc640 1 -- 172.21.15.167:0/2058635215 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe69803e070 con 0x7fe6a815d120 2024-05-11T08:06:16.992 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.973+0000 7fe6a67fc640 1 -- 172.21.15.167:0/2058635215 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe698039a00 con 0x7fe6a815d120 2024-05-11T08:06:16.992 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.973+0000 7fae11af3640 1 --2- 172.21.15.167:0/3585353483 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fadf8004800 0x7fadf8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:16.992 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.973+0000 7fae122f4640 1 --2- 172.21.15.167:0/3585353483 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fadf8003a30 0x7fadf8003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:16.992 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.973+0000 7fae12af5640 1 --2- 172.21.15.167:0/3585353483 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fadf80053c0 0x7fadf800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:16.992 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.973+0000 7fae11af3640 1 -- 172.21.15.167:0/3585353483 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fadf80053c0 msgr2=0x7fadf800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:16.993 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.973+0000 7fae11af3640 1 --2- 172.21.15.167:0/3585353483 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fadf80053c0 0x7fadf800b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:16.993 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.973+0000 7fae11af3640 1 -- 172.21.15.167:0/3585353483 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fadf8003a30 msgr2=0x7fadf8003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:16.993 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.973+0000 7fae11af3640 1 --2- 172.21.15.167:0/3585353483 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fadf8003a30 0x7fadf8003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:16.993 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.973+0000 7fae11af3640 1 -- 172.21.15.167:0/3585353483 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae0c0d0470 con 0x7fadf8004800 2024-05-11T08:06:16.993 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.971+0000 7f1247fff640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f12500cf790 0x7f12500cfc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:16.993 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.971+0000 7f1247fff640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f12500cf790 0x7f12500cfc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.167:37070/0 (socket says 172.21.15.167:37070) 2024-05-11T08:06:16.993 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.971+0000 7f1247fff640 1 -- 172.21.15.167:0/2968238466 learned_addr learned my addr 172.21.15.167:0/2968238466 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:16.993 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.972+0000 7f1247fff640 1 -- 172.21.15.167:0/2968238466 >> v2:172.21.15.167:3300/0 conn(0x7f125006cc50 msgr2=0x7f125006d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:16.993 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.972+0000 7f1247fff640 1 --2- 172.21.15.167:0/2968238466 >> v2:172.21.15.167:3300/0 conn(0x7f125006cc50 0x7f125006d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:16.993 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.972+0000 7f1247fff640 1 -- 172.21.15.167:0/2968238466 >> v1:172.21.15.167:6789/0 conn(0x7f1250160ac0 legacy=0x7f1250160ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:16.993 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.972+0000 7f1247fff640 1 -- 172.21.15.167:0/2968238466 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12500d07a0 con 0x7f12500cf790 2024-05-11T08:06:16.994 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.973+0000 7f552652e640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f55200d3590 0x7f55200d1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:16.994 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.973+0000 7f552652e640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f55200d3590 0x7f55200d1790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.167:37152/0 (socket says 172.21.15.167:37152) 2024-05-11T08:06:16.994 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.973+0000 7f552652e640 1 -- 172.21.15.167:0/2924733871 learned_addr learned my addr 172.21.15.167:0/2924733871 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:16.994 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.972+0000 7f82167fc640 1 -- 172.21.15.167:0/3052437780 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8210036070 con 0x7f821806ebf0 2024-05-11T08:06:16.994 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.972+0000 7f82167fc640 1 -- 172.21.15.167:0/3052437780 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8210031600 con 0x7f821806ebf0 2024-05-11T08:06:16.994 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.972+0000 7f82167fc640 1 -- 172.21.15.167:0/3052437780 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82100319d0 con 0x7f821806ebf0 2024-05-11T08:06:16.994 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.973+0000 7f821db51640 1 -- 172.21.15.167:0/3052437780 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f821806ebf0 msgr2=0x7f821806f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:16.994 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.973+0000 7f821db51640 1 --2- 172.21.15.167:0/3052437780 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f821806ebf0 0x7f821806f0a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f82180701f0 tx=0x7f821002a6a0 comp rx=0 tx=0).stop 2024-05-11T08:06:16.995 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.973+0000 7f80bbfff640 1 -- 172.21.15.167:0/2940300717 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f80b404fe80 msgr2=0x7f80b40eb250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:16.995 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.973+0000 7f80bbfff640 1 --2- 172.21.15.167:0/2940300717 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f80b404fe80 0x7f80b40eb250 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f80bc06b120 tx=0x7f80bc06a230 comp rx=0 tx=0).stop 2024-05-11T08:06:16.995 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.973+0000 7f80bbfff640 1 -- 172.21.15.167:0/2940300717 shutdown_connections 2024-05-11T08:06:16.995 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.973+0000 7f80bbfff640 1 --2- 172.21.15.167:0/2940300717 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f80b404fe80 0x7f80b40eb250 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:16.995 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.973+0000 7f80bbfff640 1 --2- 172.21.15.167:0/2940300717 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f80b404f4c0 0x7f80b404f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:16.995 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.973+0000 7f80bbfff640 1 --2- 172.21.15.167:0/2940300717 >> v2:172.21.15.204:3300/0 conn(0x7f80b404b3b0 0x7f80b4034010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:16.995 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.973+0000 7f80bbfff640 1 -- 172.21.15.167:0/2940300717 >> 172.21.15.167:0/2940300717 conn(0x7f80b400a3f0 msgr2=0x7f80b400a7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:16.995 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.973+0000 7f4eb37fe640 1 -- 172.21.15.167:0/2171148724 >> v2:172.21.15.204:3300/0 conn(0x7f4eac0e30b0 msgr2=0x7f4eac057d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:16.995 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.973+0000 7f4eb37fe640 1 --2- 172.21.15.167:0/2171148724 >> v2:172.21.15.204:3300/0 conn(0x7f4eac0e30b0 0x7f4eac057d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:16.996 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.973+0000 7f4eb37fe640 1 -- 172.21.15.167:0/2171148724 >> v1:172.21.15.204:6789/0 conn(0x7f4eac054120 legacy=0x7f4eac0e6c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:16.996 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.973+0000 7f4eb37fe640 1 -- 172.21.15.167:0/2171148724 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4eac058b20 con 0x7f4eac03e830 2024-05-11T08:06:16.996 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.973+0000 7f4eb3fff640 1 --2- 172.21.15.167:0/2171148724 >> v2:172.21.15.204:3300/0 conn(0x7f4eac0e30b0 0x7f4eac057d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T08:06:16.996 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.974+0000 7f1247fff640 1 --2- 172.21.15.167:0/2968238466 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f12500cf790 0x7f12500cfc00 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f125006f620 tx=0x7f123802a1e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca52e5fee7ab58bc server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:16.996 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.973+0000 7f5525d2d640 1 --2- 172.21.15.167:0/2924733871 >> v2:172.21.15.204:3300/0 conn(0x7f55201462f0 0x7f55200b6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:16.996 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.973+0000 7f5525d2d640 1 -- 172.21.15.167:0/2924733871 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f55200d3590 msgr2=0x7f55200d1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:16.996 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.973+0000 7f5525d2d640 1 --2- 172.21.15.167:0/2924733871 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f55200d3590 0x7f55200d1790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:16.996 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.973+0000 7f5525d2d640 1 -- 172.21.15.167:0/2924733871 >> v1:172.21.15.167:6789/0 conn(0x7f5520161930 legacy=0x7f55200d2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:16.996 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.973+0000 7f5525d2d640 1 -- 172.21.15.167:0/2924733871 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55200b7d00 con 0x7f55201462f0 2024-05-11T08:06:16.997 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.973+0000 7f80bbfff640 1 -- 172.21.15.167:0/2940300717 shutdown_connections 2024-05-11T08:06:16.997 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.973+0000 7f220659c640 1 -- 172.21.15.167:0/3672438570 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f21f0004800 msgr2=0x7f21f0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:16.997 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.974+0000 7f4eb37fe640 1 --2- 172.21.15.167:0/2171148724 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4eac03e830 0x7f4eac053be0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f4ea00097a0 tx=0x7f4ea002a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4eb04faaf7bcdfbe server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:16.998 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.974+0000 7f821db51640 1 -- 172.21.15.167:0/3052437780 shutdown_connections 2024-05-11T08:06:16.998 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.974+0000 7f821db51640 1 --2- 172.21.15.167:0/3052437780 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f821806ebf0 0x7f821806f0a0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:16.998 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.974+0000 7f821db51640 1 --2- 172.21.15.167:0/3052437780 >> v2:172.21.15.204:3300/0 conn(0x7f82180b6550 0x7f82180cf7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:16.998 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.974+0000 7f821db51640 1 -- 172.21.15.167:0/3052437780 >> 172.21.15.167:0/3052437780 conn(0x7f821806c450 msgr2=0x7f821806c860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:16.998 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.974+0000 7f821db51640 1 -- 172.21.15.167:0/3052437780 shutdown_connections 2024-05-11T08:06:16.998 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.974+0000 7f821db51640 1 -- 172.21.15.167:0/3052437780 wait complete. 2024-05-11T08:06:16.998 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.974+0000 7fe6aded1640 1 -- 172.21.15.167:0/2058635215 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe6a815d120 msgr2=0x7fe6a8070000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:16.999 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.974+0000 7fe6aded1640 1 --2- 172.21.15.167:0/2058635215 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe6a815d120 0x7fe6a8070000 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fe6a8074080 tx=0x7fe6980328b0 comp rx=0 tx=0).stop 2024-05-11T08:06:16.999 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.974+0000 7fe6aded1640 1 -- 172.21.15.167:0/2058635215 shutdown_connections 2024-05-11T08:06:16.999 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.974+0000 7fe6aded1640 1 --2- 172.21.15.167:0/2058635215 >> v2:172.21.15.167:3300/0 conn(0x7fe6a80cd620 0x7fe6a8070a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:16.999 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.974+0000 7fe6aded1640 1 --2- 172.21.15.167:0/2058635215 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe6a80d0100 0x7fe6a8070540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:16.999 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.974+0000 7fe6aded1640 1 --2- 172.21.15.167:0/2058635215 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe6a815d120 0x7fe6a8070000 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:16.999 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.974+0000 7fe6aded1640 1 -- 172.21.15.167:0/2058635215 >> 172.21.15.167:0/2058635215 conn(0x7fe6a80d2fd0 msgr2=0x7fe6a80d33b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:16.999 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.974+0000 7f4eb2ffd640 1 -- 172.21.15.167:0/2171148724 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea0037070 con 0x7f4eac03e830 2024-05-11T08:06:16.999 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.974+0000 7f4eb2ffd640 1 -- 172.21.15.167:0/2171148724 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ea0032440 con 0x7f4eac03e830 2024-05-11T08:06:16.999 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.974+0000 7f4eb2ffd640 1 -- 172.21.15.167:0/2171148724 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea0032810 con 0x7f4eac03e830 2024-05-11T08:06:17.000 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.974+0000 7f124e7fc640 1 -- 172.21.15.167:0/2968238466 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1238037070 con 0x7f12500cf790 2024-05-11T08:06:17.000 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.974+0000 7f220659c640 1 --2- 172.21.15.167:0/3672438570 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f21f0004800 0x7f21f0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:17.000 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.974+0000 7f220659c640 1 -- 172.21.15.167:0/3672438570 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f21f0005280 msgr2=0x7f21f000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:17.000 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.974+0000 7f220659c640 1 --2- 172.21.15.167:0/3672438570 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f21f0005280 0x7f21f000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:17.000 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.974+0000 7f220659c640 1 -- 172.21.15.167:0/3672438570 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21f0003520 con 0x7f21f0003cb0 2024-05-11T08:06:17.000 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.974+0000 7f2206d9d640 1 --2- 172.21.15.167:0/3672438570 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f21f0005280 0x7f21f000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T08:06:17.000 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.974+0000 7f2206d9d640 1 -- 172.21.15.167:0/3672438570 reap_dead start 2024-05-11T08:06:17.001 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.974+0000 7f124e7fc640 1 -- 172.21.15.167:0/2968238466 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1238032440 con 0x7f12500cf790 2024-05-11T08:06:17.001 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.974+0000 7f124e7fc640 1 -- 172.21.15.167:0/2968238466 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1238037070 con 0x7f12500cf790 2024-05-11T08:06:17.001 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.974+0000 7f80bbfff640 1 -- 172.21.15.167:0/2940300717 wait complete. 2024-05-11T08:06:17.001 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.974+0000 7fe6aded1640 1 -- 172.21.15.167:0/2058635215 shutdown_connections 2024-05-11T08:06:17.001 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.974+0000 7fe6aded1640 1 -- 172.21.15.167:0/2058635215 wait complete. 2024-05-11T08:06:17.002 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.974+0000 7f5525d2d640 1 --2- 172.21.15.167:0/2924733871 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f55201462f0 0x7f55200b6d40 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f551c0097a0 tx=0x7f551c02a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=39934655019091ed server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:17.002 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.974+0000 7fae11af3640 1 --2- 172.21.15.167:0/3585353483 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fadf8004800 0x7fadf8004c80 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fae0c0b74e0 tx=0x7fadf8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cf7ee64feaa60a0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:17.002 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.975+0000 7f5524d2b640 1 -- 172.21.15.167:0/2924733871 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f551c037070 con 0x7f55201462f0 2024-05-11T08:06:17.003 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.975+0000 7f5524d2b640 1 -- 172.21.15.167:0/2924733871 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f551c032440 con 0x7f55201462f0 2024-05-11T08:06:17.003 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.975+0000 7f5524d2b640 1 -- 172.21.15.167:0/2924733871 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f551c032810 con 0x7f55201462f0 2024-05-11T08:06:17.003 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.975+0000 7fae112f2640 1 -- 172.21.15.167:0/3585353483 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadfc037070 con 0x7fadf8004800 2024-05-11T08:06:17.003 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.975+0000 7fae112f2640 1 -- 172.21.15.167:0/3585353483 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fadfc02a930 con 0x7fadf8004800 2024-05-11T08:06:17.003 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.975+0000 7fae112f2640 1 -- 172.21.15.167:0/3585353483 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadfc0093d0 con 0x7fadf8004800 2024-05-11T08:06:17.003 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.975+0000 7f124ffff640 1 -- 172.21.15.167:0/2968238466 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f12500cf790 msgr2=0x7f12500cfc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:17.004 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.975+0000 7f124ffff640 1 --2- 172.21.15.167:0/2968238466 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f12500cf790 0x7f12500cfc00 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f125006f620 tx=0x7f123802a1e0 comp rx=0 tx=0).stop 2024-05-11T08:06:17.004 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.975+0000 7f4eba8e1640 1 -- 172.21.15.167:0/2171148724 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4eac03e830 msgr2=0x7f4eac053be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:17.004 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.975+0000 7f4eba8e1640 1 --2- 172.21.15.167:0/2171148724 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4eac03e830 0x7f4eac053be0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f4ea00097a0 tx=0x7f4ea002a180 comp rx=0 tx=0).stop 2024-05-11T08:06:17.005 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.975+0000 7f124ffff640 1 -- 172.21.15.167:0/2968238466 shutdown_connections 2024-05-11T08:06:17.005 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.975+0000 7f124ffff640 1 --2- 172.21.15.167:0/2968238466 >> v2:172.21.15.167:3300/0 conn(0x7f125006cc50 0x7f125006d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:17.005 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.975+0000 7f124ffff640 1 --2- 172.21.15.167:0/2968238466 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f12500cf790 0x7f12500cfc00 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:17.005 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.975+0000 7f4eba8e1640 1 -- 172.21.15.167:0/2171148724 shutdown_connections 2024-05-11T08:06:17.005 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.975+0000 7f4eba8e1640 1 --2- 172.21.15.167:0/2171148724 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4eac03e830 0x7f4eac053be0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:17.005 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.975+0000 7f4eba8e1640 1 --2- 172.21.15.167:0/2171148724 >> v2:172.21.15.204:3300/0 conn(0x7f4eac0e30b0 0x7f4eac057d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:17.005 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.975+0000 7f4eba8e1640 1 -- 172.21.15.167:0/2171148724 >> 172.21.15.167:0/2171148724 conn(0x7f4eac0559d0 msgr2=0x7f4eac055de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:17.006 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.975+0000 7f124ffff640 1 -- 172.21.15.167:0/2968238466 >> 172.21.15.167:0/2968238466 conn(0x7f12500d1010 msgr2=0x7f12500d1420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:17.006 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.975+0000 7f124ffff640 1 -- 172.21.15.167:0/2968238466 shutdown_connections 2024-05-11T08:06:17.006 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.975+0000 7f124ffff640 1 -- 172.21.15.167:0/2968238466 wait complete. 2024-05-11T08:06:17.006 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.976+0000 7fe6aded1640 1 Processor -- start 2024-05-11T08:06:17.006 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.976+0000 7fae14558640 1 -- 172.21.15.167:0/3585353483 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fadf8004800 msgr2=0x7fadf8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:17.007 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.976+0000 7fae14558640 1 --2- 172.21.15.167:0/3585353483 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fadf8004800 0x7fadf8004c80 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fae0c0b74e0 tx=0x7fadf8005830 comp rx=0 tx=0).stop 2024-05-11T08:06:17.007 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.976+0000 7f821db51640 1 Processor -- start 2024-05-11T08:06:17.007 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.976+0000 7f5527f91640 1 -- 172.21.15.167:0/2924733871 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f55201462f0 msgr2=0x7f55200b6d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:17.007 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.976+0000 7f5527f91640 1 --2- 172.21.15.167:0/2924733871 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f55201462f0 0x7f55200b6d40 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f551c0097a0 tx=0x7f551c02a180 comp rx=0 tx=0).stop 2024-05-11T08:06:17.007 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.976+0000 7fae12af5640 1 -- 172.21.15.167:0/3585353483 reap_dead start 2024-05-11T08:06:17.009 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.976+0000 7f4eba8e1640 1 -- 172.21.15.167:0/2171148724 shutdown_connections 2024-05-11T08:06:17.009 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.976+0000 7f5527f91640 1 -- 172.21.15.167:0/2924733871 shutdown_connections 2024-05-11T08:06:17.009 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.976+0000 7f5527f91640 1 --2- 172.21.15.167:0/2924733871 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f55200d3590 0x7f55200d1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:17.009 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.976+0000 7f5527f91640 1 --2- 172.21.15.167:0/2924733871 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f55201462f0 0x7f55200b6d40 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:17.009 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.976+0000 7f5527f91640 1 -- 172.21.15.167:0/2924733871 >> 172.21.15.167:0/2924733871 conn(0x7f55200b8300 msgr2=0x7f55200b86e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:17.010 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.976+0000 7fe6aded1640 1 -- start start 2024-05-11T08:06:17.010 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.976+0000 7fae14558640 1 -- 172.21.15.167:0/3585353483 shutdown_connections 2024-05-11T08:06:17.010 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.976+0000 7fae14558640 1 -- 172.21.15.167:0/3585353483 >> 172.21.15.167:0/3585353483 conn(0x7fae0c0d2010 msgr2=0x7fae0c0d2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:17.010 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.976+0000 7f821db51640 1 -- start start 2024-05-11T08:06:17.011 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.976+0000 7f220659c640 1 --2- 172.21.15.167:0/3672438570 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f21f0003cb0 0x7f21f0004090 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f21fc0017b0 tx=0x7f21f0005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c03a0c444511ad52 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:17.011 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.976+0000 7f5527f91640 1 -- 172.21.15.167:0/2924733871 shutdown_connections 2024-05-11T08:06:17.011 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.976+0000 7f5527f91640 1 -- 172.21.15.167:0/2924733871 wait complete. 2024-05-11T08:06:17.011 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.976+0000 7f4eba8e1640 1 -- 172.21.15.167:0/2171148724 wait complete. 2024-05-11T08:06:17.012 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.976+0000 7fae14558640 1 -- 172.21.15.167:0/3585353483 shutdown_connections 2024-05-11T08:06:17.012 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.976+0000 7f80bbfff640 1 Processor -- start 2024-05-11T08:06:17.012 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.976+0000 7f80bbfff640 1 -- start start 2024-05-11T08:06:17.012 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.977+0000 7fae14558640 1 -- 172.21.15.167:0/3585353483 wait complete. 2024-05-11T08:06:17.012 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.977+0000 7f124ffff640 1 Processor -- start 2024-05-11T08:06:17.013 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.977+0000 7f821db51640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f821806ebf0 0x7f821815b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:17.013 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.977+0000 7fe6aded1640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe6a80cd620 0x7fe6a815f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:17.013 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.977+0000 7fe6aded1640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe6a80d0100 0x7fe6a815d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:17.013 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.977+0000 7f821db51640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82180b6550 0x7f821814e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:17.014 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.977+0000 7f220559a640 1 -- 172.21.15.167:0/3672438570 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21fc007b60 con 0x7f21f0003cb0 2024-05-11T08:06:17.014 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.978+0000 7f220559a640 1 -- 172.21.15.167:0/3672438570 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21fc004b80 con 0x7f21f0003cb0 2024-05-11T08:06:17.014 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.978+0000 7f82177fe640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f821806ebf0 0x7f821815b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:17.014 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.978+0000 7f82177fe640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f821806ebf0 0x7f821815b7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.167:37162/0 (socket says 172.21.15.167:37162) 2024-05-11T08:06:17.014 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.978+0000 7f82177fe640 1 -- 172.21.15.167:0/2608700135 learned_addr learned my addr 172.21.15.167:0/2608700135 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:17.015 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.978+0000 7f8216ffd640 1 --2- 172.21.15.167:0/2608700135 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82180b6550 0x7f821814e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:17.015 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.978+0000 7f4eba8e1640 1 Processor -- start 2024-05-11T08:06:17.015 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.978+0000 7f80bbfff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f80b404b3b0 0x7f80b40e4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:17.015 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.978+0000 7f5527f91640 1 Processor -- start 2024-05-11T08:06:17.016 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.978+0000 7f80bbfff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f80b404f4c0 0x7f80b4036d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:17.016 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.978+0000 7f821db51640 1 --2- 172.21.15.167:0/2608700135 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f821815b410 0x7f821814f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:17.016 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.978+0000 7f80bbfff640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f80b404fe80 0x7f80b4035f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:17.016 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.978+0000 7f80bbfff640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f80b4054820 con 0x7f80b404f4c0 2024-05-11T08:06:17.017 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.978+0000 7f4eba8e1640 1 -- start start 2024-05-11T08:06:17.017 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.978+0000 7f80bbfff640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f80b404b790 con 0x7f80b404b3b0 2024-05-11T08:06:17.017 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.978+0000 7f80bbfff640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f80b40549a0 con 0x7f80b404fe80 2024-05-11T08:06:17.018 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.978+0000 7f5527f91640 1 -- start start 2024-05-11T08:06:17.018 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.978+0000 7f80ba7fc640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f80b404f4c0 0x7f80b4036d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:17.018 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.978+0000 7f80ba7fc640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f80b404f4c0 0x7f80b4036d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:54278/0 (socket says 172.21.15.167:54278) 2024-05-11T08:06:17.018 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.978+0000 7f80ba7fc640 1 -- 172.21.15.167:0/660165834 learned_addr learned my addr 172.21.15.167:0/660165834 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:17.018 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.977+0000 7f124ffff640 1 -- start start 2024-05-11T08:06:17.019 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.978+0000 7f124ffff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f125006cc50 0x7f125013f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:17.019 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.979+0000 7fae14558640 1 Processor -- start 2024-05-11T08:06:17.019 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.978+0000 7f124ffff640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f12500cf790 0x7f12501433e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:17.019 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.978+0000 7f124ffff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1250160ac0 0x7f125006e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:17.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.978+0000 7f124ffff640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f125006c730 con 0x7f125006cc50 2024-05-11T08:06:17.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.979+0000 7fae14558640 1 -- start start 2024-05-11T08:06:17.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.978+0000 7f124ffff640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f125006c450 con 0x7f1250160ac0 2024-05-11T08:06:17.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.978+0000 7f124ffff640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f12500d0140 con 0x7f12500cf790 2024-05-11T08:06:17.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.979+0000 7f220cd04640 1 -- 172.21.15.167:0/3672438570 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f21f0003cb0 msgr2=0x7f21f0004090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:17.021 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.979+0000 7f220cd04640 1 --2- 172.21.15.167:0/3672438570 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f21f0003cb0 0x7f21f0004090 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f21fc0017b0 tx=0x7f21f0005830 comp rx=0 tx=0).stop 2024-05-11T08:06:17.021 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.979+0000 7f1247fff640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f12500cf790 0x7f12501433e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:17.021 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.979+0000 7f80ba7fc640 1 -- 172.21.15.167:0/660165834 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f80b404fe80 msgr2=0x7f80b4035f50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-11T08:06:17.021 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.979+0000 7f80ba7fc640 1 --2- 172.21.15.167:0/660165834 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f80b404fe80 0x7f80b4035f50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:17.021 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.979+0000 7f80ba7fc640 1 -- 172.21.15.167:0/660165834 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f80b404b3b0 msgr2=0x7f80b40e4f50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-11T08:06:17.021 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.979+0000 7f80ba7fc640 1 --2- 172.21.15.167:0/660165834 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f80b404b3b0 0x7f80b40e4f50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:17.022 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.979+0000 7f80ba7fc640 1 -- 172.21.15.167:0/660165834 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80b404d750 con 0x7f80b404f4c0 2024-05-11T08:06:17.022 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.978+0000 7fe6aded1640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe6a815d120 0x7fe6a806fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:17.022 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.978+0000 7fe6aded1640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a80ceff0 con 0x7fe6a80d0100 2024-05-11T08:06:17.023 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.978+0000 7fe6aded1640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a815d500 con 0x7fe6a80cd620 2024-05-11T08:06:17.023 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.978+0000 7fe6aded1640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fe6a80d0ed0 con 0x7fe6a815d120 2024-05-11T08:06:17.023 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.978+0000 7fe6a6ffd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe6a80d0100 0x7fe6a815d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:17.023 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.978+0000 7fe6a6ffd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe6a80d0100 0x7fe6a815d680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:54276/0 (socket says 172.21.15.167:54276) 2024-05-11T08:06:17.023 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.978+0000 7fe6a6ffd640 1 -- 172.21.15.167:0/2339140798 learned_addr learned my addr 172.21.15.167:0/2339140798 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:17.023 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.979+0000 7fe6a6ffd640 1 -- 172.21.15.167:0/2339140798 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe6a815d120 msgr2=0x7fe6a806fb30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-11T08:06:17.023 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.979+0000 7fe6a6ffd640 1 --2- 172.21.15.167:0/2339140798 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe6a815d120 0x7fe6a806fb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:17.023 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.979+0000 7fe6a6ffd640 1 -- 172.21.15.167:0/2339140798 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe6a80cd620 msgr2=0x7fe6a815f1c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-11T08:06:17.023 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.979+0000 7fe6a6ffd640 1 --2- 172.21.15.167:0/2339140798 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe6a80cd620 0x7fe6a815f1c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:17.023 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.979+0000 7fe6a6ffd640 1 -- 172.21.15.167:0/2339140798 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6a806e7f0 con 0x7fe6a80d0100 2024-05-11T08:06:17.023 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.979+0000 7f5527f91640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f55200d3590 0x7f5520141210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:17.024 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.979+0000 7f5527f91640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f55201462f0 0x7f5520159430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:17.024 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.979+0000 7f5527f91640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5520161930 0x7f5520159970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:17.024 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.979+0000 7f1247fff640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f12500cf790 0x7f12501433e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.167:37174/0 (socket says 172.21.15.167:37174) 2024-05-11T08:06:17.024 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.979+0000 7f1247fff640 1 -- 172.21.15.167:0/3826782190 learned_addr learned my addr 172.21.15.167:0/3826782190 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:17.024 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.979+0000 7f124f7fe640 1 --2- 172.21.15.167:0/3826782190 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1250160ac0 0x7f125006e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:17.024 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.979+0000 7f124effd640 1 --2- 172.21.15.167:0/3826782190 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f125006cc50 0x7f125013f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:17.025 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.979+0000 7f5527f91640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5520146850 con 0x7f5520161930 2024-05-11T08:06:17.025 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.979+0000 7f5527f91640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f55201466d0 con 0x7f55200d3590 2024-05-11T08:06:17.025 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.979+0000 7f5527f91640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f55200b7280 con 0x7f55201462f0 2024-05-11T08:06:17.025 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.979+0000 7f5525d2d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f55200d3590 0x7f5520141210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:17.025 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.979+0000 7f5525d2d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f55200d3590 0x7f5520141210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.167:37126/0 (socket says 172.21.15.167:37126) 2024-05-11T08:06:17.025 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.979+0000 7f5525d2d640 1 -- 172.21.15.167:0/3478973341 learned_addr learned my addr 172.21.15.167:0/3478973341 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:17.025 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.979+0000 7f552552c640 1 --2- 172.21.15.167:0/3478973341 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f55201462f0 0x7f5520159430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:17.025 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.979+0000 7f4eba8e1640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4eac03e830 0x7f4eac0c9510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:17.026 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.979+0000 7f8217fff640 1 --2- 172.21.15.167:0/2608700135 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f821815b410 0x7f821814f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:17.026 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.979+0000 7f821db51640 1 -- 172.21.15.167:0/2608700135 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f82180b6e20 con 0x7f821815b410 2024-05-11T08:06:17.026 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.979+0000 7f220cd04640 1 -- 172.21.15.167:0/3672438570 shutdown_connections 2024-05-11T08:06:17.026 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.979+0000 7f220cd04640 1 --2- 172.21.15.167:0/3672438570 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f21f0003cb0 0x7f21f0004090 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:17.026 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.979+0000 7f220cd04640 1 -- 172.21.15.167:0/3672438570 >> 172.21.15.167:0/3672438570 conn(0x7f22080b6b80 msgr2=0x7f22080b6f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:17.027 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.980+0000 7f4eba8e1640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4eac054120 0x7f4eac052430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:17.027 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.979+0000 7f80ba7fc640 1 --2- 172.21.15.167:0/660165834 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f80b404f4c0 0x7f80b4036d60 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f80b000e9e0 tx=0x7f80b000eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:17.027 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.980+0000 7f4eba8e1640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4eac0e30b0 0x7f4eac052970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:17.027 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.980+0000 7f4eba8e1640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4eac0e3490 con 0x7f4eac03e830 2024-05-11T08:06:17.028 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.980+0000 7f552652e640 1 --2- 172.21.15.167:0/3478973341 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5520161930 0x7f5520159970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:17.028 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.979+0000 7f821db51640 1 -- 172.21.15.167:0/2608700135 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f82180b6ca0 con 0x7f82180b6550 2024-05-11T08:06:17.028 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.979+0000 7f821db51640 1 -- 172.21.15.167:0/2608700135 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f82180cfd20 con 0x7f821806ebf0 2024-05-11T08:06:17.028 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.980+0000 7f4eba8e1640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4eac03bfb0 con 0x7f4eac054120 2024-05-11T08:06:17.028 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.980+0000 7f4eba8e1640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f4eac0078d0 con 0x7f4eac0e30b0 2024-05-11T08:06:17.029 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.980+0000 7f220cd04640 1 -- 172.21.15.167:0/3672438570 shutdown_connections 2024-05-11T08:06:17.029 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.980+0000 7f5525d2d640 1 -- 172.21.15.167:0/3478973341 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f55201462f0 msgr2=0x7f5520159430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:17.029 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.980+0000 7f4eb3fff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4eac03e830 0x7f4eac0c9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:17.029 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.980+0000 7f5525d2d640 1 --2- 172.21.15.167:0/3478973341 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f55201462f0 0x7f5520159430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:17.030 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.980+0000 7f5525d2d640 1 -- 172.21.15.167:0/3478973341 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5520161930 msgr2=0x7f5520159970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:17.030 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.980+0000 7f5525d2d640 1 --2- 172.21.15.167:0/3478973341 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5520161930 0x7f5520159970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:17.030 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.980+0000 7f5525d2d640 1 -- 172.21.15.167:0/3478973341 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55201e7a60 con 0x7f55200d3590 2024-05-11T08:06:17.030 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.980+0000 7fe6a6ffd640 1 --2- 172.21.15.167:0/2339140798 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe6a80d0100 0x7fe6a815d680 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe69c00d800 tx=0x7fe69c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:17.030 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.980+0000 7f4eb8e7e640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4eac0e30b0 0x7f4eac052970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:17.030 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.980+0000 7f4eb8e7e640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4eac0e30b0 0x7f4eac052970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.167:37180/0 (socket says 172.21.15.167:37180) 2024-05-11T08:06:17.030 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.980+0000 7f4eb8e7e640 1 -- 172.21.15.167:0/3859619623 learned_addr learned my addr 172.21.15.167:0/3859619623 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:17.031 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.980+0000 7f5525d2d640 1 --2- 172.21.15.167:0/3478973341 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f55200d3590 0x7f5520141210 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f551c02a770 tx=0x7f551c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:17.031 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.980+0000 7fae14558640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fadf8004800 0x7fae0c1416f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:17.031 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.980+0000 7f220cd04640 1 -- 172.21.15.167:0/3672438570 wait complete. 2024-05-11T08:06:17.031 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.980+0000 7fae14558640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fae0c141ee0 0x7fae0c144050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:17.032 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.980+0000 7f55127fc640 1 -- 172.21.15.167:0/3478973341 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f551c040020 con 0x7f55200d3590 2024-05-11T08:06:17.032 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.980+0000 7f124f7fe640 1 -- 172.21.15.167:0/3826782190 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f12500cf790 msgr2=0x7f12501433e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:17.032 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.980+0000 7f124f7fe640 1 --2- 172.21.15.167:0/3826782190 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f12500cf790 0x7f12501433e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:17.032 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.980+0000 7f124f7fe640 1 -- 172.21.15.167:0/3826782190 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f125006cc50 msgr2=0x7f125013f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:17.032 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.980+0000 7f124f7fe640 1 --2- 172.21.15.167:0/3826782190 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f125006cc50 0x7f125013f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:17.032 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.980+0000 7f124f7fe640 1 -- 172.21.15.167:0/3826782190 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1250143e30 con 0x7f1250160ac0 2024-05-11T08:06:17.033 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.980+0000 7f55127fc640 1 -- 172.21.15.167:0/3478973341 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f551c038040 con 0x7f55200d3590 2024-05-11T08:06:17.033 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.980+0000 7f5527f91640 1 -- 172.21.15.167:0/3478973341 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55201e7d40 con 0x7f55200d3590 2024-05-11T08:06:17.033 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.980+0000 7fae122f4640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fadf8004800 0x7fae0c1416f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:17.033 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.981+0000 7f4eb8e7e640 1 -- 172.21.15.167:0/3859619623 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4eac054120 msgr2=0x7f4eac052430 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-11T08:06:17.033 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.981+0000 7f4eb8e7e640 1 --2- 172.21.15.167:0/3859619623 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4eac054120 0x7f4eac052430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:17.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.981+0000 7f4eb8e7e640 1 -- 172.21.15.167:0/3859619623 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4eac03e830 msgr2=0x7f4eac0c9510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:17.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.981+0000 7fae11af3640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fae0c141ee0 0x7fae0c144050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:17.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.981+0000 7fae11af3640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fae0c141ee0 0x7fae0c144050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:54326/0 (socket says 172.21.15.167:54326) 2024-05-11T08:06:17.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.981+0000 7fae11af3640 1 -- 172.21.15.167:0/1466469987 learned_addr learned my addr 172.21.15.167:0/1466469987 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:17.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.981+0000 7fae14558640 1 --2- 172.21.15.167:0/1466469987 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fae0c144590 0x7fae0c0701f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:17.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.981+0000 7fae14558640 1 -- 172.21.15.167:0/1466469987 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fae0c0cfd30 con 0x7fae0c141ee0 2024-05-11T08:06:17.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.981+0000 7fae14558640 1 -- 172.21.15.167:0/1466469987 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fae0c0d0a30 con 0x7fadf8004800 2024-05-11T08:06:17.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.981+0000 7fae14558640 1 -- 172.21.15.167:0/1466469987 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fae0c0cfeb0 con 0x7fae0c144590 2024-05-11T08:06:17.035 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.980+0000 7f8217fff640 1 -- 172.21.15.167:0/2608700135 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f821806ebf0 msgr2=0x7f821815b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:17.035 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.980+0000 7f8217fff640 1 --2- 172.21.15.167:0/2608700135 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f821806ebf0 0x7f821815b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:17.035 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.980+0000 7f8217fff640 1 -- 172.21.15.167:0/2608700135 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82180b6550 msgr2=0x7f821814e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:17.035 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.980+0000 7f8217fff640 1 --2- 172.21.15.167:0/2608700135 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82180b6550 0x7f821814e360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:17.035 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.980+0000 7f8217fff640 1 -- 172.21.15.167:0/2608700135 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8218150340 con 0x7f821815b410 2024-05-11T08:06:17.035 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.981+0000 7f8217fff640 1 --2- 172.21.15.167:0/2608700135 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f821815b410 0x7f821814f8c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f8210030000 tx=0x7f8210004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:17.035 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.981+0000 7f8216ffd640 1 --2- 172.21.15.167:0/2608700135 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82180b6550 0x7f821814e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T08:06:17.035 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.981+0000 7f5527f91640 1 -- 172.21.15.167:0/3478973341 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55201e8300 con 0x7f55200d3590 2024-05-11T08:06:17.036 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.981+0000 7f55127fc640 1 -- 172.21.15.167:0/3478973341 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f551c037070 con 0x7f55200d3590 2024-05-11T08:06:17.036 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.981+0000 7f552652e640 1 --2- 172.21.15.167:0/3478973341 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5520161930 0x7f5520159970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T08:06:17.036 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.980+0000 7f80a3fff640 1 -- 172.21.15.167:0/660165834 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80b0017070 con 0x7f80b404f4c0 2024-05-11T08:06:17.036 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.980+0000 7f80a3fff640 1 -- 172.21.15.167:0/660165834 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80b000c9d0 con 0x7f80b404f4c0 2024-05-11T08:06:17.036 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.981+0000 7f80a3fff640 1 -- 172.21.15.167:0/660165834 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80b000cda0 con 0x7f80b404f4c0 2024-05-11T08:06:17.036 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.981+0000 7f80bbfff640 1 -- 172.21.15.167:0/660165834 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80b4185bc0 con 0x7f80b404f4c0 2024-05-11T08:06:17.036 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.981+0000 7f80bbfff640 1 -- 172.21.15.167:0/660165834 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80b4186180 con 0x7f80b404f4c0 2024-05-11T08:06:17.036 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.981+0000 7fe68ffff640 1 -- 172.21.15.167:0/2339140798 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe69c00b6c0 con 0x7fe6a80d0100 2024-05-11T08:06:17.037 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.981+0000 7fe6aded1640 1 -- 172.21.15.167:0/2339140798 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6a8161690 con 0x7fe6a80d0100 2024-05-11T08:06:17.037 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.981+0000 7fe6aded1640 1 -- 172.21.15.167:0/2339140798 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6a806ed70 con 0x7fe6a80d0100 2024-05-11T08:06:17.037 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.981+0000 7fe68ffff640 1 -- 172.21.15.167:0/2339140798 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe69c019070 con 0x7fe6a80d0100 2024-05-11T08:06:17.037 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.981+0000 7fe68ffff640 1 -- 172.21.15.167:0/2339140798 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe69c00ba80 con 0x7fe6a80d0100 2024-05-11T08:06:17.037 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.981+0000 7f124f7fe640 1 --2- 172.21.15.167:0/3826782190 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1250160ac0 0x7f125006e410 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f124800be50 tx=0x7f124800bf40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:17.037 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.981+0000 7f4eb8e7e640 1 --2- 172.21.15.167:0/3859619623 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4eac03e830 0x7f4eac0c9510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:17.038 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.981+0000 7f4eb8e7e640 1 -- 172.21.15.167:0/3859619623 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4eac03da70 con 0x7f4eac0e30b0 2024-05-11T08:06:17.038 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.982+0000 7f124cff9640 1 -- 172.21.15.167:0/3826782190 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1248019070 con 0x7f1250160ac0 2024-05-11T08:06:17.038 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.982+0000 7f124ffff640 1 -- 172.21.15.167:0/3826782190 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1250144110 con 0x7f1250160ac0 2024-05-11T08:06:17.038 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.982+0000 7f124ffff640 1 -- 172.21.15.167:0/3826782190 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f125015e3e0 con 0x7f1250160ac0 2024-05-11T08:06:17.038 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.982+0000 7f81fffff640 1 -- 172.21.15.167:0/2608700135 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8210041020 con 0x7f821815b410 2024-05-11T08:06:17.039 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.982+0000 7f124cff9640 1 -- 172.21.15.167:0/3826782190 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1248014440 con 0x7f1250160ac0 2024-05-11T08:06:17.039 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.982+0000 7f124cff9640 1 -- 172.21.15.167:0/3826782190 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1248014810 con 0x7f1250160ac0 2024-05-11T08:06:17.039 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.982+0000 7f220cd04640 1 Processor -- start 2024-05-11T08:06:17.039 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.982+0000 7f220cd04640 1 -- start start 2024-05-11T08:06:17.040 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.982+0000 7f821db51640 1 -- 172.21.15.167:0/2608700135 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82181511d0 con 0x7f821815b410 2024-05-11T08:06:17.040 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.982+0000 7f821db51640 1 -- 172.21.15.167:0/2608700135 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f821806fb60 con 0x7f821815b410 2024-05-11T08:06:17.040 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.982+0000 7f4eb3fff640 1 --2- 172.21.15.167:0/3859619623 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4eac03e830 0x7f4eac0c9510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T08:06:17.040 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.983+0000 7f4eb8e7e640 1 --2- 172.21.15.167:0/3859619623 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4eac0e30b0 0x7f4eac052970 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f4eb407b800 tx=0x7f4eb4076680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:17.040 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.983+0000 7f821db51640 1 -- 172.21.15.167:0/2608700135 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8218150ff0 con 0x7f821815b410 2024-05-11T08:06:17.041 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.981+0000 7fae12af5640 1 --2- 172.21.15.167:0/1466469987 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fae0c144590 0x7fae0c0701f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:17.041 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.983+0000 7fe6aded1640 1 -- 172.21.15.167:0/2339140798 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe688001030 con 0x7fe6a80d0100 2024-05-11T08:06:17.041 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.983+0000 7f80bbfff640 1 -- 172.21.15.167:0/660165834 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f80b40057d0 con 0x7f80b404f4c0 2024-05-11T08:06:17.041 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.982+0000 7fae11af3640 1 -- 172.21.15.167:0/1466469987 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fae0c144590 msgr2=0x7fae0c0701f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:17.042 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.982+0000 7fae11af3640 1 --2- 172.21.15.167:0/1466469987 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fae0c144590 0x7fae0c0701f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:17.042 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.982+0000 7fae11af3640 1 -- 172.21.15.167:0/1466469987 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fadf8004800 msgr2=0x7fae0c1416f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:17.042 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.982+0000 7fae11af3640 1 --2- 172.21.15.167:0/1466469987 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fadf8004800 0x7fae0c1416f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:17.042 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.982+0000 7fae11af3640 1 -- 172.21.15.167:0/1466469987 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae0c152800 con 0x7fae0c141ee0 2024-05-11T08:06:17.042 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.983+0000 7f220cd04640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f21f0003cb0 0x7f2208162a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:17.042 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.983+0000 7fae11af3640 1 --2- 172.21.15.167:0/1466469987 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fae0c141ee0 0x7fae0c144050 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fadfc02a420 tx=0x7fadfc004700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:17.043 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.984+0000 7f81fffff640 1 -- 172.21.15.167:0/2608700135 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8210031600 con 0x7f821815b410 2024-05-11T08:06:17.043 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.984+0000 7f81fffff640 1 -- 172.21.15.167:0/2608700135 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8210036040 con 0x7f821815b410 2024-05-11T08:06:17.043 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.984+0000 7f124cff9640 1 -- 172.21.15.167:0/3826782190 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f124801c070 con 0x7f1250160ac0 2024-05-11T08:06:17.043 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.984+0000 7f124ffff640 1 -- 172.21.15.167:0/3826782190 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1234001030 con 0x7f1250160ac0 2024-05-11T08:06:17.043 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.984+0000 7f5527f91640 1 -- 172.21.15.167:0/3478973341 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f55200694b0 con 0x7f55200d3590 2024-05-11T08:06:17.044 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.984+0000 7f220cd04640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f22081f9350 0x7f22080cb690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:17.044 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.984+0000 7f220cd04640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f22080cbbd0 0x7f2208071f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:17.044 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.984+0000 7f220cd04640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f220815e0c0 con 0x7f21f0003cb0 2024-05-11T08:06:17.044 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.984+0000 7f220cd04640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f220815df40 con 0x7f22080cbbd0 2024-05-11T08:06:17.044 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.984+0000 7f220cd04640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f22080d4580 con 0x7f22081f9350 2024-05-11T08:06:17.044 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.984+0000 7f2205d9b640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f22081f9350 0x7f22080cb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:17.044 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.984+0000 7fe68ffff640 1 -- 172.21.15.167:0/2339140798 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fe69c00bc20 con 0x7fe6a80d0100 2024-05-11T08:06:17.045 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.984+0000 7fae02ffd640 1 -- 172.21.15.167:0/1466469987 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadfc042020 con 0x7fae0c141ee0 2024-05-11T08:06:17.045 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.985+0000 7f2205d9b640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f22081f9350 0x7f22080cb690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.167:37188/0 (socket says 172.21.15.167:37188) 2024-05-11T08:06:17.045 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.985+0000 7f4eb0ff9640 1 -- 172.21.15.167:0/3859619623 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb4075470 con 0x7f4eac0e30b0 2024-05-11T08:06:17.045 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.985+0000 7f2205d9b640 1 -- 172.21.15.167:0/1271819819 learned_addr learned my addr 172.21.15.167:0/1271819819 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:17.046 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.985+0000 7fae02ffd640 1 -- 172.21.15.167:0/1466469987 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fadfc033a50 con 0x7fae0c141ee0 2024-05-11T08:06:17.046 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.985+0000 7f4eb0ff9640 1 -- 172.21.15.167:0/3859619623 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4eb4090030 con 0x7f4eac0e30b0 2024-05-11T08:06:17.046 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.985+0000 7fae14558640 1 -- 172.21.15.167:0/1466469987 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae0c152ae0 con 0x7fae0c141ee0 2024-05-11T08:06:17.047 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.985+0000 7f4eba8e1640 1 -- 172.21.15.167:0/3859619623 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4eac03cef0 con 0x7f4eac0e30b0 2024-05-11T08:06:17.047 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.985+0000 7fae14558640 1 -- 172.21.15.167:0/1466469987 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae0c156760 con 0x7fae0c141ee0 2024-05-11T08:06:17.047 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.985+0000 7f124cff9640 1 --2- 172.21.15.167:0/3826782190 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f1230076320 0x7f12300787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:17.047 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.985+0000 7f124effd640 1 --2- 172.21.15.167:0/3826782190 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f1230076320 0x7f12300787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:17.047 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.985+0000 7f80a3fff640 1 -- 172.21.15.167:0/660165834 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f80b001a070 con 0x7f80b404f4c0 2024-05-11T08:06:17.048 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.985+0000 7f2206d9d640 1 --2- 172.21.15.167:0/1271819819 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f22080cbbd0 0x7f2208071f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:17.048 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.985+0000 7f2205d9b640 1 -- 172.21.15.167:0/1271819819 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f22080cbbd0 msgr2=0x7f2208071f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:17.048 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.985+0000 7f2205d9b640 1 --2- 172.21.15.167:0/1271819819 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f22080cbbd0 0x7f2208071f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:17.048 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.985+0000 7f2205d9b640 1 -- 172.21.15.167:0/1271819819 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f21f0003cb0 msgr2=0x7f2208162a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:17.048 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.985+0000 7f2205d9b640 1 --2- 172.21.15.167:0/1271819819 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f21f0003cb0 0x7f2208162a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:17.048 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.985+0000 7f2205d9b640 1 -- 172.21.15.167:0/1271819819 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22081f9810 con 0x7f22081f9350 2024-05-11T08:06:17.048 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.986+0000 7f2205d9b640 1 --2- 172.21.15.167:0/1271819819 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f22081f9350 0x7f22080cb690 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f21f8011ca0 tx=0x7f21f800f610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:17.049 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.986+0000 7f80a3fff640 1 --2- 172.21.15.167:0/660165834 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f80a80763a0 0x7f80a8078860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:17.049 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.986+0000 7f4eb0ff9640 1 -- 172.21.15.167:0/3859619623 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb4075bb0 con 0x7f4eac0e30b0 2024-05-11T08:06:17.049 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.986+0000 7f21f6ffd640 1 -- 172.21.15.167:0/1271819819 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21f800fd20 con 0x7f22081f9350 2024-05-11T08:06:17.049 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.986+0000 7f80a3fff640 1 -- 172.21.15.167:0/660165834 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f80b00978d0 con 0x7f80b404f4c0 2024-05-11T08:06:17.050 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.985+0000 7fe68ffff640 1 --2- 172.21.15.167:0/2339140798 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fe6940763d0 0x7fe694078890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:17.050 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.985+0000 7fe68ffff640 1 -- 172.21.15.167:0/2339140798 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fe69c097dc0 con 0x7fe6a80d0100 2024-05-11T08:06:17.050 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.985+0000 7fe68ffff640 1 --2- 172.21.15.167:0/2339140798 >> [v2:172.21.15.204:6808/1949534184,v1:172.21.15.204:6809/1949534184] conn(0x7fe69407fad0 0x7fe694081f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:17.050 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.985+0000 7fe68ffff640 1 -- 172.21.15.167:0/2339140798 --> [v2:172.21.15.204:6808/1949534184,v1:172.21.15.204:6809/1949534184] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe69c01aec0 con 0x7fe69407fad0 2024-05-11T08:06:17.050 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.986+0000 7f4eba8e1640 1 -- 172.21.15.167:0/3859619623 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f4eac03d4b0 con 0x7f4eac0e30b0 2024-05-11T08:06:17.051 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.986+0000 7fe68ffff640 1 -- 172.21.15.167:0/2339140798 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe69c05d310 con 0x7fe6a80d0100 2024-05-11T08:06:17.051 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.986+0000 7f80a3fff640 1 --2- 172.21.15.167:0/660165834 >> [v2:172.21.15.204:6824/2921847521,v1:172.21.15.204:6825/2921847521] conn(0x7f80a807faa0 0x7f80a8081ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:17.051 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.986+0000 7f80a3fff640 1 -- 172.21.15.167:0/660165834 --> [v2:172.21.15.204:6824/2921847521,v1:172.21.15.204:6825/2921847521] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f80b0021ec0 con 0x7f80a807faa0 2024-05-11T08:06:17.051 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.986+0000 7f81fffff640 1 -- 172.21.15.167:0/2608700135 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f8210044020 con 0x7f821815b410 2024-05-11T08:06:17.052 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.986+0000 7f55127fc640 1 -- 172.21.15.167:0/3478973341 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f551c043020 con 0x7f55200d3590 2024-05-11T08:06:17.052 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.986+0000 7f80a3fff640 1 -- 172.21.15.167:0/660165834 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f80b005ceb0 con 0x7f80b404f4c0 2024-05-11T08:06:17.052 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.986+0000 7f80baffd640 1 --2- 172.21.15.167:0/660165834 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f80a80763a0 0x7f80a8078860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:17.052 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.986+0000 7f80bb7fe640 1 --2- 172.21.15.167:0/660165834 >> [v2:172.21.15.204:6824/2921847521,v1:172.21.15.204:6825/2921847521] conn(0x7f80a807faa0 0x7f80a8081ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:17.052 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.987+0000 7f21f6ffd640 1 -- 172.21.15.167:0/1271819819 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21f8004030 con 0x7f22081f9350 2024-05-11T08:06:17.053 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.987+0000 7f55127fc640 1 --2- 172.21.15.167:0/3478973341 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f550407a6e0 0x7f550407cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:17.053 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.987+0000 7f220cd04640 1 -- 172.21.15.167:0/1271819819 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22081f9af0 con 0x7f22081f9350 2024-05-11T08:06:17.053 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.987+0000 7fae02ffd640 1 -- 172.21.15.167:0/1466469987 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadfc037040 con 0x7fae0c141ee0 2024-05-11T08:06:17.053 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.986+0000 7f124effd640 1 --2- 172.21.15.167:0/3826782190 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f1230076320 0x7f12300787e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1240006fd0 tx=0x7f1240009060 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:17.054 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.986+0000 7f124cff9640 1 -- 172.21.15.167:0/3826782190 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f1248097750 con 0x7f1250160ac0 2024-05-11T08:06:17.054 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.986+0000 7f124cff9640 1 --2- 172.21.15.167:0/3826782190 >> [v2:172.21.15.167:6824/453689293,v1:172.21.15.167:6825/453689293] conn(0x7f123007faa0 0x7f1230081ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:17.054 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.987+0000 7fe6a77fe640 1 --2- 172.21.15.167:0/2339140798 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fe6940763d0 0x7fe694078890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:17.054 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.987+0000 7f220cd04640 1 -- 172.21.15.167:0/1271819819 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f22081fa0b0 con 0x7f22081f9350 2024-05-11T08:06:17.055 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.987+0000 7f55127fc640 1 -- 172.21.15.167:0/3478973341 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f551c0b8420 con 0x7f55200d3590 2024-05-11T08:06:17.055 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.987+0000 7f124cff9640 1 -- 172.21.15.167:0/3826782190 --> [v2:172.21.15.167:6824/453689293,v1:172.21.15.167:6825/453689293] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1248022e50 con 0x7f123007faa0 2024-05-11T08:06:17.055 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.987+0000 7f1247fff640 1 --2- 172.21.15.167:0/3826782190 >> [v2:172.21.15.167:6824/453689293,v1:172.21.15.167:6825/453689293] conn(0x7f123007faa0 0x7f1230081ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:17.056 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.987+0000 7f55127fc640 1 --2- 172.21.15.167:0/3478973341 >> [v2:172.21.15.204:6800/1311831500,v1:172.21.15.204:6801/1311831500] conn(0x7f5504083e60 0x7f55040862a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:17.056 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.987+0000 7f55127fc640 1 -- 172.21.15.167:0/3478973341 --> [v2:172.21.15.204:6800/1311831500,v1:172.21.15.204:6801/1311831500] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f551c03de50 con 0x7f5504083e60 2024-05-11T08:06:17.056 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.987+0000 7f55127fc640 1 -- 172.21.15.167:0/3478973341 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f551c0b8810 con 0x7f55200d3590 2024-05-11T08:06:17.056 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.987+0000 7f552652e640 1 --2- 172.21.15.167:0/3478973341 >> [v2:172.21.15.204:6800/1311831500,v1:172.21.15.204:6801/1311831500] conn(0x7f5504083e60 0x7f55040862a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:17.056 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.987+0000 7f81fffff640 1 --2- 172.21.15.167:0/2608700135 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f82040762d0 0x7f8204078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:17.056 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.987+0000 7f81fffff640 1 -- 172.21.15.167:0/2608700135 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f82100b8bd0 con 0x7f821815b410 2024-05-11T08:06:17.057 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.987+0000 7f552552c640 1 --2- 172.21.15.167:0/3478973341 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f550407a6e0 0x7f550407cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:17.057 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.987+0000 7f4e9e7fc640 1 -- 172.21.15.167:0/3859619623 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4e8c001030 con 0x7f4eac0e30b0 2024-05-11T08:06:17.057 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.988+0000 7fe6a7fff640 1 --2- 172.21.15.167:0/2339140798 >> [v2:172.21.15.204:6808/1949534184,v1:172.21.15.204:6809/1949534184] conn(0x7fe69407fad0 0x7fe694081f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:17.057 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.988+0000 7fe6a77fe640 1 --2- 172.21.15.167:0/2339140798 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fe6940763d0 0x7fe694078890 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe69803d000 tx=0x7fe698008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:17.058 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.988+0000 7fae02ffd640 1 -- 172.21.15.167:0/1466469987 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fadfc045020 con 0x7fae0c141ee0 2024-05-11T08:06:17.058 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.988+0000 7f82177fe640 1 --2- 172.21.15.167:0/2608700135 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f82040762d0 0x7f8204078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:17.058 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.988+0000 7f81fffff640 1 --2- 172.21.15.167:0/2608700135 >> [v2:172.21.15.167:6800/3242166900,v1:172.21.15.167:6801/3242166900] conn(0x7f820407fa50 0x7f8204081e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:17.058 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.987+0000 7f21f6ffd640 1 -- 172.21.15.167:0/1271819819 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21f8004400 con 0x7f22081f9350 2024-05-11T08:06:17.059 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.988+0000 7f81fffff640 1 -- 172.21.15.167:0/2608700135 --> [v2:172.21.15.167:6800/3242166900,v1:172.21.15.167:6801/3242166900] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f821003be50 con 0x7f820407fa50 2024-05-11T08:06:17.059 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.988+0000 7f81fffff640 1 -- 172.21.15.167:0/2608700135 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f82100b8fc0 con 0x7f821815b410 2024-05-11T08:06:17.059 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.987+0000 7f80bb7fe640 1 --2- 172.21.15.167:0/660165834 >> [v2:172.21.15.204:6824/2921847521,v1:172.21.15.204:6825/2921847521] conn(0x7f80a807faa0 0x7f80a8081ee0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:17.059 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.988+0000 7f21f4ff9640 1 -- 172.21.15.167:0/1271819819 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f22080694b0 con 0x7f22081f9350 2024-05-11T08:06:17.059 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.988+0000 7f552552c640 1 --2- 172.21.15.167:0/3478973341 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f550407a6e0 0x7f550407cba0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f55200b3710 tx=0x7f5514009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:17.060 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.988+0000 7f8216ffd640 1 --2- 172.21.15.167:0/2608700135 >> [v2:172.21.15.167:6800/3242166900,v1:172.21.15.167:6801/3242166900] conn(0x7f820407fa50 0x7f8204081e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:17.060 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.988+0000 7f552652e640 1 --2- 172.21.15.167:0/3478973341 >> [v2:172.21.15.204:6800/1311831500,v1:172.21.15.204:6801/1311831500] conn(0x7f5504083e60 0x7f55040862a0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:17.060 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.988+0000 7f82177fe640 1 --2- 172.21.15.167:0/2608700135 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f82040762d0 0x7f8204078790 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f8208000f60 tx=0x7f8208009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:17.061 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.989+0000 7fae14558640 1 -- 172.21.15.167:0/1466469987 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fae0c1525b0 con 0x7fae0c141ee0 2024-05-11T08:06:17.061 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.989+0000 7f124cff9640 1 -- 172.21.15.167:0/3826782190 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f124809e050 con 0x7f1250160ac0 2024-05-11T08:06:17.061 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.989+0000 7f80baffd640 1 --2- 172.21.15.167:0/660165834 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f80a80763a0 0x7f80a8078860 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f80ac009670 tx=0x7f80ac006bd0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:17.061 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.989+0000 7f80a3fff640 1 -- 172.21.15.167:0/660165834 <== osd.5 v2:172.21.15.204:6824/2921847521 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f80b0021ec0 con 0x7f80a807faa0 2024-05-11T08:06:17.061 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.989+0000 7f4eb0ff9640 1 -- 172.21.15.167:0/3859619623 <== mon.2 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f4eb4075d50 con 0x7f4eac0e30b0 2024-05-11T08:06:17.062 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.989+0000 7f1247fff640 1 --2- 172.21.15.167:0/3826782190 >> [v2:172.21.15.167:6824/453689293,v1:172.21.15.167:6825/453689293] conn(0x7f123007faa0 0x7f1230081ee0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:17.062 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.989+0000 7fae02ffd640 1 --2- 172.21.15.167:0/1466469987 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fadf00763a0 0x7fadf0078860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:17.062 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.989+0000 7f55127fc640 1 -- 172.21.15.167:0/3478973341 <== osd.4 v2:172.21.15.204:6800/1311831500 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f551c03de50 con 0x7f5504083e60 2024-05-11T08:06:17.063 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.989+0000 7fae02ffd640 1 -- 172.21.15.167:0/1466469987 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fadfc0b7e20 con 0x7fae0c141ee0 2024-05-11T08:06:17.063 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.989+0000 7f21f6ffd640 1 -- 172.21.15.167:0/1271819819 <== mon.2 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f21f8012040 con 0x7f22081f9350 2024-05-11T08:06:17.063 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.989+0000 7fae122f4640 1 --2- 172.21.15.167:0/1466469987 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fadf00763a0 0x7fadf0078860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:17.063 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.990+0000 7f4eb0ff9640 1 --2- 172.21.15.167:0/3859619623 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f4e980763a0 0x7f4e98078860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:17.064 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.990+0000 7fae02ffd640 1 --2- 172.21.15.167:0/1466469987 >> [v2:172.21.15.167:6816/1585024048,v1:172.21.15.167:6817/1585024048] conn(0x7fadf007fa90 0x7fadf0081ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:17.064 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.990+0000 7fae02ffd640 1 -- 172.21.15.167:0/1466469987 --> [v2:172.21.15.167:6816/1585024048,v1:172.21.15.167:6817/1585024048] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fadfc02ce50 con 0x7fadf007fa90 2024-05-11T08:06:17.064 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.990+0000 7fae02ffd640 1 -- 172.21.15.167:0/1466469987 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fadfc07cdb0 con 0x7fae0c141ee0 2024-05-11T08:06:17.064 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.990+0000 7f4eb3fff640 1 --2- 172.21.15.167:0/3859619623 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f4e980763a0 0x7f4e98078860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:17.064 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.990+0000 7fae12af5640 1 --2- 172.21.15.167:0/1466469987 >> [v2:172.21.15.167:6816/1585024048,v1:172.21.15.167:6817/1585024048] conn(0x7fadf007fa90 0x7fadf0081ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:17.065 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.990+0000 7f124cff9640 1 -- 172.21.15.167:0/3826782190 <== osd.3 v2:172.21.15.167:6824/453689293 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1248022e50 con 0x7f123007faa0 2024-05-11T08:06:17.065 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.990+0000 7f4eb0ff9640 1 -- 172.21.15.167:0/3859619623 <== mon.2 v2:172.21.15.204:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f4eb4113a80 con 0x7f4eac0e30b0 2024-05-11T08:06:17.065 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.990+0000 7f21f6ffd640 1 --2- 172.21.15.167:0/1271819819 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f21e8076320 0x7f21e80787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:17.065 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.989+0000 7f8216ffd640 1 --2- 172.21.15.167:0/2608700135 >> [v2:172.21.15.167:6800/3242166900,v1:172.21.15.167:6801/3242166900] conn(0x7f820407fa50 0x7f8204081e90 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:17.066 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.990+0000 7f4eb0ff9640 1 --2- 172.21.15.167:0/3859619623 >> [v2:172.21.15.167:6804/3863307356,v1:172.21.15.167:6806/3863307356] conn(0x7f4e9807fb20 0x7f4e98081f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:17.066 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.990+0000 7fae122f4640 1 --2- 172.21.15.167:0/1466469987 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fadf00763a0 0x7fadf0078860 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fae08001920 tx=0x7fae08007040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:17.066 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.990+0000 7f4eb3fff640 1 --2- 172.21.15.167:0/3859619623 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f4e980763a0 0x7f4e98078860 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4ea8006c80 tx=0x7f4ea8008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:17.066 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.990+0000 7f4eb0ff9640 1 -- 172.21.15.167:0/3859619623 --> [v2:172.21.15.167:6804/3863307356,v1:172.21.15.167:6806/3863307356] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4eb4090030 con 0x7f4e9807fb20 2024-05-11T08:06:17.066 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.990+0000 7f4eb0ff9640 1 -- 172.21.15.167:0/3859619623 <== mon.2 v2:172.21.15.204:3301/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4eb4113e70 con 0x7f4eac0e30b0 2024-05-11T08:06:17.067 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.990+0000 7f81fffff640 1 -- 172.21.15.167:0/2608700135 <== osd.2 v2:172.21.15.167:6800/3242166900 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f821003be50 con 0x7f820407fa50 2024-05-11T08:06:17.067 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.991+0000 7f4eb37fe640 1 --2- 172.21.15.167:0/3859619623 >> [v2:172.21.15.167:6804/3863307356,v1:172.21.15.167:6806/3863307356] conn(0x7f4e9807fb20 0x7f4e98081f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:17.067 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.991+0000 7f220659c640 1 --2- 172.21.15.167:0/1271819819 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f21e8076320 0x7f21e80787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:17.067 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.991+0000 7fe6a7fff640 1 --2- 172.21.15.167:0/2339140798 >> [v2:172.21.15.204:6808/1949534184,v1:172.21.15.204:6809/1949534184] conn(0x7fe69407fad0 0x7fe694081f10 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:17.068 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.991+0000 7f4eb37fe640 1 --2- 172.21.15.167:0/3859619623 >> [v2:172.21.15.167:6804/3863307356,v1:172.21.15.167:6806/3863307356] conn(0x7f4e9807fb20 0x7f4e98081f60 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:17.068 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.991+0000 7f21f6ffd640 1 -- 172.21.15.167:0/1271819819 <== mon.2 v2:172.21.15.204:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f21f8097f20 con 0x7f22081f9350 2024-05-11T08:06:17.068 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.991+0000 7f220659c640 1 --2- 172.21.15.167:0/1271819819 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f21e8076320 0x7f21e80787e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f21fc001350 tx=0x7f21fc006000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:17.069 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.991+0000 7fae12af5640 1 --2- 172.21.15.167:0/1466469987 >> [v2:172.21.15.167:6816/1585024048,v1:172.21.15.167:6817/1585024048] conn(0x7fadf007fa90 0x7fadf0081ed0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:17.069 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.991+0000 7f21f6ffd640 1 --2- 172.21.15.167:0/1271819819 >> [v2:172.21.15.204:6816/2889851077,v1:172.21.15.204:6817/2889851077] conn(0x7f21e807fa90 0x7f21e8081ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:17.069 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.991+0000 7f21f6ffd640 1 -- 172.21.15.167:0/1271819819 --> [v2:172.21.15.204:6816/2889851077,v1:172.21.15.204:6817/2889851077] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f21f8015e50 con 0x7f21e807fa90 2024-05-11T08:06:17.069 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.991+0000 7f21f6ffd640 1 -- 172.21.15.167:0/1271819819 <== mon.2 v2:172.21.15.204:3301/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f21f809a540 con 0x7f22081f9350 2024-05-11T08:06:17.069 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.992+0000 7fe68ffff640 1 -- 172.21.15.167:0/2339140798 <== osd.7 v2:172.21.15.204:6808/1949534184 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe69c01aec0 con 0x7fe69407fad0 2024-05-11T08:06:17.070 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.992+0000 7f2206d9d640 1 --2- 172.21.15.167:0/1271819819 >> [v2:172.21.15.204:6816/2889851077,v1:172.21.15.204:6817/2889851077] conn(0x7f21e807fa90 0x7f21e8081ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:17.070 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.992+0000 7f4eb0ff9640 1 -- 172.21.15.167:0/3859619623 <== osd.0 v2:172.21.15.167:6804/3863307356 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4eb4090030 con 0x7f4e9807fb20 2024-05-11T08:06:17.070 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.992+0000 7f2206d9d640 1 --2- 172.21.15.167:0/1271819819 >> [v2:172.21.15.204:6816/2889851077,v1:172.21.15.204:6817/2889851077] conn(0x7f21e807fa90 0x7f21e8081ed0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:17.070 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.992+0000 7fae02ffd640 1 -- 172.21.15.167:0/1466469987 <== osd.1 v2:172.21.15.167:6816/1585024048 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fadfc02ce50 con 0x7fadf007fa90 2024-05-11T08:06:17.071 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:16.993+0000 7f21f6ffd640 1 -- 172.21.15.167:0/1271819819 <== osd.6 v2:172.21.15.204:6816/2889851077 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f21f8015e50 con 0x7f21e807fa90 2024-05-11T08:06:17.071 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.021+0000 7f21f4ff9640 1 -- 172.21.15.167:0/1271819819 --> [v2:172.21.15.204:6816/2889851077,v1:172.21.15.204:6817/2889851077] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f22080700c0 con 0x7f21e807fa90 2024-05-11T08:06:17.071 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.022+0000 7f5527f91640 1 -- 172.21.15.167:0/3478973341 --> [v2:172.21.15.204:6800/1311831500,v1:172.21.15.204:6801/1311831500] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5520069ef0 con 0x7f5504083e60 2024-05-11T08:06:17.071 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.022+0000 7f21f6ffd640 1 -- 172.21.15.167:0/1271819819 <== osd.6 v2:172.21.15.204:6816/2889851077 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f22000018f0 con 0x7f21e807fa90 2024-05-11T08:06:17.072 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.022+0000 7f55127fc640 1 -- 172.21.15.167:0/3478973341 <== osd.4 v2:172.21.15.204:6800/1311831500 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f5520069ef0 con 0x7f5504083e60 2024-05-11T08:06:17.073 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.022+0000 7f220cd04640 1 -- 172.21.15.167:0/1271819819 >> [v2:172.21.15.204:6816/2889851077,v1:172.21.15.204:6817/2889851077] conn(0x7f21e807fa90 msgr2=0x7f21e8081ed0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:17.073 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.022+0000 7f220cd04640 1 --2- 172.21.15.167:0/1271819819 >> [v2:172.21.15.204:6816/2889851077,v1:172.21.15.204:6817/2889851077] conn(0x7f21e807fa90 0x7f21e8081ed0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:17.073 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.023+0000 7f220cd04640 1 -- 172.21.15.167:0/1271819819 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f21e8076320 msgr2=0x7f21e80787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:17.073 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.023+0000 7f220cd04640 1 --2- 172.21.15.167:0/1271819819 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f21e8076320 0x7f21e80787e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f21fc001350 tx=0x7f21fc006000 comp rx=0 tx=0).stop 2024-05-11T08:06:17.073 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.023+0000 7f220cd04640 1 -- 172.21.15.167:0/1271819819 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f22081f9350 msgr2=0x7f22080cb690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:17.073 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.023+0000 7f220cd04640 1 --2- 172.21.15.167:0/1271819819 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f22081f9350 0x7f22080cb690 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f21f8011ca0 tx=0x7f21f800f610 comp rx=0 tx=0).stop 2024-05-11T08:06:17.073 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.023+0000 7f2206d9d640 1 -- 172.21.15.167:0/1271819819 reap_dead start 2024-05-11T08:06:17.074 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.023+0000 7f124ffff640 1 -- 172.21.15.167:0/3826782190 --> [v2:172.21.15.167:6824/453689293,v1:172.21.15.167:6825/453689293] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1234002dd0 con 0x7f123007faa0 2024-05-11T08:06:17.074 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.023+0000 7f220cd04640 1 -- 172.21.15.167:0/1271819819 shutdown_connections 2024-05-11T08:06:17.074 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.023+0000 7f220cd04640 1 -- 172.21.15.167:0/1271819819 >> 172.21.15.167:0/1271819819 conn(0x7f22080b6b80 msgr2=0x7f22080cc240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:17.074 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.023+0000 7f5527f91640 1 -- 172.21.15.167:0/3478973341 >> [v2:172.21.15.204:6800/1311831500,v1:172.21.15.204:6801/1311831500] conn(0x7f5504083e60 msgr2=0x7f55040862a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:17.074 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.023+0000 7f5527f91640 1 --2- 172.21.15.167:0/3478973341 >> [v2:172.21.15.204:6800/1311831500,v1:172.21.15.204:6801/1311831500] conn(0x7f5504083e60 0x7f55040862a0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:17.075 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.023+0000 7f5527f91640 1 -- 172.21.15.167:0/3478973341 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f550407a6e0 msgr2=0x7f550407cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:17.075 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.023+0000 7f5527f91640 1 --2- 172.21.15.167:0/3478973341 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f550407a6e0 0x7f550407cba0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f55200b3710 tx=0x7f5514009040 comp rx=0 tx=0).stop 2024-05-11T08:06:17.075 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.023+0000 7f5527f91640 1 -- 172.21.15.167:0/3478973341 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f55200d3590 msgr2=0x7f5520141210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:17.075 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.023+0000 7f5527f91640 1 --2- 172.21.15.167:0/3478973341 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f55200d3590 0x7f5520141210 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f551c02a770 tx=0x7f551c0046c0 comp rx=0 tx=0).stop 2024-05-11T08:06:17.075 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.023+0000 7f552652e640 1 -- 172.21.15.167:0/3478973341 reap_dead start 2024-05-11T08:06:17.075 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.023+0000 7f80bbfff640 1 -- 172.21.15.167:0/660165834 --> [v2:172.21.15.204:6824/2921847521,v1:172.21.15.204:6825/2921847521] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f80b400b2f0 con 0x7f80a807faa0 2024-05-11T08:06:17.075 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.023+0000 7f5527f91640 1 -- 172.21.15.167:0/3478973341 shutdown_connections 2024-05-11T08:06:17.076 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.023+0000 7f5527f91640 1 -- 172.21.15.167:0/3478973341 >> 172.21.15.167:0/3478973341 conn(0x7f55200b8300 msgr2=0x7f55200b4ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:17.076 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.023+0000 7f5527f91640 1 -- 172.21.15.167:0/3478973341 shutdown_connections 2024-05-11T08:06:17.076 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.023+0000 7f220cd04640 1 -- 172.21.15.167:0/1271819819 shutdown_connections 2024-05-11T08:06:17.076 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.023+0000 7f5527f91640 1 -- 172.21.15.167:0/3478973341 wait complete. 2024-05-11T08:06:17.076 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.023+0000 7f220cd04640 1 -- 172.21.15.167:0/1271819819 wait complete. 2024-05-11T08:06:17.077 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.024+0000 7f80a3fff640 1 -- 172.21.15.167:0/660165834 <== osd.5 v2:172.21.15.204:6824/2921847521 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f80b400b2f0 con 0x7f80a807faa0 2024-05-11T08:06:17.077 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.024+0000 7f124cff9640 1 -- 172.21.15.167:0/3826782190 <== osd.3 v2:172.21.15.167:6824/453689293 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f1234002dd0 con 0x7f123007faa0 2024-05-11T08:06:17.077 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.023+0000 7f821db51640 1 -- 172.21.15.167:0/2608700135 --> [v2:172.21.15.167:6800/3242166900,v1:172.21.15.167:6801/3242166900] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f82180694b0 con 0x7f820407fa50 2024-05-11T08:06:17.077 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.024+0000 7f81fffff640 1 -- 172.21.15.167:0/2608700135 <== osd.2 v2:172.21.15.167:6800/3242166900 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f82180694b0 con 0x7f820407fa50 2024-05-11T08:06:17.078 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.024+0000 7f80a1ffb640 1 -- 172.21.15.167:0/660165834 >> [v2:172.21.15.204:6824/2921847521,v1:172.21.15.204:6825/2921847521] conn(0x7f80a807faa0 msgr2=0x7f80a8081ee0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:17.078 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.024+0000 7f80a1ffb640 1 --2- 172.21.15.167:0/660165834 >> [v2:172.21.15.204:6824/2921847521,v1:172.21.15.204:6825/2921847521] conn(0x7f80a807faa0 0x7f80a8081ee0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:17.078 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.024+0000 7fe6aded1640 1 -- 172.21.15.167:0/2339140798 --> [v2:172.21.15.204:6808/1949534184,v1:172.21.15.204:6809/1949534184] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe688002dd0 con 0x7fe69407fad0 2024-05-11T08:06:17.079 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.024+0000 7f80a1ffb640 1 -- 172.21.15.167:0/660165834 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f80a80763a0 msgr2=0x7f80a8078860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:17.079 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.024+0000 7f80a1ffb640 1 --2- 172.21.15.167:0/660165834 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f80a80763a0 0x7f80a8078860 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f80ac009670 tx=0x7f80ac006bd0 comp rx=0 tx=0).stop 2024-05-11T08:06:17.079 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.025+0000 7f80a1ffb640 1 -- 172.21.15.167:0/660165834 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f80b404f4c0 msgr2=0x7f80b4036d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:17.079 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.025+0000 7f80a1ffb640 1 --2- 172.21.15.167:0/660165834 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f80b404f4c0 0x7f80b4036d60 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f80b000e9e0 tx=0x7f80b000eea0 comp rx=0 tx=0).stop 2024-05-11T08:06:17.079 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.025+0000 7f80bb7fe640 1 -- 172.21.15.167:0/660165834 reap_dead start 2024-05-11T08:06:17.080 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.025+0000 7f81fdffb640 1 -- 172.21.15.167:0/2608700135 >> [v2:172.21.15.167:6800/3242166900,v1:172.21.15.167:6801/3242166900] conn(0x7f820407fa50 msgr2=0x7f8204081e90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:17.080 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.025+0000 7f81fdffb640 1 --2- 172.21.15.167:0/2608700135 >> [v2:172.21.15.167:6800/3242166900,v1:172.21.15.167:6801/3242166900] conn(0x7f820407fa50 0x7f8204081e90 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:17.080 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.025+0000 7f80a1ffb640 1 -- 172.21.15.167:0/660165834 shutdown_connections 2024-05-11T08:06:17.080 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.025+0000 7f80a1ffb640 1 -- 172.21.15.167:0/660165834 >> 172.21.15.167:0/660165834 conn(0x7f80b400a3f0 msgr2=0x7f80b404a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:17.080 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.024+0000 7f4e9e7fc640 1 -- 172.21.15.167:0/3859619623 --> [v2:172.21.15.167:6804/3863307356,v1:172.21.15.167:6806/3863307356] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4e8c002dd0 con 0x7f4e9807fb20 2024-05-11T08:06:17.081 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.025+0000 7f81fdffb640 1 -- 172.21.15.167:0/2608700135 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f82040762d0 msgr2=0x7f8204078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:17.081 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.025+0000 7f81fdffb640 1 --2- 172.21.15.167:0/2608700135 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f82040762d0 0x7f8204078790 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f8208000f60 tx=0x7f8208009040 comp rx=0 tx=0).stop 2024-05-11T08:06:17.081 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.025+0000 7f81fdffb640 1 -- 172.21.15.167:0/2608700135 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f821815b410 msgr2=0x7f821814f8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:17.081 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.025+0000 7f81fdffb640 1 --2- 172.21.15.167:0/2608700135 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f821815b410 0x7f821814f8c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f8210030000 tx=0x7f8210004a40 comp rx=0 tx=0).stop 2024-05-11T08:06:17.081 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.025+0000 7f8217fff640 1 -- 172.21.15.167:0/2608700135 reap_dead start 2024-05-11T08:06:17.081 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.025+0000 7f81fdffb640 1 -- 172.21.15.167:0/2608700135 shutdown_connections 2024-05-11T08:06:17.081 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.025+0000 7f81fdffb640 1 -- 172.21.15.167:0/2608700135 >> 172.21.15.167:0/2608700135 conn(0x7f821806c450 msgr2=0x7f8218073290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:17.081 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.025+0000 7f124ffff640 1 -- 172.21.15.167:0/3826782190 >> [v2:172.21.15.167:6824/453689293,v1:172.21.15.167:6825/453689293] conn(0x7f123007faa0 msgr2=0x7f1230081ee0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:17.081 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.025+0000 7f124ffff640 1 --2- 172.21.15.167:0/3826782190 >> [v2:172.21.15.167:6824/453689293,v1:172.21.15.167:6825/453689293] conn(0x7f123007faa0 0x7f1230081ee0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:17.082 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.025+0000 7fe68ffff640 1 -- 172.21.15.167:0/2339140798 <== osd.7 v2:172.21.15.204:6808/1949534184 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fe688002dd0 con 0x7fe69407fad0 2024-05-11T08:06:17.082 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.025+0000 7f80a1ffb640 1 -- 172.21.15.167:0/660165834 shutdown_connections 2024-05-11T08:06:17.082 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.025+0000 7f80a1ffb640 1 -- 172.21.15.167:0/660165834 wait complete. 2024-05-11T08:06:17.083 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.025+0000 7f124ffff640 1 -- 172.21.15.167:0/3826782190 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f1230076320 msgr2=0x7f12300787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:17.083 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.025+0000 7f124ffff640 1 --2- 172.21.15.167:0/3826782190 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f1230076320 0x7f12300787e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1240006fd0 tx=0x7f1240009060 comp rx=0 tx=0).stop 2024-05-11T08:06:17.083 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.025+0000 7f124ffff640 1 -- 172.21.15.167:0/3826782190 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1250160ac0 msgr2=0x7f125006e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:17.083 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.025+0000 7f124ffff640 1 --2- 172.21.15.167:0/3826782190 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1250160ac0 0x7f125006e410 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f124800be50 tx=0x7f124800bf40 comp rx=0 tx=0).stop 2024-05-11T08:06:17.083 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.025+0000 7f81fdffb640 1 -- 172.21.15.167:0/2608700135 shutdown_connections 2024-05-11T08:06:17.083 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.025+0000 7f4eb0ff9640 1 -- 172.21.15.167:0/3859619623 <== osd.0 v2:172.21.15.167:6804/3863307356 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f4e8c002dd0 con 0x7f4e9807fb20 2024-05-11T08:06:17.084 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.025+0000 7f81fdffb640 1 -- 172.21.15.167:0/2608700135 wait complete. 2024-05-11T08:06:17.084 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.026+0000 7f124f7fe640 1 -- 172.21.15.167:0/3826782190 reap_dead start 2024-05-11T08:06:17.085 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.026+0000 7f124ffff640 1 -- 172.21.15.167:0/3826782190 shutdown_connections 2024-05-11T08:06:17.085 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.026+0000 7f124ffff640 1 -- 172.21.15.167:0/3826782190 >> 172.21.15.167:0/3826782190 conn(0x7f12500d1010 msgr2=0x7f12500b50c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:17.085 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.026+0000 7f124ffff640 1 -- 172.21.15.167:0/3826782190 shutdown_connections 2024-05-11T08:06:17.085 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.026+0000 7f124ffff640 1 -- 172.21.15.167:0/3826782190 wait complete. 2024-05-11T08:06:17.086 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.026+0000 7fe6aded1640 1 -- 172.21.15.167:0/2339140798 >> [v2:172.21.15.204:6808/1949534184,v1:172.21.15.204:6809/1949534184] conn(0x7fe69407fad0 msgr2=0x7fe694081f10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:17.086 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.026+0000 7fe6aded1640 1 --2- 172.21.15.167:0/2339140798 >> [v2:172.21.15.204:6808/1949534184,v1:172.21.15.204:6809/1949534184] conn(0x7fe69407fad0 0x7fe694081f10 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:17.086 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.026+0000 7fe6aded1640 1 -- 172.21.15.167:0/2339140798 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fe6940763d0 msgr2=0x7fe694078890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:17.086 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.026+0000 7fe6aded1640 1 --2- 172.21.15.167:0/2339140798 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fe6940763d0 0x7fe694078890 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe69803d000 tx=0x7fe698008030 comp rx=0 tx=0).stop 2024-05-11T08:06:17.086 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.026+0000 7fe6aded1640 1 -- 172.21.15.167:0/2339140798 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe6a80d0100 msgr2=0x7fe6a815d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:17.086 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.026+0000 7fe6aded1640 1 --2- 172.21.15.167:0/2339140798 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe6a80d0100 0x7fe6a815d680 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe69c00d800 tx=0x7fe69c00dcc0 comp rx=0 tx=0).stop 2024-05-11T08:06:17.086 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.026+0000 7fe6a7fff640 1 -- 172.21.15.167:0/2339140798 reap_dead start 2024-05-11T08:06:17.086 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.026+0000 7fe6aded1640 1 -- 172.21.15.167:0/2339140798 shutdown_connections 2024-05-11T08:06:17.086 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.026+0000 7fe6aded1640 1 -- 172.21.15.167:0/2339140798 >> 172.21.15.167:0/2339140798 conn(0x7fe6a80d2fd0 msgr2=0x7fe6a815b7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:17.086 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.027+0000 7fe6aded1640 1 -- 172.21.15.167:0/2339140798 shutdown_connections 2024-05-11T08:06:17.086 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.027+0000 7fe6aded1640 1 -- 172.21.15.167:0/2339140798 wait complete. 2024-05-11T08:06:17.087 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.027+0000 7f4e9e7fc640 1 -- 172.21.15.167:0/3859619623 >> [v2:172.21.15.167:6804/3863307356,v1:172.21.15.167:6806/3863307356] conn(0x7f4e9807fb20 msgr2=0x7f4e98081f60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:17.087 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.027+0000 7f4e9e7fc640 1 --2- 172.21.15.167:0/3859619623 >> [v2:172.21.15.167:6804/3863307356,v1:172.21.15.167:6806/3863307356] conn(0x7f4e9807fb20 0x7f4e98081f60 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:17.087 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.027+0000 7f4e9e7fc640 1 -- 172.21.15.167:0/3859619623 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f4e980763a0 msgr2=0x7f4e98078860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:17.087 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.027+0000 7f4e9e7fc640 1 --2- 172.21.15.167:0/3859619623 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f4e980763a0 0x7f4e98078860 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4ea8006c80 tx=0x7f4ea8008040 comp rx=0 tx=0).stop 2024-05-11T08:06:17.087 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.027+0000 7f4e9e7fc640 1 -- 172.21.15.167:0/3859619623 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4eac0e30b0 msgr2=0x7f4eac052970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:17.087 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.027+0000 7f4e9e7fc640 1 --2- 172.21.15.167:0/3859619623 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4eac0e30b0 0x7f4eac052970 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f4eb407b800 tx=0x7f4eb4076680 comp rx=0 tx=0).stop 2024-05-11T08:06:17.088 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.027+0000 7fae14558640 1 -- 172.21.15.167:0/1466469987 --> [v2:172.21.15.167:6816/1585024048,v1:172.21.15.167:6817/1585024048] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fae0c141c30 con 0x7fadf007fa90 2024-05-11T08:06:17.088 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.027+0000 7f4eb8e7e640 1 -- 172.21.15.167:0/3859619623 reap_dead start 2024-05-11T08:06:17.088 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.027+0000 7f4e9e7fc640 1 -- 172.21.15.167:0/3859619623 shutdown_connections 2024-05-11T08:06:17.088 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.027+0000 7f4e9e7fc640 1 -- 172.21.15.167:0/3859619623 >> 172.21.15.167:0/3859619623 conn(0x7f4eac0559d0 msgr2=0x7f4eac0d7e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:17.088 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.028+0000 7fae02ffd640 1 -- 172.21.15.167:0/1466469987 <== osd.1 v2:172.21.15.167:6816/1585024048 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fae0c141c30 con 0x7fadf007fa90 2024-05-11T08:06:17.089 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.029+0000 7fae14558640 1 -- 172.21.15.167:0/1466469987 >> [v2:172.21.15.167:6816/1585024048,v1:172.21.15.167:6817/1585024048] conn(0x7fadf007fa90 msgr2=0x7fadf0081ed0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:17.089 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.029+0000 7fae14558640 1 --2- 172.21.15.167:0/1466469987 >> [v2:172.21.15.167:6816/1585024048,v1:172.21.15.167:6817/1585024048] conn(0x7fadf007fa90 0x7fadf0081ed0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:17.089 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.029+0000 7f4e9e7fc640 1 -- 172.21.15.167:0/3859619623 shutdown_connections 2024-05-11T08:06:17.089 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.029+0000 7f4e9e7fc640 1 -- 172.21.15.167:0/3859619623 wait complete. 2024-05-11T08:06:17.090 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.029+0000 7fae14558640 1 -- 172.21.15.167:0/1466469987 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fadf00763a0 msgr2=0x7fadf0078860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:17.090 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.029+0000 7fae14558640 1 --2- 172.21.15.167:0/1466469987 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fadf00763a0 0x7fadf0078860 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fae08001920 tx=0x7fae08007040 comp rx=0 tx=0).stop 2024-05-11T08:06:17.090 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.029+0000 7fae14558640 1 -- 172.21.15.167:0/1466469987 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fae0c141ee0 msgr2=0x7fae0c144050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:17.090 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.029+0000 7fae14558640 1 --2- 172.21.15.167:0/1466469987 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fae0c141ee0 0x7fae0c144050 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fadfc02a420 tx=0x7fadfc004700 comp rx=0 tx=0).stop 2024-05-11T08:06:17.091 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.029+0000 7fae12af5640 1 -- 172.21.15.167:0/1466469987 reap_dead start 2024-05-11T08:06:17.091 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.030+0000 7fae14558640 1 -- 172.21.15.167:0/1466469987 shutdown_connections 2024-05-11T08:06:17.091 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.030+0000 7fae14558640 1 -- 172.21.15.167:0/1466469987 >> 172.21.15.167:0/1466469987 conn(0x7fae0c0d2010 msgr2=0x7fae0c0cf620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:17.091 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.030+0000 7fae14558640 1 -- 172.21.15.167:0/1466469987 shutdown_connections 2024-05-11T08:06:17.091 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.031+0000 7fae14558640 1 -- 172.21.15.167:0/1466469987 wait complete. 2024-05-11T08:06:17.092 INFO:teuthology.orchestra.run.smithi167.stdout:64424509450 2024-05-11T08:06:17.092 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-05-11T08:06:17.093 INFO:teuthology.orchestra.run.smithi167.stdout:64424509450 2024-05-11T08:06:17.093 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-05-11T08:06:17.094 INFO:teuthology.orchestra.run.smithi167.stdout:64424509450 2024-05-11T08:06:17.094 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-05-11T08:06:17.095 INFO:teuthology.orchestra.run.smithi167.stdout:55834574859 2024-05-11T08:06:17.095 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-05-11T08:06:17.096 INFO:teuthology.orchestra.run.smithi167.stdout:64424509450 2024-05-11T08:06:17.096 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-05-11T08:06:17.097 INFO:teuthology.orchestra.run.smithi167.stdout:64424509451 2024-05-11T08:06:17.097 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-05-11T08:06:17.098 INFO:teuthology.orchestra.run.smithi167.stdout:64424509451 2024-05-11T08:06:17.098 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-05-11T08:06:17.099 INFO:teuthology.orchestra.run.smithi167.stdout:64424509450 2024-05-11T08:06:17.099 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-05-11T08:06:17.302 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.301+0000 7f8ad4454640 1 Processor -- start 2024-05-11T08:06:17.303 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.301+0000 7f8ad4454640 1 -- start start 2024-05-11T08:06:17.303 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.302+0000 7fb2db4c2640 1 Processor -- start 2024-05-11T08:06:17.303 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.302+0000 7f8ad4454640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f8acc0ade30 0x7f8acc0ae210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:17.304 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.303+0000 7f8ad4454640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8acc15b5d0 0x7f8acc15ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:17.304 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.303+0000 7f8ad4454640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8acc0ae750 con 0x7f8acc0d08f0 2024-05-11T08:06:17.304 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.303+0000 7f8ad4454640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f8acc15fdf0 con 0x7f8acc0ade30 2024-05-11T08:06:17.304 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.303+0000 7f8ad4454640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f8acc15f030 con 0x7f8acc15b5d0 2024-05-11T08:06:17.304 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.302+0000 7fb2db4c2640 1 -- start start 2024-05-11T08:06:17.304 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.303+0000 7f8ad21f0640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f8acc0ade30 0x7f8acc0ae210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:17.305 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.303+0000 7f8ad19ef640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f8acc0d08f0 0x7f8acc0706a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.167:45666/0 (socket says 172.21.15.167:45666) 2024-05-11T08:06:17.305 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.303+0000 7f8ad19ef640 1 -- 172.21.15.167:0/1016011483 learned_addr learned my addr 172.21.15.167:0/1016011483 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T08:06:17.305 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.303+0000 7f566eb22640 1 Processor -- start 2024-05-11T08:06:17.305 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.303+0000 7f8ad29f1640 1 --2- 172.21.15.167:0/1016011483 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8acc15b5d0 0x7f8acc15ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:17.306 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.304+0000 7f5b1acc2640 1 Processor -- start 2024-05-11T08:06:17.306 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.303+0000 7f566eb22640 1 -- start start 2024-05-11T08:06:17.306 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.303+0000 7f8ad21f0640 1 -- 172.21.15.167:0/1016011483 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8acc15b5d0 msgr2=0x7f8acc15ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:17.306 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.303+0000 7f8ad21f0640 1 --2- 172.21.15.167:0/1016011483 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8acc15b5d0 0x7f8acc15ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:17.307 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.304+0000 7f21497e4640 1 Processor -- start 2024-05-11T08:06:17.307 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.305+0000 7f21497e4640 1 -- start start 2024-05-11T08:06:17.307 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.303+0000 7f8ad21f0640 1 -- 172.21.15.167:0/1016011483 >> v1:172.21.15.204:6789/0 conn(0x7f8acc0d08f0 legacy=0x7f8acc0706a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:17.307 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.303+0000 7f8ad21f0640 1 -- 172.21.15.167:0/1016011483 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8acc15f310 con 0x7f8acc0ade30 2024-05-11T08:06:17.308 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.305+0000 7f566eb22640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f566815f3f0 0x7f56680b83a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:17.308 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.305+0000 7f8ad21f0640 1 --2- 172.21.15.167:0/1016011483 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8acc0ade30 0x7f8acc0ae210 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8abc00ba40 tx=0x7f8abc02c7e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc98726906a0a296 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:17.308 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.305+0000 7f566eb22640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f56680d0640 0x7f56680b88e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:17.309 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.304+0000 7fb2db4c2640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fb2cc054b40 0x7fb2cc051ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:17.309 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.304+0000 7fb2db4c2640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb2cc053bf0 0x7fb2cc052020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:17.309 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.304+0000 7fb2db4c2640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2cc054fd0 con 0x7fb2cc0e12a0 2024-05-11T08:06:17.309 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.304+0000 7fb2db4c2640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fb2cc0e2690 con 0x7fb2cc054b40 2024-05-11T08:06:17.309 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.304+0000 7fb2db4c2640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fb2cc0552b0 con 0x7fb2cc053bf0 2024-05-11T08:06:17.309 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.304+0000 7fb2d9cbf640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fb2cc054b40 0x7fb2cc051ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:17.309 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.304+0000 7fb2da4c0640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7fb2cc0e12a0 0x7fb2cc0534e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:60800/0 (socket says 172.21.15.167:60800) 2024-05-11T08:06:17.309 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.304+0000 7fb2d9cbf640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fb2cc054b40 0x7fb2cc051ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.167:37176/0 (socket says 172.21.15.167:37176) 2024-05-11T08:06:17.309 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.304+0000 7fb2da4c0640 1 -- 172.21.15.167:0/1450142401 learned_addr learned my addr 172.21.15.167:0/1450142401 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T08:06:17.309 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.304+0000 7fb2d9cbf640 1 -- 172.21.15.167:0/1450142401 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb2cc053bf0 msgr2=0x7fb2cc052020 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-11T08:06:17.309 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.304+0000 7fb2d9cbf640 1 --2- 172.21.15.167:0/1450142401 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb2cc053bf0 0x7fb2cc052020 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:17.309 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.304+0000 7fb2d9cbf640 1 -- 172.21.15.167:0/1450142401 >> v1:172.21.15.167:6789/0 conn(0x7fb2cc0e12a0 legacy=0x7fb2cc0534e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:17.310 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.305+0000 7fb2d9cbf640 1 -- 172.21.15.167:0/1450142401 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2cc055590 con 0x7fb2cc054b40 2024-05-11T08:06:17.310 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.305+0000 7f566eb22640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f566806c7f0 0x7f56680d3e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:17.310 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.305+0000 7f566eb22640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f566815f7d0 con 0x7f566815f3f0 2024-05-11T08:06:17.310 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.305+0000 7f566eb22640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f56680d2db0 con 0x7f566806c7f0 2024-05-11T08:06:17.310 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.305+0000 7f566eb22640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f56680d3090 con 0x7f56680d0640 2024-05-11T08:06:17.311 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.305+0000 7f5b1acc2640 1 -- start start 2024-05-11T08:06:17.311 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.305+0000 7f566c8be640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f566815f3f0 0x7f56680b83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:17.311 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.305+0000 7f4d80a1d640 1 Processor -- start 2024-05-11T08:06:17.312 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.305+0000 7f566c8be640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f566815f3f0 0x7f56680b83a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:54336/0 (socket says 172.21.15.167:54336) 2024-05-11T08:06:17.312 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.305+0000 7f566c8be640 1 -- 172.21.15.167:0/3996714720 learned_addr learned my addr 172.21.15.167:0/3996714720 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:17.312 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.305+0000 7f8ad11ee640 1 -- 172.21.15.167:0/1016011483 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8abc031710 con 0x7f8acc0ade30 2024-05-11T08:06:17.312 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.305+0000 7f8ad11ee640 1 -- 172.21.15.167:0/1016011483 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8abc0318b0 con 0x7f8acc0ade30 2024-05-11T08:06:17.312 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.305+0000 7f8ad11ee640 1 -- 172.21.15.167:0/1016011483 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8abc031c60 con 0x7f8acc0ade30 2024-05-11T08:06:17.312 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.305+0000 7fb2d9cbf640 1 --2- 172.21.15.167:0/1450142401 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb2cc054b40 0x7fb2cc051ae0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb2d00097a0 tx=0x7fb2d002a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=95f095aef42636ee server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:17.313 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.306+0000 7f566c8be640 1 -- 172.21.15.167:0/3996714720 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f56680d0640 msgr2=0x7f56680b88e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:17.313 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.306+0000 7f566c8be640 1 --2- 172.21.15.167:0/3996714720 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f56680d0640 0x7f56680b88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:17.313 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.306+0000 7f566c8be640 1 -- 172.21.15.167:0/3996714720 >> v2:172.21.15.204:3300/0 conn(0x7f566806c7f0 msgr2=0x7f56680d3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:17.313 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.306+0000 7f566c8be640 1 --2- 172.21.15.167:0/3996714720 >> v2:172.21.15.204:3300/0 conn(0x7f566806c7f0 0x7f56680d3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:17.313 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.306+0000 7f566c8be640 1 -- 172.21.15.167:0/3996714720 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56680d3370 con 0x7f566815f3f0 2024-05-11T08:06:17.314 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.305+0000 7f4d80a1d640 1 -- start start 2024-05-11T08:06:17.314 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.306+0000 7f8a1110f640 1 Processor -- start 2024-05-11T08:06:17.314 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.306+0000 7f8a1110f640 1 -- start start 2024-05-11T08:06:17.315 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.306+0000 7f21497e4640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f214415f160 0x7f21440d05c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:17.315 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.306+0000 7f2143fff640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f214415f160 0x7f21440d05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:17.315 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.306+0000 7f2143fff640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f214415f160 0x7f21440d05c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.167:37200/0 (socket says 172.21.15.167:37200) 2024-05-11T08:06:17.315 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.306+0000 7f21437fe640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f21440cf410 0x7f21440b43e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.167:45672/0 (socket says 172.21.15.167:45672) 2024-05-11T08:06:17.315 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.306+0000 7f5b1acc2640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f5b140d0d10 0x7f5b1415ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:17.315 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.306+0000 7f5b1acc2640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5b1406c450 0x7f5b1406c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:17.315 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.306+0000 7f5b1acc2640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b1406cdf0 con 0x7f5b1415f170 2024-05-11T08:06:17.316 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.306+0000 7f2143fff640 1 -- 172.21.15.167:0/613893642 learned_addr learned my addr 172.21.15.167:0/613893642 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:17.316 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.307+0000 7f21497e4640 1 -- 172.21.15.167:0/613893642 --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21440cfc00 con 0x7f2144072630 2024-05-11T08:06:17.316 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.306+0000 7f5b1acc2640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f5b1415ef80 con 0x7f5b140d0d10 2024-05-11T08:06:17.316 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.306+0000 7fb2d94be640 1 -- 172.21.15.167:0/1450142401 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2d0037070 con 0x7fb2cc054b40 2024-05-11T08:06:17.316 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.306+0000 7fb2d94be640 1 -- 172.21.15.167:0/1450142401 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2d0032440 con 0x7fb2cc054b40 2024-05-11T08:06:17.316 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.306+0000 7fb2d94be640 1 -- 172.21.15.167:0/1450142401 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2d0032810 con 0x7fb2cc054b40 2024-05-11T08:06:17.317 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.307+0000 7f21497e4640 1 -- 172.21.15.167:0/613893642 --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21440aea70 con 0x7f21440cf410 2024-05-11T08:06:17.317 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.307+0000 7f21497e4640 1 -- 172.21.15.167:0/613893642 --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f21440ae820 con 0x7f214415f160 2024-05-11T08:06:17.317 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.306+0000 7f5b1acc2640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f5b140d10f0 con 0x7f5b1406c450 2024-05-11T08:06:17.317 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.307+0000 7f5b19cc0640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f5b140d0d10 0x7f5b1415ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:17.317 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.307+0000 7f5b19cc0640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f5b140d0d10 0x7f5b1415ea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:54338/0 (socket says 172.21.15.167:54338) 2024-05-11T08:06:17.318 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.307+0000 7fb2db4c2640 1 -- 172.21.15.167:0/1450142401 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb2cc054b40 msgr2=0x7fb2cc051ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:17.318 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.307+0000 7fb2db4c2640 1 --2- 172.21.15.167:0/1450142401 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb2cc054b40 0x7fb2cc051ae0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb2d00097a0 tx=0x7fb2d002a1a0 comp rx=0 tx=0).stop 2024-05-11T08:06:17.318 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.307+0000 7f5b19cc0640 1 -- 172.21.15.167:0/2806661591 learned_addr learned my addr 172.21.15.167:0/2806661591 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:17.318 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.307+0000 7fca3b59e640 1 Processor -- start 2024-05-11T08:06:17.319 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.307+0000 7f566c8be640 1 --2- 172.21.15.167:0/3996714720 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f566815f3f0 0x7f56680b83a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f56680cf970 tx=0x7f565c02a1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4fbb9c580d67a03 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:17.319 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.307+0000 7f8ad4454640 1 -- 172.21.15.167:0/1016011483 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8acc0ade30 msgr2=0x7f8acc0ae210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:17.319 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.307+0000 7f8ad4454640 1 --2- 172.21.15.167:0/1016011483 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8acc0ade30 0x7f8acc0ae210 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8abc00ba40 tx=0x7f8abc02c7e0 comp rx=0 tx=0).stop 2024-05-11T08:06:17.319 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.307+0000 7f8ad4454640 1 -- 172.21.15.167:0/1016011483 shutdown_connections 2024-05-11T08:06:17.319 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.307+0000 7f8ad4454640 1 --2- 172.21.15.167:0/1016011483 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8acc15b5d0 0x7f8acc15ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:17.319 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.307+0000 7f8ad4454640 1 --2- 172.21.15.167:0/1016011483 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8acc0ade30 0x7f8acc0ae210 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:17.319 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.307+0000 7f8ad4454640 1 -- 172.21.15.167:0/1016011483 >> 172.21.15.167:0/1016011483 conn(0x7f8acc0d1200 msgr2=0x7f8acc0d1610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:17.320 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.307+0000 7fb2db4c2640 1 -- 172.21.15.167:0/1450142401 shutdown_connections 2024-05-11T08:06:17.320 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.307+0000 7fb2db4c2640 1 --2- 172.21.15.167:0/1450142401 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb2cc053bf0 0x7fb2cc052020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:17.320 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.307+0000 7fb2db4c2640 1 --2- 172.21.15.167:0/1450142401 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb2cc054b40 0x7fb2cc051ae0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:17.320 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.307+0000 7fb2db4c2640 1 -- 172.21.15.167:0/1450142401 >> 172.21.15.167:0/1450142401 conn(0x7fb2cc020030 msgr2=0x7fb2cc056d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:17.320 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.307+0000 7f8a1110f640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f8a0c0b7e70 0x7f8a0c0cf670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:17.320 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.307+0000 7f8a1110f640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8a0c06cee0 0x7f8a0c0d3800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:17.320 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.307+0000 7f8a1110f640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a0c1638f0 con 0x7f8a0c163510 2024-05-11T08:06:17.320 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.307+0000 7f8a1110f640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f8a0c0d0c20 con 0x7f8a0c0b7e70 2024-05-11T08:06:17.321 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.307+0000 7f8a1110f640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f8a0c0d3370 con 0x7f8a0c06cee0 2024-05-11T08:06:17.321 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.307+0000 7f8a1090e640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8a0c06cee0 0x7f8a0c0d3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:17.321 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.307+0000 7f8a1090e640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8a0c06cee0 0x7f8a0c0d3800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.167:37246/0 (socket says 172.21.15.167:37246) 2024-05-11T08:06:17.321 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.307+0000 7f8a1090e640 1 -- 172.21.15.167:0/1708191113 learned_addr learned my addr 172.21.15.167:0/1708191113 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:17.321 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.307+0000 7f5b19cc0640 1 -- 172.21.15.167:0/2806661591 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5b1406c450 msgr2=0x7f5b1406c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:17.321 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.307+0000 7f5b194bf640 1 --2- 172.21.15.167:0/2806661591 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5b1406c450 0x7f5b1406c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:17.321 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.307+0000 7f5b19cc0640 1 --2- 172.21.15.167:0/2806661591 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5b1406c450 0x7f5b1406c8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:17.321 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.307+0000 7f56677fe640 1 -- 172.21.15.167:0/3996714720 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f565c037070 con 0x7f566815f3f0 2024-05-11T08:06:17.322 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.307+0000 7f4d80a1d640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f4d7c06d460 0x7f4d7c06d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:17.322 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.307+0000 7f4d80a1d640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f4d7c0b47d0 0x7f4d7c0b4c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:17.322 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.307+0000 7f4d80a1d640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d7c0cfc10 con 0x7f4d7c14c8a0 2024-05-11T08:06:17.322 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.307+0000 7f4d80a1d640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f4d7c0cfef0 con 0x7f4d7c0b47d0 2024-05-11T08:06:17.322 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.307+0000 7f4d80a1d640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f4d7c0d01d0 con 0x7f4d7c06d460 2024-05-11T08:06:17.322 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.307+0000 7f56677fe640 1 -- 172.21.15.167:0/3996714720 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f565c032440 con 0x7f566815f3f0 2024-05-11T08:06:17.323 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.308+0000 7fca3b59e640 1 -- start start 2024-05-11T08:06:17.323 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.307+0000 7f2143fff640 1 -- 172.21.15.167:0/613893642 >> v1:172.21.15.204:6789/0 conn(0x7f21440cf410 legacy=0x7f21440b43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:17.323 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.307+0000 7f2143fff640 1 -- 172.21.15.167:0/613893642 >> v1:172.21.15.167:6789/0 conn(0x7f2144072630 legacy=0x7f21440af550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:17.323 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.307+0000 7f2143fff640 1 -- 172.21.15.167:0/613893642 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21440aed50 con 0x7f214415f160 2024-05-11T08:06:17.323 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.308+0000 7f4d7a59c640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f4d7c14c8a0 0x7f4d7c0cf1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:60818/0 (socket says 172.21.15.167:60818) 2024-05-11T08:06:17.324 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.308+0000 7f4d7a59c640 1 -- 172.21.15.167:0/3978438577 learned_addr learned my addr 172.21.15.167:0/3978438577 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T08:06:17.324 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.308+0000 7f4d7ad9d640 1 --2- 172.21.15.167:0/3978438577 >> v2:172.21.15.167:3300/0 conn(0x7f4d7c0b47d0 0x7f4d7c0b4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:17.324 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.307+0000 7f5b19cc0640 1 -- 172.21.15.167:0/2806661591 >> v1:172.21.15.204:6789/0 conn(0x7f5b1415f170 legacy=0x7f5b1415f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:17.324 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.307+0000 7f5b19cc0640 1 -- 172.21.15.167:0/2806661591 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b140b41c0 con 0x7f5b140d0d10 2024-05-11T08:06:17.324 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.308+0000 7f56677fe640 1 -- 172.21.15.167:0/3996714720 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f565c032810 con 0x7f566815f3f0 2024-05-11T08:06:17.325 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.308+0000 7f4d79d9b640 1 --2- 172.21.15.167:0/3978438577 >> v2:172.21.15.204:3300/0 conn(0x7f4d7c06d460 0x7f4d7c06d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:17.325 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.308+0000 7f2143fff640 1 --2- 172.21.15.167:0/613893642 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f214415f160 0x7f21440d05c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f213c009870 tx=0x7f213c02a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=337eef6057304b84 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:17.325 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.307+0000 7f8ad4454640 1 -- 172.21.15.167:0/1016011483 shutdown_connections 2024-05-11T08:06:17.325 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.307+0000 7f8ad4454640 1 -- 172.21.15.167:0/1016011483 wait complete. 2024-05-11T08:06:17.325 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.308+0000 7f8a1090e640 1 -- 172.21.15.167:0/1708191113 >> v2:172.21.15.204:3300/0 conn(0x7f8a0c0b7e70 msgr2=0x7f8a0c0cf670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:17.326 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.308+0000 7f8a1090e640 1 --2- 172.21.15.167:0/1708191113 >> v2:172.21.15.204:3300/0 conn(0x7f8a0c0b7e70 0x7f8a0c0cf670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:17.326 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.308+0000 7f8a1090e640 1 -- 172.21.15.167:0/1708191113 >> v1:172.21.15.204:6789/0 conn(0x7f8a0c163510 legacy=0x7f8a0c06c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:17.326 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.308+0000 7f8a1090e640 1 -- 172.21.15.167:0/1708191113 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a0c0d0660 con 0x7f8a0c06cee0 2024-05-11T08:06:17.326 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.309+0000 7f4d7ad9d640 1 -- 172.21.15.167:0/3978438577 >> v2:172.21.15.204:3300/0 conn(0x7f4d7c06d460 msgr2=0x7f4d7c06d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:17.326 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.309+0000 7f4d7ad9d640 1 --2- 172.21.15.167:0/3978438577 >> v2:172.21.15.204:3300/0 conn(0x7f4d7c06d460 0x7f4d7c06d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:17.326 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.309+0000 7f4d7ad9d640 1 -- 172.21.15.167:0/3978438577 >> v1:172.21.15.167:6789/0 conn(0x7f4d7c14c8a0 legacy=0x7f4d7c0cf1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:17.326 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.309+0000 7f4d7ad9d640 1 -- 172.21.15.167:0/3978438577 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d7c0b42b0 con 0x7f4d7c0b47d0 2024-05-11T08:06:17.327 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.309+0000 7fb2db4c2640 1 -- 172.21.15.167:0/1450142401 shutdown_connections 2024-05-11T08:06:17.327 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.309+0000 7f566eb22640 1 -- 172.21.15.167:0/3996714720 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f566815f3f0 msgr2=0x7f56680b83a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:17.327 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.309+0000 7f8a1090e640 1 --2- 172.21.15.167:0/1708191113 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8a0c06cee0 0x7f8a0c0d3800 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c0d0da0 tx=0x7f8a0402a5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a92d2b39044885c server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:17.328 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.309+0000 7f2142ffd640 1 -- 172.21.15.167:0/613893642 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f213c037070 con 0x7f214415f160 2024-05-11T08:06:17.328 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.309+0000 7f2142ffd640 1 -- 172.21.15.167:0/613893642 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f213c032440 con 0x7f214415f160 2024-05-11T08:06:17.328 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.309+0000 7f2142ffd640 1 -- 172.21.15.167:0/613893642 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f213c037070 con 0x7f214415f160 2024-05-11T08:06:17.328 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.309+0000 7f5b19cc0640 1 --2- 172.21.15.167:0/2806661591 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5b140d0d10 0x7f5b1415ea40 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5b10009870 tx=0x7f5b1002a280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8cdeff78d5672eb server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:17.328 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.309+0000 7fca3b59e640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fca340e1090 0x7fca34038630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:17.329 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.309+0000 7f566eb22640 1 --2- 172.21.15.167:0/3996714720 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f566815f3f0 0x7f56680b83a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f56680cf970 tx=0x7f565c02a1d0 comp rx=0 tx=0).stop 2024-05-11T08:06:17.329 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.309+0000 7fca3b59e640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fca3403a5d0 0x7fca34005f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:17.329 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.309+0000 7f566eb22640 1 -- 172.21.15.167:0/3996714720 shutdown_connections 2024-05-11T08:06:17.329 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.309+0000 7f566eb22640 1 --2- 172.21.15.167:0/3996714720 >> v2:172.21.15.204:3300/0 conn(0x7f566806c7f0 0x7f56680d3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:17.329 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.309+0000 7f566eb22640 1 --2- 172.21.15.167:0/3996714720 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f56680d0640 0x7f56680b88e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:17.329 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.309+0000 7f566eb22640 1 --2- 172.21.15.167:0/3996714720 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f566815f3f0 0x7f56680b83a0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:17.330 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.309+0000 7f566eb22640 1 -- 172.21.15.167:0/3996714720 >> 172.21.15.167:0/3996714720 conn(0x7f56680d51e0 msgr2=0x7f56680d55c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:17.330 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.309+0000 7f8a0affd640 1 -- 172.21.15.167:0/1708191113 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a04036070 con 0x7f8a0c06cee0 2024-05-11T08:06:17.330 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.309+0000 7f566eb22640 1 -- 172.21.15.167:0/3996714720 shutdown_connections 2024-05-11T08:06:17.330 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.309+0000 7f8a0affd640 1 -- 172.21.15.167:0/1708191113 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a04031400 con 0x7f8a0c06cee0 2024-05-11T08:06:17.332 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.309+0000 7fca3a59c640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fca340e1090 0x7fca34038630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:17.332 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.309+0000 7fca3a59c640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fca340e1090 0x7fca34038630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:54368/0 (socket says 172.21.15.167:54368) 2024-05-11T08:06:17.332 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.309+0000 7fca3b59e640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fca3404f540 con 0x7fca34050dd0 2024-05-11T08:06:17.332 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.309+0000 7fca3b59e640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7fca3404f820 con 0x7fca340e1090 2024-05-11T08:06:17.332 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.309+0000 7fca3b59e640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fca3404fb00 con 0x7fca3403a5d0 2024-05-11T08:06:17.332 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.309+0000 7fca3a59c640 1 -- 172.21.15.167:0/1349352003 learned_addr learned my addr 172.21.15.167:0/1349352003 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:17.332 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.310+0000 7fca3a59c640 1 -- 172.21.15.167:0/1349352003 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fca3403a5d0 msgr2=0x7fca34005f90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-11T08:06:17.332 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.310+0000 7fca3a59c640 1 --2- 172.21.15.167:0/1349352003 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fca3403a5d0 0x7fca34005f90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:17.332 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.310+0000 7fca3a59c640 1 -- 172.21.15.167:0/1349352003 >> v1:172.21.15.167:6789/0 conn(0x7fca34050dd0 legacy=0x7fca34039ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:17.332 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.310+0000 7f5b18cbe640 1 -- 172.21.15.167:0/2806661591 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b10037070 con 0x7f5b140d0d10 2024-05-11T08:06:17.333 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.310+0000 7fca3a59c640 1 -- 172.21.15.167:0/1349352003 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca3404fde0 con 0x7fca340e1090 2024-05-11T08:06:17.333 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.310+0000 7f21497e4640 1 -- 172.21.15.167:0/613893642 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f214415f160 msgr2=0x7f21440d05c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:17.333 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.310+0000 7f21497e4640 1 --2- 172.21.15.167:0/613893642 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f214415f160 0x7f21440d05c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f213c009870 tx=0x7f213c02a1c0 comp rx=0 tx=0).stop 2024-05-11T08:06:17.333 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.310+0000 7f8a0affd640 1 -- 172.21.15.167:0/1708191113 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a040317d0 con 0x7f8a0c06cee0 2024-05-11T08:06:17.334 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.309+0000 7f566eb22640 1 -- 172.21.15.167:0/3996714720 wait complete. 2024-05-11T08:06:17.334 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.310+0000 7f21497e4640 1 -- 172.21.15.167:0/613893642 shutdown_connections 2024-05-11T08:06:17.334 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.310+0000 7f21497e4640 1 --2- 172.21.15.167:0/613893642 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f214415f160 0x7f21440d05c0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:17.334 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.310+0000 7f21497e4640 1 -- 172.21.15.167:0/613893642 >> 172.21.15.167:0/613893642 conn(0x7f21440b33d0 msgr2=0x7f21440d0b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:17.334 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.310+0000 7f8a1110f640 1 -- 172.21.15.167:0/1708191113 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8a0c06cee0 msgr2=0x7f8a0c0d3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:17.335 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.310+0000 7f8a1110f640 1 --2- 172.21.15.167:0/1708191113 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8a0c06cee0 0x7f8a0c0d3800 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c0d0da0 tx=0x7f8a0402a5d0 comp rx=0 tx=0).stop 2024-05-11T08:06:17.335 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.309+0000 7f8ad4454640 1 Processor -- start 2024-05-11T08:06:17.335 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.309+0000 7f8ad4454640 1 -- start start 2024-05-11T08:06:17.335 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.310+0000 7f8ad4454640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8acc0ade30 0x7f8acc14c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:17.335 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.310+0000 7fb2db4c2640 1 -- 172.21.15.167:0/1450142401 wait complete. 2024-05-11T08:06:17.336 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.310+0000 7f5b18cbe640 1 -- 172.21.15.167:0/2806661591 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b10032440 con 0x7f5b140d0d10 2024-05-11T08:06:17.336 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.310+0000 7f8ad4454640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8acc0d08f0 0x7f8acc0c80f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:17.336 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.310+0000 7f8ad4454640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8acc15b5d0 0x7f8acc0c8630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:17.336 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.310+0000 7f8a1110f640 1 -- 172.21.15.167:0/1708191113 shutdown_connections 2024-05-11T08:06:17.337 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.310+0000 7f8a1110f640 1 --2- 172.21.15.167:0/1708191113 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8a0c06cee0 0x7f8a0c0d3800 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:17.337 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.310+0000 7f8a1110f640 1 --2- 172.21.15.167:0/1708191113 >> v2:172.21.15.204:3300/0 conn(0x7f8a0c0b7e70 0x7f8a0c0cf670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:17.337 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.310+0000 7f8a1110f640 1 -- 172.21.15.167:0/1708191113 >> 172.21.15.167:0/1708191113 conn(0x7f8a0c0d5020 msgr2=0x7f8a0c0d5430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:17.337 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.311+0000 7f8a1110f640 1 -- 172.21.15.167:0/1708191113 shutdown_connections 2024-05-11T08:06:17.337 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.311+0000 7f8a1110f640 1 -- 172.21.15.167:0/1708191113 wait complete. 2024-05-11T08:06:17.337 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.311+0000 7fca3a59c640 1 --2- 172.21.15.167:0/1349352003 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca340e1090 0x7fca34038630 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fca3000dc90 tx=0x7fca3002e4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9a87af2a1a152ff server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:17.338 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.311+0000 7f8ad29f1640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8acc15b5d0 0x7f8acc0c8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:17.338 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.311+0000 7f8ad29f1640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8acc15b5d0 0x7f8acc0c8630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:54384/0 (socket says 172.21.15.167:54384) 2024-05-11T08:06:17.338 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.311+0000 7f4d7ad9d640 1 --2- 172.21.15.167:0/3978438577 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4d7c0b47d0 0x7f4d7c0b4c00 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c0ceb00 tx=0x7f4d7402a570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ac90689dc764815 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:17.338 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.311+0000 7f8ad21f0640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8acc0ade30 0x7f8acc14c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:17.338 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.311+0000 7f8ad21f0640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8acc0ade30 0x7f8acc14c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.167:37234/0 (socket says 172.21.15.167:37234) 2024-05-11T08:06:17.339 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.311+0000 7f5b1acc2640 1 -- 172.21.15.167:0/2806661591 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5b140d0d10 msgr2=0x7f5b1415ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:17.339 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.311+0000 7f5b1acc2640 1 --2- 172.21.15.167:0/2806661591 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5b140d0d10 0x7f5b1415ea40 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5b10009870 tx=0x7f5b1002a280 comp rx=0 tx=0).stop 2024-05-11T08:06:17.339 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.311+0000 7f8ad19ef640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8acc0d08f0 0x7f8acc0c80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:17.339 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.311+0000 7f21497e4640 1 -- 172.21.15.167:0/613893642 shutdown_connections 2024-05-11T08:06:17.339 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.311+0000 7f8ad19ef640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8acc0d08f0 0x7f8acc0c80f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.167:37250/0 (socket says 172.21.15.167:37250) 2024-05-11T08:06:17.340 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.311+0000 7f5b1acc2640 1 -- 172.21.15.167:0/2806661591 shutdown_connections 2024-05-11T08:06:17.340 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.311+0000 7f5b1acc2640 1 --2- 172.21.15.167:0/2806661591 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5b1406c450 0x7f5b1406c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:17.340 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.311+0000 7f5b1acc2640 1 --2- 172.21.15.167:0/2806661591 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5b140d0d10 0x7f5b1415ea40 secure :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5b10009870 tx=0x7f5b1002a280 comp rx=0 tx=0).stop 2024-05-11T08:06:17.340 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.311+0000 7f5b1acc2640 1 -- 172.21.15.167:0/2806661591 >> 172.21.15.167:0/2806661591 conn(0x7f5b14070f00 msgr2=0x7f5b14072810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:17.340 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.311+0000 7f8ad29f1640 1 -- 172.21.15.167:0/2979835464 learned_addr learned my addr 172.21.15.167:0/2979835464 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:17.341 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.311+0000 7f21497e4640 1 -- 172.21.15.167:0/613893642 wait complete. 2024-05-11T08:06:17.341 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.311+0000 7f4d7959a640 1 -- 172.21.15.167:0/3978438577 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d74007740 con 0x7f4d7c0b47d0 2024-05-11T08:06:17.341 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.311+0000 7fca3959a640 1 -- 172.21.15.167:0/1349352003 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca3003a070 con 0x7fca340e1090 2024-05-11T08:06:17.341 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.311+0000 7fca3959a640 1 -- 172.21.15.167:0/1349352003 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca3000b650 con 0x7fca340e1090 2024-05-11T08:06:17.341 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.311+0000 7fca3959a640 1 -- 172.21.15.167:0/1349352003 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca3003a070 con 0x7fca340e1090 2024-05-11T08:06:17.342 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.311+0000 7f8ad4454640 1 -- 172.21.15.167:0/2979835464 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8acc0cee10 con 0x7f8acc15b5d0 2024-05-11T08:06:17.342 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.311+0000 7f566eb22640 1 Processor -- start 2024-05-11T08:06:17.342 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.311+0000 7f8ad4454640 1 -- 172.21.15.167:0/2979835464 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8acc15f8d0 con 0x7f8acc0ade30 2024-05-11T08:06:17.342 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.311+0000 7f8ad4454640 1 -- 172.21.15.167:0/2979835464 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f8acc0694b0 con 0x7f8acc0d08f0 2024-05-11T08:06:17.343 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.311+0000 7f4d7959a640 1 -- 172.21.15.167:0/3978438577 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d740078e0 con 0x7f4d7c0b47d0 2024-05-11T08:06:17.343 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.311+0000 7f4d7959a640 1 -- 172.21.15.167:0/3978438577 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d74007cb0 con 0x7f4d7c0b47d0 2024-05-11T08:06:17.343 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.311+0000 7f566eb22640 1 -- start start 2024-05-11T08:06:17.343 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.312+0000 7f8ad29f1640 1 -- 172.21.15.167:0/2979835464 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8acc0d08f0 msgr2=0x7f8acc0c80f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:17.343 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.312+0000 7f8ad29f1640 1 --2- 172.21.15.167:0/2979835464 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8acc0d08f0 0x7f8acc0c80f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:17.344 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.312+0000 7f8ad29f1640 1 -- 172.21.15.167:0/2979835464 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8acc0ade30 msgr2=0x7f8acc14c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:17.344 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.312+0000 7f8ad29f1640 1 --2- 172.21.15.167:0/2979835464 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8acc0ade30 0x7f8acc14c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:17.344 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.312+0000 7f8ad29f1640 1 -- 172.21.15.167:0/2979835464 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8acc09da40 con 0x7f8acc15b5d0 2024-05-11T08:06:17.344 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.312+0000 7fca3b59e640 1 -- 172.21.15.167:0/1349352003 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca340e1090 msgr2=0x7fca34038630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:17.344 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.312+0000 7fca3b59e640 1 --2- 172.21.15.167:0/1349352003 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca340e1090 0x7fca34038630 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fca3000dc90 tx=0x7fca3002e4d0 comp rx=0 tx=0).stop 2024-05-11T08:06:17.344 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.312+0000 7f4d80a1d640 1 -- 172.21.15.167:0/3978438577 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4d7c0b47d0 msgr2=0x7f4d7c0b4c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:17.345 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.312+0000 7f4d80a1d640 1 --2- 172.21.15.167:0/3978438577 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4d7c0b47d0 0x7f4d7c0b4c00 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c0ceb00 tx=0x7f4d7402a570 comp rx=0 tx=0).stop 2024-05-11T08:06:17.345 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.312+0000 7f8a1110f640 1 Processor -- start 2024-05-11T08:06:17.345 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.312+0000 7fb2db4c2640 1 Processor -- start 2024-05-11T08:06:17.345 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.312+0000 7fb2db4c2640 1 -- start start 2024-05-11T08:06:17.345 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.312+0000 7f8ad29f1640 1 --2- 172.21.15.167:0/2979835464 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8acc15b5d0 0x7f8acc0c8630 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f8ac400e1c0 tx=0x7f8ac4009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:17.346 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.313+0000 7f4d80a1d640 1 -- 172.21.15.167:0/3978438577 shutdown_connections 2024-05-11T08:06:17.346 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.313+0000 7f4d80a1d640 1 --2- 172.21.15.167:0/3978438577 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4d7c0b47d0 0x7f4d7c0b4c00 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:17.346 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.313+0000 7f4d80a1d640 1 --2- 172.21.15.167:0/3978438577 >> v2:172.21.15.204:3300/0 conn(0x7f4d7c06d460 0x7f4d7c06d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:17.346 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.313+0000 7f4d80a1d640 1 -- 172.21.15.167:0/3978438577 >> 172.21.15.167:0/3978438577 conn(0x7f4d7c0cd8f0 msgr2=0x7f4d7c0cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:17.346 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.313+0000 7f21497e4640 1 Processor -- start 2024-05-11T08:06:17.346 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.313+0000 7f8a1110f640 1 -- start start 2024-05-11T08:06:17.347 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.313+0000 7fca3b59e640 1 -- 172.21.15.167:0/1349352003 shutdown_connections 2024-05-11T08:06:17.347 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.313+0000 7fca3b59e640 1 --2- 172.21.15.167:0/1349352003 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fca3403a5d0 0x7fca34005f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:17.347 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.313+0000 7fca3b59e640 1 --2- 172.21.15.167:0/1349352003 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca340e1090 0x7fca34038630 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:17.347 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.313+0000 7fca3b59e640 1 -- 172.21.15.167:0/1349352003 >> 172.21.15.167:0/1349352003 conn(0x7fca34009fb0 msgr2=0x7fca3400a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:17.347 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.313+0000 7f8ad21f0640 1 --2- 172.21.15.167:0/2979835464 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8acc0ade30 0x7f8acc14c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T08:06:17.348 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.313+0000 7fca3b59e640 1 -- 172.21.15.167:0/1349352003 shutdown_connections 2024-05-11T08:06:17.348 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.313+0000 7f8ad19ef640 1 --2- 172.21.15.167:0/2979835464 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8acc0d08f0 0x7f8acc0c80f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T08:06:17.348 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.313+0000 7f8ac2ffd640 1 -- 172.21.15.167:0/2979835464 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ac4014420 con 0x7f8acc15b5d0 2024-05-11T08:06:17.348 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.313+0000 7f21497e4640 1 -- start start 2024-05-11T08:06:17.349 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.313+0000 7f4d80a1d640 1 -- 172.21.15.167:0/3978438577 shutdown_connections 2024-05-11T08:06:17.349 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.313+0000 7f8ac2ffd640 1 -- 172.21.15.167:0/2979835464 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ac4019070 con 0x7f8acc15b5d0 2024-05-11T08:06:17.349 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.313+0000 7f566eb22640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f566806c7f0 0x7f56680cf160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:17.349 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.313+0000 7f566eb22640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f56680d0640 0x7f56680c6820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:17.349 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.313+0000 7fca3b59e640 1 -- 172.21.15.167:0/1349352003 wait complete. 2024-05-11T08:06:17.350 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.313+0000 7f566eb22640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f566815f3f0 0x7f56680c6d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:17.350 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.313+0000 7f566eb22640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f566806da70 con 0x7f56680d0640 2024-05-11T08:06:17.350 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.313+0000 7f4d80a1d640 1 -- 172.21.15.167:0/3978438577 wait complete. 2024-05-11T08:06:17.351 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.313+0000 7f566eb22640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f56680d3930 con 0x7f566815f3f0 2024-05-11T08:06:17.351 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.314+0000 7f566eb22640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f566806dbf0 con 0x7f566806c7f0 2024-05-11T08:06:17.351 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.314+0000 7fb2db4c2640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb2cc053bf0 0x7fb2cc0d3b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:17.351 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.314+0000 7f21497e4640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2144072630 0x7f214415b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:17.352 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.314+0000 7fb2db4c2640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb2cc0e12a0 0x7fb2cc04de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:17.352 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.314+0000 7fb2db4c2640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb2cc050e50 0x7fb2cc0de380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:17.353 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.314+0000 7f21497e4640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f21440cf410 0x7f214414b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:17.353 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.314+0000 7f21497e4640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f214415f160 0x7f2144148bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:17.353 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.314+0000 7f21497e4640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f214415f540 con 0x7f214415f160 2024-05-11T08:06:17.353 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.314+0000 7f21497e4640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f21440cf870 con 0x7f21440cf410 2024-05-11T08:06:17.353 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.314+0000 7f21497e4640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f214415f6c0 con 0x7f2144072630 2024-05-11T08:06:17.353 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.314+0000 7f8a1110f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a0c06cee0 0x7f8a0c1512c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:17.354 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.314+0000 7f5667fff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f56680d0640 0x7f56680c6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:17.354 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.314+0000 7f5667fff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f56680d0640 0x7f56680c6820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:54400/0 (socket says 172.21.15.167:54400) 2024-05-11T08:06:17.354 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.314+0000 7fb2db4c2640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb2cc057810 con 0x7fb2cc0e12a0 2024-05-11T08:06:17.354 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.314+0000 7f5667fff640 1 -- 172.21.15.167:0/940586582 learned_addr learned my addr 172.21.15.167:0/940586582 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:17.354 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.314+0000 7f8a1110f640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8a0c0b7e70 0x7f8a0c163b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:17.355 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.314+0000 7f2148fe3640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f214415f160 0x7f2144148bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:17.355 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.314+0000 7f21437fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f21440cf410 0x7f214414b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:17.355 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.314+0000 7f2148fe3640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f214415f160 0x7f2144148bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:54416/0 (socket says 172.21.15.167:54416) 2024-05-11T08:06:17.355 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.314+0000 7f2148fe3640 1 -- 172.21.15.167:0/3389626582 learned_addr learned my addr 172.21.15.167:0/3389626582 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:17.355 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.314+0000 7f2143fff640 1 --2- 172.21.15.167:0/3389626582 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2144072630 0x7f214415b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:17.355 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.314+0000 7fb2db4c2640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb2cc057690 con 0x7fb2cc053bf0 2024-05-11T08:06:17.356 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.313+0000 7f8ac2ffd640 1 -- 172.21.15.167:0/2979835464 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ac4014800 con 0x7f8acc15b5d0 2024-05-11T08:06:17.356 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.314+0000 7f8ad4454640 1 -- 172.21.15.167:0/2979835464 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8acc09dd20 con 0x7f8acc15b5d0 2024-05-11T08:06:17.356 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.314+0000 7f8ad4454640 1 -- 172.21.15.167:0/2979835464 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8acc09e2e0 con 0x7f8acc15b5d0 2024-05-11T08:06:17.356 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.314+0000 7f8a0bfff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a0c06cee0 0x7f8a0c1512c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:17.356 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.314+0000 7f8a0bfff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a0c06cee0 0x7f8a0c1512c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.167:37262/0 (socket says 172.21.15.167:37262) 2024-05-11T08:06:17.357 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.314+0000 7f8a0bfff640 1 -- 172.21.15.167:0/2251475889 learned_addr learned my addr 172.21.15.167:0/2251475889 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:17.357 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.314+0000 7f8a1110f640 1 --2- 172.21.15.167:0/2251475889 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8a0c163510 0x7f8a0c165970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:17.357 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.314+0000 7f8a0b7fe640 1 --2- 172.21.15.167:0/2251475889 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8a0c0b7e70 0x7f8a0c163b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:17.357 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.314+0000 7fb2db4c2640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fb2cc0e4d20 con 0x7fb2cc050e50 2024-05-11T08:06:17.357 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.314+0000 7fb2dacc1640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb2cc050e50 0x7fb2cc0de380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:17.357 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.314+0000 7fb2dacc1640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb2cc050e50 0x7fb2cc0de380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.167:37260/0 (socket says 172.21.15.167:37260) 2024-05-11T08:06:17.357 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.314+0000 7fb2dacc1640 1 -- 172.21.15.167:0/3918353499 learned_addr learned my addr 172.21.15.167:0/3918353499 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:17.358 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.315+0000 7f8a1110f640 1 -- 172.21.15.167:0/2251475889 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8a0c0d3000 con 0x7f8a0c163510 2024-05-11T08:06:17.358 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.315+0000 7f8a1110f640 1 -- 172.21.15.167:0/2251475889 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8a0c0d2d20 con 0x7f8a0c06cee0 2024-05-11T08:06:17.358 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.315+0000 7f8a1110f640 1 -- 172.21.15.167:0/2251475889 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f8a0c06ea70 con 0x7f8a0c0b7e70 2024-05-11T08:06:17.358 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.315+0000 7f2148fe3640 1 -- 172.21.15.167:0/3389626582 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2144072630 msgr2=0x7f214415b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:17.358 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.315+0000 7f2148fe3640 1 --2- 172.21.15.167:0/3389626582 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2144072630 0x7f214415b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:17.358 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.315+0000 7f2148fe3640 1 -- 172.21.15.167:0/3389626582 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f21440cf410 msgr2=0x7f214414b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:17.358 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.315+0000 7f2148fe3640 1 --2- 172.21.15.167:0/3389626582 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f21440cf410 0x7f214414b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:17.358 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.315+0000 7f2148fe3640 1 -- 172.21.15.167:0/3389626582 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f214414bc70 con 0x7f214415f160 2024-05-11T08:06:17.358 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.315+0000 7f21437fe640 1 --2- 172.21.15.167:0/3389626582 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f21440cf410 0x7f214414b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T08:06:17.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.315+0000 7fca3b59e640 1 Processor -- start 2024-05-11T08:06:17.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.315+0000 7f5b1acc2640 1 -- 172.21.15.167:0/2806661591 shutdown_connections 2024-05-11T08:06:17.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.315+0000 7fca3b59e640 1 -- start start 2024-05-11T08:06:17.360 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.315+0000 7fb2dacc1640 1 -- 172.21.15.167:0/3918353499 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb2cc053bf0 msgr2=0x7fb2cc0d3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:17.360 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.315+0000 7fb2dacc1640 1 --2- 172.21.15.167:0/3918353499 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb2cc053bf0 0x7fb2cc0d3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:17.360 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.315+0000 7fb2d9cbf640 1 --2- 172.21.15.167:0/3918353499 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb2cc0e12a0 0x7fb2cc04de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:17.360 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.315+0000 7fb2dacc1640 1 -- 172.21.15.167:0/3918353499 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb2cc0e12a0 msgr2=0x7fb2cc04de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:17.360 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.315+0000 7fb2dacc1640 1 --2- 172.21.15.167:0/3918353499 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb2cc0e12a0 0x7fb2cc04de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:17.360 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.315+0000 7fb2dacc1640 1 -- 172.21.15.167:0/3918353499 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2cc04fb10 con 0x7fb2cc050e50 2024-05-11T08:06:17.360 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.315+0000 7fb2d9cbf640 1 --2- 172.21.15.167:0/3918353499 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb2cc0e12a0 0x7fb2cc04de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T08:06:17.360 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.315+0000 7f566c8be640 1 --2- 172.21.15.167:0/940586582 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f566806c7f0 0x7f56680cf160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:17.361 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.315+0000 7f8a1090e640 1 --2- 172.21.15.167:0/2251475889 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8a0c163510 0x7f8a0c165970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:17.361 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.315+0000 7f2148fe3640 1 --2- 172.21.15.167:0/3389626582 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f214415f160 0x7f2144148bb0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f213400b990 tx=0x7f213400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:17.361 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.315+0000 7f8a0bfff640 1 -- 172.21.15.167:0/2251475889 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8a0c0b7e70 msgr2=0x7f8a0c163b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:17.361 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.315+0000 7f8a0bfff640 1 --2- 172.21.15.167:0/2251475889 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8a0c0b7e70 0x7f8a0c163b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:17.362 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.315+0000 7f8a0bfff640 1 -- 172.21.15.167:0/2251475889 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8a0c163510 msgr2=0x7f8a0c165970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:17.362 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.315+0000 7f8a0bfff640 1 --2- 172.21.15.167:0/2251475889 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8a0c163510 0x7f8a0c165970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:17.362 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.315+0000 7f8a0bfff640 1 -- 172.21.15.167:0/2251475889 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a0c165530 con 0x7f8a0c06cee0 2024-05-11T08:06:17.362 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.315+0000 7f5b1acc2640 1 -- 172.21.15.167:0/2806661591 wait complete. 2024-05-11T08:06:17.362 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.315+0000 7fb2dacc1640 1 --2- 172.21.15.167:0/3918353499 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb2cc050e50 0x7fb2cc0de380 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fb2c400b3c0 tx=0x7fb2c400b880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:17.363 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.314+0000 7f566d0bf640 1 --2- 172.21.15.167:0/940586582 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f566815f3f0 0x7f56680c6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:17.363 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.315+0000 7f5667fff640 1 -- 172.21.15.167:0/940586582 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f566806c7f0 msgr2=0x7f56680cf160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:17.363 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.315+0000 7f5667fff640 1 --2- 172.21.15.167:0/940586582 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f566806c7f0 0x7f56680cf160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:17.363 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.315+0000 7fb2caffd640 1 -- 172.21.15.167:0/3918353499 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2c4011830 con 0x7fb2cc050e50 2024-05-11T08:06:17.364 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.315+0000 7f5667fff640 1 -- 172.21.15.167:0/940586582 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f566815f3f0 msgr2=0x7f56680c6d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:17.364 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.316+0000 7f5667fff640 1 --2- 172.21.15.167:0/940586582 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f566815f3f0 0x7f56680c6d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:17.364 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.316+0000 7f5667fff640 1 -- 172.21.15.167:0/940586582 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56680707f0 con 0x7f56680d0640 2024-05-11T08:06:17.364 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.316+0000 7f8a0bfff640 1 --2- 172.21.15.167:0/2251475889 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a0c06cee0 0x7f8a0c1512c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f89fc00b3e0 tx=0x7f89fc00b7e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:17.364 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.316+0000 7f5667fff640 1 --2- 172.21.15.167:0/940586582 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f56680d0640 0x7f56680c6820 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f566000ba50 tx=0x7f566000bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:17.365 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.316+0000 7fca3b59e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca3403a5d0 0x7fca340cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:17.365 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.316+0000 7fca3b59e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fca34050dd0 0x7fca3404e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:17.365 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.316+0000 7fca3b59e640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fca340e1090 0x7fca340376e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:17.365 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.316+0000 7fca3a59c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca3403a5d0 0x7fca340cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:17.366 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.316+0000 7f8a08ff9640 1 -- 172.21.15.167:0/2251475889 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89fc015020 con 0x7f8a0c06cee0 2024-05-11T08:06:17.366 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.316+0000 7f8a1110f640 1 -- 172.21.15.167:0/2251475889 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a0c164490 con 0x7f8a0c06cee0 2024-05-11T08:06:17.366 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.316+0000 7fca3b59e640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fca340064d0 con 0x7fca3403a5d0 2024-05-11T08:06:17.366 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.316+0000 7f8a1110f640 1 -- 172.21.15.167:0/2251475889 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a0c164a50 con 0x7f8a0c06cee0 2024-05-11T08:06:17.367 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.316+0000 7fca3a59c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca3403a5d0 0x7fca340cfde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:54442/0 (socket says 172.21.15.167:54442) 2024-05-11T08:06:17.367 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.316+0000 7fca3a59c640 1 -- 172.21.15.167:0/1866920501 learned_addr learned my addr 172.21.15.167:0/1866920501 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:17.367 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.316+0000 7fca3b59e640 1 -- 172.21.15.167:0/1866920501 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fca340503a0 con 0x7fca34050dd0 2024-05-11T08:06:17.367 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.316+0000 7fca3b59e640 1 -- 172.21.15.167:0/1866920501 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fca34006650 con 0x7fca340e1090 2024-05-11T08:06:17.367 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.316+0000 7fca39d9b640 1 --2- 172.21.15.167:0/1866920501 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fca34050dd0 0x7fca3404e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:17.368 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.317+0000 7f5b1acc2640 1 Processor -- start 2024-05-11T08:06:17.368 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.316+0000 7fca3ad9d640 1 --2- 172.21.15.167:0/1866920501 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fca340e1090 0x7fca340376e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:17.368 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.317+0000 7f56657fa640 1 -- 172.21.15.167:0/940586582 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5660019070 con 0x7f56680d0640 2024-05-11T08:06:17.369 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.317+0000 7f8ad4454640 1 -- 172.21.15.167:0/2979835464 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8acc069bc0 con 0x7f8acc15b5d0 2024-05-11T08:06:17.369 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.317+0000 7f566eb22640 1 -- 172.21.15.167:0/940586582 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5668153660 con 0x7f56680d0640 2024-05-11T08:06:17.369 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.317+0000 7fca3a59c640 1 -- 172.21.15.167:0/1866920501 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fca340e1090 msgr2=0x7fca340376e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:17.369 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.317+0000 7fca3a59c640 1 --2- 172.21.15.167:0/1866920501 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fca340e1090 0x7fca340376e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:17.369 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.317+0000 7fca3a59c640 1 -- 172.21.15.167:0/1866920501 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fca34050dd0 msgr2=0x7fca3404e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:17.369 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.317+0000 7fca3a59c640 1 --2- 172.21.15.167:0/1866920501 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fca34050dd0 0x7fca3404e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:17.369 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.317+0000 7fca3a59c640 1 -- 172.21.15.167:0/1866920501 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca340dcf50 con 0x7fca3403a5d0 2024-05-11T08:06:17.370 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.317+0000 7f8a08ff9640 1 -- 172.21.15.167:0/2251475889 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89fc011850 con 0x7f8a0c06cee0 2024-05-11T08:06:17.370 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.317+0000 7f8a08ff9640 1 -- 172.21.15.167:0/2251475889 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89fc011c40 con 0x7f8a0c06cee0 2024-05-11T08:06:17.370 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.317+0000 7f566eb22640 1 -- 172.21.15.167:0/940586582 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5668070230 con 0x7f56680d0640 2024-05-11T08:06:17.371 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.316+0000 7f4d80a1d640 1 Processor -- start 2024-05-11T08:06:17.371 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.317+0000 7fca3a59c640 1 --2- 172.21.15.167:0/1866920501 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca3403a5d0 0x7fca340cfde0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fca30038040 tx=0x7fca300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:17.371 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.315+0000 7fb2caffd640 1 -- 172.21.15.167:0/3918353499 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2c4016070 con 0x7fb2cc050e50 2024-05-11T08:06:17.371 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.315+0000 7fb2caffd640 1 -- 172.21.15.167:0/3918353499 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2c4011c30 con 0x7fb2cc050e50 2024-05-11T08:06:17.371 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.316+0000 7fb2db4c2640 1 -- 172.21.15.167:0/3918353499 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2cc053830 con 0x7fb2cc050e50 2024-05-11T08:06:17.371 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.316+0000 7fb2db4c2640 1 -- 172.21.15.167:0/3918353499 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2cc0d5160 con 0x7fb2cc050e50 2024-05-11T08:06:17.372 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.317+0000 7f5b1acc2640 1 -- start start 2024-05-11T08:06:17.372 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.317+0000 7f2140ff9640 1 -- 172.21.15.167:0/3389626582 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2134019070 con 0x7f214415f160 2024-05-11T08:06:17.372 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.318+0000 7f2140ff9640 1 -- 172.21.15.167:0/3389626582 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f213400cba0 con 0x7f214415f160 2024-05-11T08:06:17.372 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.318+0000 7f2140ff9640 1 -- 172.21.15.167:0/3389626582 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2134014440 con 0x7f214415f160 2024-05-11T08:06:17.372 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.318+0000 7f5b1acc2640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5b1406c450 0x7f5b14149640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:17.372 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.318+0000 7f5b1acc2640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5b1415f170 0x7f5b14151f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:17.373 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.318+0000 7f5b1acc2640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5b1414f520 0x7f5b1414f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:17.373 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.318+0000 7f5b1acc2640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5b140d0530 con 0x7f5b1406c450 2024-05-11T08:06:17.373 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.318+0000 7f21497e4640 1 -- 172.21.15.167:0/3389626582 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f214414bf50 con 0x7f214415f160 2024-05-11T08:06:17.373 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.318+0000 7f21497e4640 1 -- 172.21.15.167:0/3389626582 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2144149680 con 0x7f214415f160 2024-05-11T08:06:17.373 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.318+0000 7f5b19cc0640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5b1406c450 0x7f5b14149640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:17.373 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.318+0000 7f5b19cc0640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5b1406c450 0x7f5b14149640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:54452/0 (socket says 172.21.15.167:54452) 2024-05-11T08:06:17.373 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.318+0000 7f5b19cc0640 1 -- 172.21.15.167:0/2681710861 learned_addr learned my addr 172.21.15.167:0/2681710861 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:17.374 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.318+0000 7f5b1acc2640 1 -- 172.21.15.167:0/2681710861 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5b141624e0 con 0x7f5b1414f520 2024-05-11T08:06:17.374 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.318+0000 7f5b1acc2640 1 -- 172.21.15.167:0/2681710861 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f5b140d0810 con 0x7f5b1415f170 2024-05-11T08:06:17.374 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.318+0000 7f566eb22640 1 -- 172.21.15.167:0/940586582 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56680b83d0 con 0x7f56680d0640 2024-05-11T08:06:17.374 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.318+0000 7f8a1110f640 1 -- 172.21.15.167:0/2251475889 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a0c0694b0 con 0x7f8a0c06cee0 2024-05-11T08:06:17.374 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.318+0000 7f5b194bf640 1 --2- 172.21.15.167:0/2681710861 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5b1415f170 0x7f5b14151f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:17.375 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.319+0000 7f566d0bf640 1 --2- 172.21.15.167:0/940586582 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f566815f3f0 0x7f56680c6d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T08:06:17.375 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.319+0000 7f8ac2ffd640 1 -- 172.21.15.167:0/2979835464 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f8ac40149a0 con 0x7f8acc15b5d0 2024-05-11T08:06:17.375 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.319+0000 7f5b19cc0640 1 -- 172.21.15.167:0/2681710861 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5b1415f170 msgr2=0x7f5b14151f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:17.375 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.319+0000 7f5b19cc0640 1 --2- 172.21.15.167:0/2681710861 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5b1415f170 0x7f5b14151f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:17.375 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.319+0000 7f5b19cc0640 1 -- 172.21.15.167:0/2681710861 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5b1414f520 msgr2=0x7f5b1414f9b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-11T08:06:17.376 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.319+0000 7f5b19cc0640 1 --2- 172.21.15.167:0/2681710861 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5b1414f520 0x7f5b1414f9b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:17.376 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.319+0000 7f5b19cc0640 1 -- 172.21.15.167:0/2681710861 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b14144620 con 0x7f5b1406c450 2024-05-11T08:06:17.376 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.319+0000 7fb2caffd640 1 -- 172.21.15.167:0/3918353499 <== mon.2 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fb2c401b460 con 0x7fb2cc050e50 2024-05-11T08:06:17.376 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.319+0000 7f56657fa640 1 -- 172.21.15.167:0/940586582 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f566000cbe0 con 0x7f56680d0640 2024-05-11T08:06:17.376 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.319+0000 7f56657fa640 1 -- 172.21.15.167:0/940586582 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5660014440 con 0x7f56680d0640 2024-05-11T08:06:17.376 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.319+0000 7f5b19cc0640 1 --2- 172.21.15.167:0/2681710861 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5b1406c450 0x7f5b14149640 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5b1002a7f0 tx=0x7f5b100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:17.377 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.319+0000 7f21497e4640 1 -- 172.21.15.167:0/3389626582 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21440694b0 con 0x7f214415f160 2024-05-11T08:06:17.377 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.319+0000 7f8ac2ffd640 1 --2- 172.21.15.167:0/2979835464 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f8ab00763a0 0x7f8ab0078860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:17.377 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.319+0000 7f8ad21f0640 1 --2- 172.21.15.167:0/2979835464 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f8ab00763a0 0x7f8ab0078860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:17.377 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.319+0000 7f8ac2ffd640 1 -- 172.21.15.167:0/2979835464 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f8ac40980a0 con 0x7f8acc15b5d0 2024-05-11T08:06:17.378 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.317+0000 7f4d80a1d640 1 -- start start 2024-05-11T08:06:17.378 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.318+0000 7f4d80a1d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4d7c06d460 0x7f4d7c163280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:17.378 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.320+0000 7f4d80a1d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4d7c0b47d0 0x7f4d7c145760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:17.378 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.320+0000 7fb2caffd640 1 --2- 172.21.15.167:0/3918353499 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fb2b80762d0 0x7fb2b8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:17.378 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.320+0000 7f4d80a1d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4d7c14c8a0 0x7f4d7c1faa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:17.379 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.320+0000 7f4d80a1d640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4d7c06fd30 con 0x7f4d7c06d460 2024-05-11T08:06:17.379 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.320+0000 7f4d80a1d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4d7c0cf8d0 con 0x7f4d7c14c8a0 2024-05-11T08:06:17.379 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.320+0000 7f4d80a1d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f4d7c06feb0 con 0x7f4d7c0b47d0 2024-05-11T08:06:17.379 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.320+0000 7f4d79d9b640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4d7c0b47d0 0x7f4d7c145760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:17.379 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.320+0000 7fb2da4c0640 1 --2- 172.21.15.167:0/3918353499 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fb2b80762d0 0x7fb2b8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:17.379 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.320+0000 7f4d79d9b640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4d7c0b47d0 0x7f4d7c145760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.167:37314/0 (socket says 172.21.15.167:37314) 2024-05-11T08:06:17.380 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.320+0000 7f8ad21f0640 1 --2- 172.21.15.167:0/2979835464 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f8ab00763a0 0x7f8ab0078860 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8abc036000 tx=0x7f8abc031000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:17.380 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.320+0000 7f4d79d9b640 1 -- 172.21.15.167:0/1160539987 learned_addr learned my addr 172.21.15.167:0/1160539987 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:17.380 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.320+0000 7fb2caffd640 1 -- 172.21.15.167:0/3918353499 <== mon.2 v2:172.21.15.204:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fb2c4097710 con 0x7fb2cc050e50 2024-05-11T08:06:17.381 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.320+0000 7f4d79d9b640 1 -- 172.21.15.167:0/1160539987 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4d7c14c8a0 msgr2=0x7f4d7c1faa90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-11T08:06:17.381 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.320+0000 7f4d79d9b640 1 --2- 172.21.15.167:0/1160539987 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4d7c14c8a0 0x7f4d7c1faa90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:17.381 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.320+0000 7f4d79d9b640 1 -- 172.21.15.167:0/1160539987 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4d7c06d460 msgr2=0x7f4d7c163280 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-11T08:06:17.381 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.320+0000 7f4d79d9b640 1 --2- 172.21.15.167:0/1160539987 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4d7c06d460 0x7f4d7c163280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:17.381 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.320+0000 7f4d79d9b640 1 -- 172.21.15.167:0/1160539987 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d7c143a80 con 0x7f4d7c0b47d0 2024-05-11T08:06:17.382 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.320+0000 7f4d79d9b640 1 --2- 172.21.15.167:0/1160539987 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4d7c0b47d0 0x7f4d7c145760 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f4d7000e9b0 tx=0x7f4d7000ee70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:17.382 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.320+0000 7fb2db4c2640 1 -- 172.21.15.167:0/3918353499 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2cc0057d0 con 0x7fb2cc050e50 2024-05-11T08:06:17.382 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.321+0000 7f4d6affd640 1 -- 172.21.15.167:0/1160539987 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d70017070 con 0x7f4d7c0b47d0 2024-05-11T08:06:17.383 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.321+0000 7f4d80a1d640 1 -- 172.21.15.167:0/1160539987 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d7c143d60 con 0x7f4d7c0b47d0 2024-05-11T08:06:17.383 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.321+0000 7f4d6affd640 1 -- 172.21.15.167:0/1160539987 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d7000c9a0 con 0x7f4d7c0b47d0 2024-05-11T08:06:17.383 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.321+0000 7f4d6affd640 1 -- 172.21.15.167:0/1160539987 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d7000cd90 con 0x7f4d7c0b47d0 2024-05-11T08:06:17.383 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.321+0000 7f4d80a1d640 1 -- 172.21.15.167:0/1160539987 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d7c144320 con 0x7f4d7c0b47d0 2024-05-11T08:06:17.383 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.321+0000 7fb2da4c0640 1 --2- 172.21.15.167:0/3918353499 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fb2b80762d0 0x7fb2b8078790 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb2d40515a0 tx=0x7fb2d406a230 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:17.383 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.322+0000 7fca2affd640 1 -- 172.21.15.167:0/1866920501 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca30045070 con 0x7fca3403a5d0 2024-05-11T08:06:17.384 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.322+0000 7fca2affd640 1 -- 172.21.15.167:0/1866920501 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca30035790 con 0x7fca3403a5d0 2024-05-11T08:06:17.384 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.322+0000 7f8a08ff9640 1 -- 172.21.15.167:0/2251475889 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f89fc018020 con 0x7f8a0c06cee0 2024-05-11T08:06:17.384 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.322+0000 7fca3b59e640 1 -- 172.21.15.167:0/1866920501 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca340dd230 con 0x7fca3403a5d0 2024-05-11T08:06:17.384 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.322+0000 7fca2affd640 1 -- 172.21.15.167:0/1866920501 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca3003a070 con 0x7fca3403a5d0 2024-05-11T08:06:17.384 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.322+0000 7fca3b59e640 1 -- 172.21.15.167:0/1866920501 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca340dd7f0 con 0x7fca3403a5d0 2024-05-11T08:06:17.385 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.322+0000 7f4d80a1d640 1 -- 172.21.15.167:0/1160539987 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d7c069590 con 0x7f4d7c0b47d0 2024-05-11T08:06:17.385 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.322+0000 7f8a08ff9640 1 --2- 172.21.15.167:0/2251475889 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f89f80763a0 0x7f89f8078860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:17.385 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.323+0000 7f8a0b7fe640 1 --2- 172.21.15.167:0/2251475889 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f89f80763a0 0x7f89f8078860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:17.385 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.323+0000 7fca3b59e640 1 -- 172.21.15.167:0/1866920501 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca34053e10 con 0x7fca3403a5d0 2024-05-11T08:06:17.386 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.323+0000 7f8a08ff9640 1 -- 172.21.15.167:0/2251475889 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f89fc097730 con 0x7f8a0c06cee0 2024-05-11T08:06:17.386 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.323+0000 7f8a0b7fe640 1 --2- 172.21.15.167:0/2251475889 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f89f80763a0 0x7f89f8078860 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c0cf0f0 tx=0x7f8a00009110 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:17.386 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.326+0000 7fb2caffd640 1 -- 172.21.15.167:0/3918353499 <== mon.2 v2:172.21.15.204:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fb2c4060c00 con 0x7fb2cc050e50 2024-05-11T08:06:17.386 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.326+0000 7f8ac2ffd640 1 -- 172.21.15.167:0/2979835464 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f8ac4061730 con 0x7f8acc15b5d0 2024-05-11T08:06:17.387 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.328+0000 7f4d6affd640 1 -- 172.21.15.167:0/1160539987 <== mon.2 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f4d7001a070 con 0x7f4d7c0b47d0 2024-05-11T08:06:17.387 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.328+0000 7f2140ff9640 1 -- 172.21.15.167:0/3389626582 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f213401c070 con 0x7f214415f160 2024-05-11T08:06:17.387 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.329+0000 7f8a08ff9640 1 -- 172.21.15.167:0/2251475889 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f89fc01f020 con 0x7f8a0c06cee0 2024-05-11T08:06:17.388 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.329+0000 7fca2affd640 1 -- 172.21.15.167:0/1866920501 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fca30035b80 con 0x7fca3403a5d0 2024-05-11T08:06:17.388 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.329+0000 7f4d6affd640 1 --2- 172.21.15.167:0/1160539987 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f4d5c076320 0x7f4d5c0787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:17.388 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.329+0000 7f4d6affd640 1 -- 172.21.15.167:0/1160539987 <== mon.2 v2:172.21.15.204:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f4d70097ff0 con 0x7f4d7c0b47d0 2024-05-11T08:06:17.388 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.329+0000 7fca2affd640 1 --2- 172.21.15.167:0/1866920501 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fca1c076320 0x7fca1c0787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:17.389 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.329+0000 7f2140ff9640 1 --2- 172.21.15.167:0/3389626582 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f2124076320 0x7f21240787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:17.389 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.329+0000 7f56657fa640 1 -- 172.21.15.167:0/940586582 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f566001c070 con 0x7f56680d0640 2024-05-11T08:06:17.389 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.329+0000 7fca2affd640 1 -- 172.21.15.167:0/1866920501 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fca300bc400 con 0x7fca3403a5d0 2024-05-11T08:06:17.389 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.330+0000 7fca39d9b640 1 --2- 172.21.15.167:0/1866920501 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fca1c076320 0x7fca1c0787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:17.390 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.330+0000 7f5b0a7fc640 1 -- 172.21.15.167:0/2681710861 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b10042070 con 0x7f5b1406c450 2024-05-11T08:06:17.390 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.330+0000 7f5b0a7fc640 1 -- 172.21.15.167:0/2681710861 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b10030a40 con 0x7f5b1406c450 2024-05-11T08:06:17.390 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.330+0000 7f5b0a7fc640 1 -- 172.21.15.167:0/2681710861 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b10030e10 con 0x7f5b1406c450 2024-05-11T08:06:17.390 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.330+0000 7f5b1acc2640 1 -- 172.21.15.167:0/2681710861 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b1414b110 con 0x7f5b1406c450 2024-05-11T08:06:17.390 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.330+0000 7f5b1acc2640 1 -- 172.21.15.167:0/2681710861 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b1414b6b0 con 0x7f5b1406c450 2024-05-11T08:06:17.390 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.330+0000 7f4d7a59c640 1 --2- 172.21.15.167:0/1160539987 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f4d5c076320 0x7f4d5c0787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:17.390 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.330+0000 7f4d7a59c640 1 --2- 172.21.15.167:0/1160539987 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f4d5c076320 0x7f4d5c0787e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f4d6c007fd0 tx=0x7f4d6c0079a0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:17.391 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.330+0000 7f56657fa640 1 --2- 172.21.15.167:0/940586582 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f56540762d0 0x7f5654078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:17.391 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.330+0000 7f2143fff640 1 --2- 172.21.15.167:0/3389626582 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f2124076320 0x7f21240787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:17.391 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.331+0000 7f566c8be640 1 --2- 172.21.15.167:0/940586582 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f56540762d0 0x7f5654078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:17.392 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.331+0000 7fca39d9b640 1 --2- 172.21.15.167:0/1866920501 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fca1c076320 0x7fca1c0787e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fca340de6b0 tx=0x7fca2c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:17.392 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.331+0000 7f56657fa640 1 -- 172.21.15.167:0/940586582 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f5660097ee0 con 0x7f56680d0640 2024-05-11T08:06:17.392 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.331+0000 7f566c8be640 1 --2- 172.21.15.167:0/940586582 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f56540762d0 0x7f5654078790 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f565c034000 tx=0x7f565c02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:17.392 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.331+0000 7f4d6affd640 1 -- 172.21.15.167:0/1160539987 <== mon.2 v2:172.21.15.204:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f4d700616e0 con 0x7f4d7c0b47d0 2024-05-11T08:06:17.393 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.331+0000 7f2140ff9640 1 -- 172.21.15.167:0/3389626582 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f213405cd40 con 0x7f214415f160 2024-05-11T08:06:17.393 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.332+0000 7f2143fff640 1 --2- 172.21.15.167:0/3389626582 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f2124076320 0x7f21240787e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f213c034000 tx=0x7f213c02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:17.393 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.332+0000 7f5b1acc2640 1 -- 172.21.15.167:0/2681710861 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b14069bc0 con 0x7f5b1406c450 2024-05-11T08:06:17.393 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.334+0000 7f2140ff9640 1 -- 172.21.15.167:0/3389626582 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f2134023020 con 0x7f214415f160 2024-05-11T08:06:17.394 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.339+0000 7fca2affd640 1 -- 172.21.15.167:0/1866920501 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fca300853b0 con 0x7fca3403a5d0 2024-05-11T08:06:17.394 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.341+0000 7f5b0a7fc640 1 -- 172.21.15.167:0/2681710861 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f5b10045070 con 0x7f5b1406c450 2024-05-11T08:06:17.395 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.342+0000 7f5b0a7fc640 1 --2- 172.21.15.167:0/2681710861 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f5af8076320 0x7f5af80787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:17.395 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.343+0000 7f5b194bf640 1 --2- 172.21.15.167:0/2681710861 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f5af8076320 0x7f5af80787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:17.395 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.344+0000 7f5b194bf640 1 --2- 172.21.15.167:0/2681710861 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f5af8076320 0x7f5af80787e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f5b1414dbc0 tx=0x7f5b04008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:17.395 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.346+0000 7f5b0a7fc640 1 -- 172.21.15.167:0/2681710861 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f5b10009430 con 0x7f5b1406c450 2024-05-11T08:06:17.395 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.347+0000 7f56657fa640 1 -- 172.21.15.167:0/940586582 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f56680b83d0 con 0x7f56680d0640 2024-05-11T08:06:17.395 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.355+0000 7f5b0a7fc640 1 -- 172.21.15.167:0/2681710861 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f5b10081a70 con 0x7f5b1406c450 2024-05-11T08:06:17.760 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.758+0000 7f8a1110f640 1 -- 172.21.15.167:0/2251475889 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f8a0c0759b0 con 0x7f8a0c06cee0 2024-05-11T08:06:17.760 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.759+0000 7f8a08ff9640 1 -- 172.21.15.167:0/2251475889 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f8a0c165530 con 0x7f8a0c06cee0 2024-05-11T08:06:17.760 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.759+0000 7f8ad4454640 1 -- 172.21.15.167:0/2979835464 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f8acc0cebf0 con 0x7f8acc15b5d0 2024-05-11T08:06:17.761 INFO:teuthology.orchestra.run.smithi167.stdout:64424509449 2024-05-11T08:06:17.761 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.760+0000 7f4d80a1d640 1 -- 172.21.15.167:0/1160539987 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f4d7c0b5570 con 0x7f4d7c0b47d0 2024-05-11T08:06:17.761 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.760+0000 7f4d6affd640 1 -- 172.21.15.167:0/1160539987 <== mon.2 v2:172.21.15.204:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f4d70060fe0 con 0x7f4d7c0b47d0 2024-05-11T08:06:17.761 INFO:teuthology.orchestra.run.smithi167.stdout:64424509449 2024-05-11T08:06:17.762 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.761+0000 7f8ac2ffd640 1 -- 172.21.15.167:0/2979835464 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f8ac4061030 con 0x7f8acc15b5d0 2024-05-11T08:06:17.762 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.761+0000 7fb2db4c2640 1 -- 172.21.15.167:0/3918353499 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fb2cc057470 con 0x7fb2cc050e50 2024-05-11T08:06:17.762 INFO:teuthology.orchestra.run.smithi167.stdout:64424509449 2024-05-11T08:06:17.763 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.762+0000 7fb2caffd640 1 -- 172.21.15.167:0/3918353499 <== mon.2 v2:172.21.15.204:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fb2c4060500 con 0x7fb2cc050e50 2024-05-11T08:06:17.763 INFO:teuthology.orchestra.run.smithi167.stdout:55834574857 2024-05-11T08:06:17.765 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.762+0000 7f8a1110f640 1 -- 172.21.15.167:0/2251475889 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f89f80763a0 msgr2=0x7f89f8078860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:17.765 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.762+0000 7f8a1110f640 1 --2- 172.21.15.167:0/2251475889 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f89f80763a0 0x7f89f8078860 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c0cf0f0 tx=0x7f8a00009110 comp rx=0 tx=0).stop 2024-05-11T08:06:17.765 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.763+0000 7f8a1110f640 1 -- 172.21.15.167:0/2251475889 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a0c06cee0 msgr2=0x7f8a0c1512c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:17.765 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.763+0000 7f8a1110f640 1 --2- 172.21.15.167:0/2251475889 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a0c06cee0 0x7f8a0c1512c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f89fc00b3e0 tx=0x7f89fc00b7e0 comp rx=0 tx=0).stop 2024-05-11T08:06:17.765 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.763+0000 7f8a1110f640 1 -- 172.21.15.167:0/2251475889 shutdown_connections 2024-05-11T08:06:17.765 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.763+0000 7f8a1110f640 1 --2- 172.21.15.167:0/2251475889 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f89f80763a0 0x7f89f8078860 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:17.765 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.763+0000 7f8a1110f640 1 --2- 172.21.15.167:0/2251475889 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8a0c163510 0x7f8a0c165970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:17.765 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.763+0000 7f8a1110f640 1 --2- 172.21.15.167:0/2251475889 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8a0c0b7e70 0x7f8a0c163b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:17.765 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.763+0000 7f8a1110f640 1 --2- 172.21.15.167:0/2251475889 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a0c06cee0 0x7f8a0c1512c0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:17.765 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.763+0000 7f8a1110f640 1 -- 172.21.15.167:0/2251475889 >> 172.21.15.167:0/2251475889 conn(0x7f8a0c0d5020 msgr2=0x7f8a0c0b8430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:17.765 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.763+0000 7f8a1110f640 1 -- 172.21.15.167:0/2251475889 shutdown_connections 2024-05-11T08:06:17.765 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.763+0000 7f8a1110f640 1 -- 172.21.15.167:0/2251475889 wait complete. 2024-05-11T08:06:17.766 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.764+0000 7f8ad4454640 1 -- 172.21.15.167:0/2979835464 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f8ab00763a0 msgr2=0x7f8ab0078860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:17.766 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.764+0000 7f8ad4454640 1 --2- 172.21.15.167:0/2979835464 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f8ab00763a0 0x7f8ab0078860 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8abc036000 tx=0x7f8abc031000 comp rx=0 tx=0).stop 2024-05-11T08:06:17.766 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.764+0000 7f8ad4454640 1 -- 172.21.15.167:0/2979835464 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8acc15b5d0 msgr2=0x7f8acc0c8630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:17.766 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.764+0000 7f8ad4454640 1 --2- 172.21.15.167:0/2979835464 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8acc15b5d0 0x7f8acc0c8630 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f8ac400e1c0 tx=0x7f8ac4009f90 comp rx=0 tx=0).stop 2024-05-11T08:06:17.766 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.764+0000 7f8ad4454640 1 -- 172.21.15.167:0/2979835464 shutdown_connections 2024-05-11T08:06:17.766 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.764+0000 7f8ad4454640 1 --2- 172.21.15.167:0/2979835464 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f8ab00763a0 0x7f8ab0078860 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:17.767 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.764+0000 7f8ad4454640 1 --2- 172.21.15.167:0/2979835464 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8acc15b5d0 0x7f8acc0c8630 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:17.767 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.764+0000 7f8ad4454640 1 --2- 172.21.15.167:0/2979835464 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8acc0d08f0 0x7f8acc0c80f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:17.767 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.764+0000 7f8ad4454640 1 --2- 172.21.15.167:0/2979835464 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8acc0ade30 0x7f8acc14c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:17.767 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.764+0000 7f8ad4454640 1 -- 172.21.15.167:0/2979835464 >> 172.21.15.167:0/2979835464 conn(0x7f8acc0d1200 msgr2=0x7f8acc0cef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:17.768 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.764+0000 7f4d80a1d640 1 -- 172.21.15.167:0/1160539987 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f4d5c076320 msgr2=0x7f4d5c0787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:17.768 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.764+0000 7f4d80a1d640 1 --2- 172.21.15.167:0/1160539987 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f4d5c076320 0x7f4d5c0787e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f4d6c007fd0 tx=0x7f4d6c0079a0 comp rx=0 tx=0).stop 2024-05-11T08:06:17.768 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.764+0000 7f4d80a1d640 1 -- 172.21.15.167:0/1160539987 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4d7c0b47d0 msgr2=0x7f4d7c145760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:17.768 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.764+0000 7f4d80a1d640 1 --2- 172.21.15.167:0/1160539987 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4d7c0b47d0 0x7f4d7c145760 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f4d7000e9b0 tx=0x7f4d7000ee70 comp rx=0 tx=0).stop 2024-05-11T08:06:17.768 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.765+0000 7f4d80a1d640 1 -- 172.21.15.167:0/1160539987 shutdown_connections 2024-05-11T08:06:17.768 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.765+0000 7f4d80a1d640 1 --2- 172.21.15.167:0/1160539987 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f4d5c076320 0x7f4d5c0787e0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:17.768 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.765+0000 7f4d80a1d640 1 --2- 172.21.15.167:0/1160539987 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4d7c14c8a0 0x7f4d7c1faa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:17.768 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.765+0000 7f4d80a1d640 1 --2- 172.21.15.167:0/1160539987 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4d7c0b47d0 0x7f4d7c145760 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:17.768 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.765+0000 7f4d80a1d640 1 --2- 172.21.15.167:0/1160539987 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4d7c06d460 0x7f4d7c163280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:17.768 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.765+0000 7f4d80a1d640 1 -- 172.21.15.167:0/1160539987 >> 172.21.15.167:0/1160539987 conn(0x7f4d7c0cd8f0 msgr2=0x7f4d7c14a330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:17.768 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.765+0000 7f4d80a1d640 1 -- 172.21.15.167:0/1160539987 shutdown_connections 2024-05-11T08:06:17.768 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.765+0000 7f4d80a1d640 1 -- 172.21.15.167:0/1160539987 wait complete. 2024-05-11T08:06:17.769 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.765+0000 7f8ad4454640 1 -- 172.21.15.167:0/2979835464 shutdown_connections 2024-05-11T08:06:17.769 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.765+0000 7f8ad4454640 1 -- 172.21.15.167:0/2979835464 wait complete. 2024-05-11T08:06:17.770 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.767+0000 7fb2db4c2640 1 -- 172.21.15.167:0/3918353499 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fb2b80762d0 msgr2=0x7fb2b8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:17.770 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.767+0000 7fb2db4c2640 1 --2- 172.21.15.167:0/3918353499 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fb2b80762d0 0x7fb2b8078790 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb2d40515a0 tx=0x7fb2d406a230 comp rx=0 tx=0).stop 2024-05-11T08:06:17.770 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.767+0000 7fb2db4c2640 1 -- 172.21.15.167:0/3918353499 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb2cc050e50 msgr2=0x7fb2cc0de380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:17.770 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.767+0000 7fb2db4c2640 1 --2- 172.21.15.167:0/3918353499 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb2cc050e50 0x7fb2cc0de380 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fb2c400b3c0 tx=0x7fb2c400b880 comp rx=0 tx=0).stop 2024-05-11T08:06:17.770 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.768+0000 7f21497e4640 1 -- 172.21.15.167:0/3389626582 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f2144072f40 con 0x7f214415f160 2024-05-11T08:06:17.770 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.769+0000 7f2140ff9640 1 -- 172.21.15.167:0/3389626582 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f2134060ab0 con 0x7f214415f160 2024-05-11T08:06:17.771 INFO:teuthology.orchestra.run.smithi167.stdout:64424509449 2024-05-11T08:06:17.771 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.770+0000 7fca3b59e640 1 -- 172.21.15.167:0/1866920501 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fca34052930 con 0x7fca3403a5d0 2024-05-11T08:06:17.771 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.770+0000 7fca2affd640 1 -- 172.21.15.167:0/1866920501 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fca300896b0 con 0x7fca3403a5d0 2024-05-11T08:06:17.772 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.770+0000 7fb2db4c2640 1 -- 172.21.15.167:0/3918353499 shutdown_connections 2024-05-11T08:06:17.773 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.770+0000 7fb2db4c2640 1 --2- 172.21.15.167:0/3918353499 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fb2b80762d0 0x7fb2b8078790 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:17.773 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.770+0000 7fb2db4c2640 1 --2- 172.21.15.167:0/3918353499 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb2cc050e50 0x7fb2cc0de380 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:17.773 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.770+0000 7fb2db4c2640 1 --2- 172.21.15.167:0/3918353499 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb2cc0e12a0 0x7fb2cc04de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:17.773 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.770+0000 7fb2db4c2640 1 --2- 172.21.15.167:0/3918353499 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb2cc053bf0 0x7fb2cc0d3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:17.773 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.770+0000 7fb2db4c2640 1 -- 172.21.15.167:0/3918353499 >> 172.21.15.167:0/3918353499 conn(0x7fb2cc020030 msgr2=0x7fb2cc0c7150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:17.773 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.770+0000 7fb2db4c2640 1 -- 172.21.15.167:0/3918353499 shutdown_connections 2024-05-11T08:06:17.773 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.771+0000 7fb2db4c2640 1 -- 172.21.15.167:0/3918353499 wait complete. 2024-05-11T08:06:17.773 INFO:teuthology.orchestra.run.smithi167.stdout:64424509450 2024-05-11T08:06:17.775 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.773+0000 7f21497e4640 1 -- 172.21.15.167:0/3389626582 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f2124076320 msgr2=0x7f21240787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:17.775 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.773+0000 7f21497e4640 1 --2- 172.21.15.167:0/3389626582 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f2124076320 0x7f21240787e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f213c034000 tx=0x7f213c02e000 comp rx=0 tx=0).stop 2024-05-11T08:06:17.775 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.773+0000 7f21497e4640 1 -- 172.21.15.167:0/3389626582 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f214415f160 msgr2=0x7f2144148bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:17.775 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.773+0000 7f21497e4640 1 --2- 172.21.15.167:0/3389626582 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f214415f160 0x7f2144148bb0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f213400b990 tx=0x7f213400be50 comp rx=0 tx=0).stop 2024-05-11T08:06:17.775 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.774+0000 7f21497e4640 1 -- 172.21.15.167:0/3389626582 shutdown_connections 2024-05-11T08:06:17.775 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.774+0000 7f21497e4640 1 --2- 172.21.15.167:0/3389626582 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f2124076320 0x7f21240787e0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:17.775 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.774+0000 7f21497e4640 1 --2- 172.21.15.167:0/3389626582 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f214415f160 0x7f2144148bb0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:17.775 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.774+0000 7f21497e4640 1 --2- 172.21.15.167:0/3389626582 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f21440cf410 0x7f214414b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:17.776 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.774+0000 7f21497e4640 1 --2- 172.21.15.167:0/3389626582 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2144072630 0x7f214415b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:17.776 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.774+0000 7fca3b59e640 1 -- 172.21.15.167:0/1866920501 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fca1c076320 msgr2=0x7fca1c0787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:17.776 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.774+0000 7f21497e4640 1 -- 172.21.15.167:0/3389626582 >> 172.21.15.167:0/3389626582 conn(0x7f21440b33d0 msgr2=0x7f21440ccdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:17.776 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.774+0000 7fca3b59e640 1 --2- 172.21.15.167:0/1866920501 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fca1c076320 0x7fca1c0787e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fca340de6b0 tx=0x7fca2c009040 comp rx=0 tx=0).stop 2024-05-11T08:06:17.776 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.774+0000 7fca3b59e640 1 -- 172.21.15.167:0/1866920501 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca3403a5d0 msgr2=0x7fca340cfde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:17.777 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.774+0000 7fca3b59e640 1 --2- 172.21.15.167:0/1866920501 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca3403a5d0 0x7fca340cfde0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fca30038040 tx=0x7fca300046c0 comp rx=0 tx=0).stop 2024-05-11T08:06:17.777 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.774+0000 7f21497e4640 1 -- 172.21.15.167:0/3389626582 shutdown_connections 2024-05-11T08:06:17.777 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.774+0000 7f21497e4640 1 -- 172.21.15.167:0/3389626582 wait complete. 2024-05-11T08:06:17.778 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.774+0000 7fca3b59e640 1 -- 172.21.15.167:0/1866920501 shutdown_connections 2024-05-11T08:06:17.778 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.774+0000 7fca3b59e640 1 --2- 172.21.15.167:0/1866920501 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fca1c076320 0x7fca1c0787e0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:17.778 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.774+0000 7fca3b59e640 1 --2- 172.21.15.167:0/1866920501 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fca340e1090 0x7fca340376e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:17.778 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.774+0000 7fca3b59e640 1 --2- 172.21.15.167:0/1866920501 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fca34050dd0 0x7fca3404e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:17.778 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.774+0000 7fca3b59e640 1 --2- 172.21.15.167:0/1866920501 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca3403a5d0 0x7fca340cfde0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:17.778 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.775+0000 7fca3b59e640 1 -- 172.21.15.167:0/1866920501 >> 172.21.15.167:0/1866920501 conn(0x7fca34009fb0 msgr2=0x7fca34034e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:17.778 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.775+0000 7fca3b59e640 1 -- 172.21.15.167:0/1866920501 shutdown_connections 2024-05-11T08:06:17.778 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.775+0000 7fca3b59e640 1 -- 172.21.15.167:0/1866920501 wait complete. 2024-05-11T08:06:17.788 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509450 got 64424509449 for osd.7 2024-05-11T08:06:17.789 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.788+0000 7f566eb22640 1 -- 172.21.15.167:0/940586582 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f56680d46e0 con 0x7f56680d0640 2024-05-11T08:06:17.790 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509450 got 64424509449 for osd.5 2024-05-11T08:06:17.790 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.789+0000 7f56657fa640 1 -- 172.21.15.167:0/940586582 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f56680d46e0 con 0x7f56680d0640 2024-05-11T08:06:17.790 INFO:teuthology.orchestra.run.smithi167.stdout:64424509449 2024-05-11T08:06:17.791 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509450 got 64424509449 for osd.0 2024-05-11T08:06:17.792 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.791+0000 7f5b1acc2640 1 -- 172.21.15.167:0/2681710861 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f5b140b69c0 con 0x7f5b1406c450 2024-05-11T08:06:17.792 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.791+0000 7f566eb22640 1 -- 172.21.15.167:0/940586582 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f56540762d0 msgr2=0x7f5654078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:17.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.791+0000 7f566eb22640 1 --2- 172.21.15.167:0/940586582 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f56540762d0 0x7f5654078790 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f565c034000 tx=0x7f565c02e000 comp rx=0 tx=0).stop 2024-05-11T08:06:17.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.791+0000 7f566eb22640 1 -- 172.21.15.167:0/940586582 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f56680d0640 msgr2=0x7f56680c6820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:17.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.791+0000 7f566eb22640 1 --2- 172.21.15.167:0/940586582 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f56680d0640 0x7f56680c6820 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f566000ba50 tx=0x7f566000bf10 comp rx=0 tx=0).stop 2024-05-11T08:06:17.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.791+0000 7f566eb22640 1 -- 172.21.15.167:0/940586582 shutdown_connections 2024-05-11T08:06:17.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.791+0000 7f566eb22640 1 --2- 172.21.15.167:0/940586582 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f56540762d0 0x7f5654078790 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:17.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.791+0000 7f566eb22640 1 --2- 172.21.15.167:0/940586582 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f566815f3f0 0x7f56680c6d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:17.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.791+0000 7f566eb22640 1 --2- 172.21.15.167:0/940586582 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f56680d0640 0x7f56680c6820 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:17.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.791+0000 7f566eb22640 1 --2- 172.21.15.167:0/940586582 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f566806c7f0 0x7f56680cf160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:17.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.792+0000 7f566eb22640 1 -- 172.21.15.167:0/940586582 >> 172.21.15.167:0/940586582 conn(0x7f56680d51e0 msgr2=0x7f56680d0b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:17.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.792+0000 7f5b0a7fc640 1 -- 172.21.15.167:0/2681710861 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f5b10081370 con 0x7f5b1406c450 2024-05-11T08:06:17.794 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.792+0000 7f566eb22640 1 -- 172.21.15.167:0/940586582 shutdown_connections 2024-05-11T08:06:17.794 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.792+0000 7f566eb22640 1 -- 172.21.15.167:0/940586582 wait complete. 2024-05-11T08:06:17.794 INFO:teuthology.orchestra.run.smithi167.stdout:64424509450 2024-05-11T08:06:17.795 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574857 for osd.2 2024-05-11T08:06:17.796 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.795+0000 7f5b1acc2640 1 -- 172.21.15.167:0/2681710861 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f5af8076320 msgr2=0x7f5af80787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:17.796 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.795+0000 7f5b1acc2640 1 --2- 172.21.15.167:0/2681710861 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f5af8076320 0x7f5af80787e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f5b1414dbc0 tx=0x7f5b04008040 comp rx=0 tx=0).stop 2024-05-11T08:06:17.796 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.795+0000 7f5b1acc2640 1 -- 172.21.15.167:0/2681710861 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5b1406c450 msgr2=0x7f5b14149640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:17.796 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.795+0000 7f5b1acc2640 1 --2- 172.21.15.167:0/2681710861 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5b1406c450 0x7f5b14149640 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5b1002a7f0 tx=0x7f5b100046c0 comp rx=0 tx=0).stop 2024-05-11T08:06:17.796 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.795+0000 7f5b1acc2640 1 -- 172.21.15.167:0/2681710861 shutdown_connections 2024-05-11T08:06:17.796 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.795+0000 7f5b1acc2640 1 --2- 172.21.15.167:0/2681710861 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f5af8076320 0x7f5af80787e0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:17.796 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.795+0000 7f5b1acc2640 1 --2- 172.21.15.167:0/2681710861 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5b1414f520 0x7f5b1414f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:17.796 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.795+0000 7f5b1acc2640 1 --2- 172.21.15.167:0/2681710861 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5b1415f170 0x7f5b14151f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:17.797 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.795+0000 7f5b1acc2640 1 --2- 172.21.15.167:0/2681710861 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5b1406c450 0x7f5b14149640 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:17.797 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.795+0000 7f5b1acc2640 1 -- 172.21.15.167:0/2681710861 >> 172.21.15.167:0/2681710861 conn(0x7f5b14070f00 msgr2=0x7f5b14161a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:17.797 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.795+0000 7f5b1acc2640 1 -- 172.21.15.167:0/2681710861 shutdown_connections 2024-05-11T08:06:17.797 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:17.795+0000 7f5b1acc2640 1 -- 172.21.15.167:0/2681710861 wait complete. 2024-05-11T08:06:17.797 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509450 got 64424509449 for osd.4 2024-05-11T08:06:17.798 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509451 got 64424509450 for osd.3 2024-05-11T08:06:17.806 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509450 got 64424509449 for osd.6 2024-05-11T08:06:17.808 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509451 got 64424509450 for osd.1 2024-05-11T08:06:18.789 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-05-11T08:06:18.790 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-05-11T08:06:18.792 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-05-11T08:06:18.796 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-05-11T08:06:18.798 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-05-11T08:06:18.799 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-05-11T08:06:18.806 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-05-11T08:06:18.809 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-05-11T08:06:18.947 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:18.945+0000 7f0ea5a72640 1 Processor -- start 2024-05-11T08:06:18.947 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:18.945+0000 7f0ea5a72640 1 -- start start 2024-05-11T08:06:18.948 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:18.946+0000 7f0ea5a72640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f0e980df7f0 0x7f0e9804fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:18.948 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:18.947+0000 7f0ea5a72640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0e98006210 0x7f0e98006620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:18.949 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:18.947+0000 7f0ea5a72640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e98006c20 con 0x7f0e98055d00 2024-05-11T08:06:18.949 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:18.947+0000 7f0ea5a72640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f0e98006f00 con 0x7f0e980df7f0 2024-05-11T08:06:18.949 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:18.947+0000 7f0ea5a72640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f0e980071e0 con 0x7f0e98006210 2024-05-11T08:06:18.950 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:18.947+0000 7f0e9effd640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f0e980df7f0 0x7f0e9804fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:18.950 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:18.947+0000 7f0e9effd640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f0e980df7f0 0x7f0e9804fa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:54454/0 (socket says 172.21.15.167:54454) 2024-05-11T08:06:18.950 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:18.947+0000 7f0e9effd640 1 -- 172.21.15.167:0/3105682189 learned_addr learned my addr 172.21.15.167:0/3105682189 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:18.950 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:18.947+0000 7f0e9effd640 1 -- 172.21.15.167:0/3105682189 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0e98006210 msgr2=0x7f0e98006620 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-11T08:06:18.950 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:18.947+0000 7f0e9effd640 1 --2- 172.21.15.167:0/3105682189 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0e98006210 0x7f0e98006620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:18.950 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:18.947+0000 7f0e9effd640 1 -- 172.21.15.167:0/3105682189 >> v1:172.21.15.204:6789/0 conn(0x7f0e98055d00 legacy=0x7f0e98036690 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-11T08:06:18.950 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:18.947+0000 7f0e9effd640 1 -- 172.21.15.167:0/3105682189 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e980074c0 con 0x7f0e980df7f0 2024-05-11T08:06:18.950 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:18.948+0000 7f4f8cb38640 1 Processor -- start 2024-05-11T08:06:18.950 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:18.948+0000 7f4f8cb38640 1 -- start start 2024-05-11T08:06:18.951 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:18.949+0000 7f4f8cb38640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f4f8806db70 0x7f4f88072340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:18.951 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:18.949+0000 7f4f8cb38640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f88073030 con 0x7f4f88146840 2024-05-11T08:06:18.951 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:18.949+0000 7f4f8cb38640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f881490c0 con 0x7f4f88072880 2024-05-11T08:06:18.951 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:18.949+0000 7f4f8cb38640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f4f881493a0 con 0x7f4f8806db70 2024-05-11T08:06:18.951 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:18.950+0000 7f4f8659c640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f4f88146840 0x7f4f8806d460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:60840/0 (socket says 172.21.15.167:60840) 2024-05-11T08:06:18.951 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:18.950+0000 7f4f8659c640 1 -- 172.21.15.167:0/1900265187 learned_addr learned my addr 172.21.15.167:0/1900265187 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T08:06:18.953 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:18.950+0000 7f4f8559a640 1 -- 172.21.15.167:0/1900265187 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 470369210 0 0) 0x7f4f88073030 con 0x7f4f88146840 2024-05-11T08:06:18.954 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:18.950+0000 7f4f85d9b640 1 --2- 172.21.15.167:0/1900265187 >> v2:172.21.15.204:3300/0 conn(0x7f4f8806db70 0x7f4f88072340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:18.954 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:18.950+0000 7f4f8559a640 1 -- 172.21.15.167:0/1900265187 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4f70003520 con 0x7f4f88146840 2024-05-11T08:06:18.954 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:18.950+0000 7f4f8559a640 1 -- 172.21.15.167:0/1900265187 <== mon.0 v1:172.21.15.167:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3040930460 0 0) 0x7f4f70003520 con 0x7f4f88146840 2024-05-11T08:06:18.954 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:18.950+0000 7f4f8559a640 1 -- 172.21.15.167:0/1900265187 >> v2:172.21.15.204:3300/0 conn(0x7f4f8806db70 msgr2=0x7f4f88072340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:18.954 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:18.950+0000 7f4f8559a640 1 --2- 172.21.15.167:0/1900265187 >> v2:172.21.15.204:3300/0 conn(0x7f4f8806db70 0x7f4f88072340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:18.954 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:18.950+0000 7f4f8559a640 1 -- 172.21.15.167:0/1900265187 >> v1:172.21.15.204:6789/0 conn(0x7f4f88072880 legacy=0x7f4f8806e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:18.954 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:18.951+0000 7f4f8559a640 1 -- 172.21.15.167:0/1900265187 --> v1:172.21.15.167:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f88149680 con 0x7f4f88146840 2024-05-11T08:06:18.954 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:18.951+0000 7f4f8559a640 1 -- 172.21.15.167:0/1900265187 <== mon.0 v1:172.21.15.167:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3602277846 0 0) 0x7f4f780016f0 con 0x7f4f88146840 2024-05-11T08:06:18.954 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:18.951+0000 7f4f8559a640 1 -- 172.21.15.167:0/1900265187 >> v1:172.21.15.167:6789/0 conn(0x7f4f88146840 legacy=0x7f4f8806d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:18.954 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:18.951+0000 7f4f8559a640 1 --2- 172.21.15.167:0/1900265187 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4f70003a50 0x7f4f70003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:18.954 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:18.951+0000 7f4f8559a640 1 --2- 172.21.15.167:0/1900265187 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4f70004800 0x7f4f70004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:18.954 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:18.951+0000 7f4f8559a640 1 --2- 172.21.15.167:0/1900265187 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4f70005210 0x7f4f7000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:18.955 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:18.951+0000 7f4f8559a640 1 -- 172.21.15.167:0/1900265187 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4f881493a0 con 0x7f4f70003a50 2024-05-11T08:06:18.955 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:18.951+0000 7f4f8559a640 1 -- 172.21.15.167:0/1900265187 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4f7000c160 con 0x7f4f70004800 2024-05-11T08:06:18.955 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:18.951+0000 7f4f8559a640 1 -- 172.21.15.167:0/1900265187 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f4f7000c360 con 0x7f4f70005210 2024-05-11T08:06:18.955 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:18.951+0000 7f4f85d9b640 1 --2- 172.21.15.167:0/1900265187 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4f70004800 0x7f4f70004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:18.955 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:18.951+0000 7f4f86d9d640 1 --2- 172.21.15.167:0/1900265187 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4f70005210 0x7f4f7000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:18.955 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:18.951+0000 7f4f85d9b640 1 -- 172.21.15.167:0/1900265187 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4f70005210 msgr2=0x7f4f7000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:18.955 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:18.951+0000 7f4f85d9b640 1 --2- 172.21.15.167:0/1900265187 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4f70005210 0x7f4f7000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:18.955 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:18.951+0000 7f4f85d9b640 1 -- 172.21.15.167:0/1900265187 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4f70003a50 msgr2=0x7f4f70003e70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-11T08:06:18.955 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:18.951+0000 7f4f85d9b640 1 --2- 172.21.15.167:0/1900265187 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4f70003a50 0x7f4f70003e70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:18.955 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:18.951+0000 7f4f85d9b640 1 -- 172.21.15.167:0/1900265187 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f88149680 con 0x7f4f70004800 2024-05-11T08:06:18.955 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:18.952+0000 7f4f86d9d640 1 -- 172.21.15.167:0/1900265187 reap_dead start 2024-05-11T08:06:18.956 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:18.952+0000 7f0e9effd640 1 --2- 172.21.15.167:0/3105682189 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0e980df7f0 0x7f0e9804fa30 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f0e94009870 tx=0x7f0e9402a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d38a1408ce34fd4 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:18.956 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:18.952+0000 7f0e9e7fc640 1 -- 172.21.15.167:0/3105682189 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e94037070 con 0x7f0e980df7f0 2024-05-11T08:06:18.956 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:18.952+0000 7f0e9e7fc640 1 -- 172.21.15.167:0/3105682189 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e94032440 con 0x7f0e980df7f0 2024-05-11T08:06:18.957 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:18.953+0000 7f4f85d9b640 1 --2- 172.21.15.167:0/1900265187 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4f70004800 0x7f4f70004c80 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4f80007360 tx=0x7f4f70005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=39627ac9fb4d9a8d server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:18.957 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:18.953+0000 7f4f8559a640 1 -- 172.21.15.167:0/1900265187 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f80033070 con 0x7f4f70004800 2024-05-11T08:06:18.957 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:18.953+0000 7f4f8559a640 1 -- 172.21.15.167:0/1900265187 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f8002f680 con 0x7f4f70004800 2024-05-11T08:06:18.957 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:18.953+0000 7f4f8559a640 1 -- 172.21.15.167:0/1900265187 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f80009be0 con 0x7f4f70004800 2024-05-11T08:06:18.957 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:18.954+0000 7f0ea5a72640 1 -- 172.21.15.167:0/3105682189 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0e980df7f0 msgr2=0x7f0e9804fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:18.957 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:18.954+0000 7f0ea5a72640 1 --2- 172.21.15.167:0/3105682189 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0e980df7f0 0x7f0e9804fa30 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f0e94009870 tx=0x7f0e9402a250 comp rx=0 tx=0).stop 2024-05-11T08:06:18.957 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:18.954+0000 7f0ea5a72640 1 -- 172.21.15.167:0/3105682189 shutdown_connections 2024-05-11T08:06:18.958 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:18.954+0000 7f0ea5a72640 1 --2- 172.21.15.167:0/3105682189 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0e98006210 0x7f0e98006620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:18.958 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:18.954+0000 7f0ea5a72640 1 --2- 172.21.15.167:0/3105682189 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0e980df7f0 0x7f0e9804fa30 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:18.958 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:18.954+0000 7f0ea5a72640 1 -- 172.21.15.167:0/3105682189 >> 172.21.15.167:0/3105682189 conn(0x7f0e9801e6b0 msgr2=0x7f0e980553e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:18.958 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:18.954+0000 7f0ea5a72640 1 -- 172.21.15.167:0/3105682189 shutdown_connections 2024-05-11T08:06:18.959 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:18.955+0000 7f4f8cb38640 1 -- 172.21.15.167:0/1900265187 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4f70004800 msgr2=0x7f4f70004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:18.959 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:18.955+0000 7f4f8cb38640 1 --2- 172.21.15.167:0/1900265187 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4f70004800 0x7f4f70004c80 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4f80007360 tx=0x7f4f70005830 comp rx=0 tx=0).stop 2024-05-11T08:06:18.959 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:18.955+0000 7f4f8cb38640 1 -- 172.21.15.167:0/1900265187 shutdown_connections 2024-05-11T08:06:18.959 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:18.955+0000 7f4f8cb38640 1 --2- 172.21.15.167:0/1900265187 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4f70004800 0x7f4f70004c80 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:18.959 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:18.955+0000 7f4f8cb38640 1 -- 172.21.15.167:0/1900265187 >> 172.21.15.167:0/1900265187 conn(0x7f4f880cd8f0 msgr2=0x7f4f880cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:18.959 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:18.955+0000 7f4f8cb38640 1 -- 172.21.15.167:0/1900265187 shutdown_connections 2024-05-11T08:06:18.960 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:18.954+0000 7f0ea5a72640 1 -- 172.21.15.167:0/3105682189 wait complete. 2024-05-11T08:06:18.960 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:18.956+0000 7f0ea5a72640 1 Processor -- start 2024-05-11T08:06:18.960 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:18.955+0000 7f4f8cb38640 1 -- 172.21.15.167:0/1900265187 wait complete. 2024-05-11T08:06:18.960 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:18.956+0000 7f0ea5a72640 1 -- start start 2024-05-11T08:06:18.960 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:18.957+0000 7f0ea5a72640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0e98006210 0x7f0e98185b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:18.961 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:18.957+0000 7f0ea5a72640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0e98055d00 0x7f0e9804b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:18.961 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:18.957+0000 7f0ea5a72640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0e980df7f0 0x7f0e9804d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:18.961 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:18.958+0000 7f4f8cb38640 1 Processor -- start 2024-05-11T08:06:18.961 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:18.958+0000 7f0ea5a72640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0e9804ff70 con 0x7f0e980df7f0 2024-05-11T08:06:18.962 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:18.958+0000 7f0ea5a72640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0e980560e0 con 0x7f0e98006210 2024-05-11T08:06:18.962 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:18.958+0000 7f0ea5a72640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f0e980500f0 con 0x7f0e98055d00 2024-05-11T08:06:18.962 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:18.958+0000 7f0e9effd640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0e98055d00 0x7f0e9804b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:18.962 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:18.958+0000 7f0e9ffff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0e980df7f0 0x7f0e9804d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:18.962 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:18.958+0000 7f0e9effd640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0e98055d00 0x7f0e9804b890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.167:37340/0 (socket says 172.21.15.167:37340) 2024-05-11T08:06:18.962 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:18.958+0000 7f0e9effd640 1 -- 172.21.15.167:0/3743198298 learned_addr learned my addr 172.21.15.167:0/3743198298 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:18.962 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:18.958+0000 7f0e9f7fe640 1 --2- 172.21.15.167:0/3743198298 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0e98006210 0x7f0e98185b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:18.962 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:18.958+0000 7f0e9effd640 1 -- 172.21.15.167:0/3743198298 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0e98006210 msgr2=0x7f0e98185b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:18.962 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:18.958+0000 7f0e9effd640 1 --2- 172.21.15.167:0/3743198298 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0e98006210 0x7f0e98185b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:18.962 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:18.958+0000 7f0e9effd640 1 -- 172.21.15.167:0/3743198298 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0e980df7f0 msgr2=0x7f0e9804d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:18.962 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:18.958+0000 7f0e9effd640 1 --2- 172.21.15.167:0/3743198298 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0e980df7f0 0x7f0e9804d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:18.962 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:18.959+0000 7f0e9effd640 1 -- 172.21.15.167:0/3743198298 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e98186180 con 0x7f0e98055d00 2024-05-11T08:06:18.963 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:18.959+0000 7f4f8cb38640 1 -- start start 2024-05-11T08:06:18.963 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:18.960+0000 7f0e9effd640 1 --2- 172.21.15.167:0/3743198298 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0e98055d00 0x7f0e9804b890 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f0e94009870 tx=0x7f0e940046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:18.963 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:18.960+0000 7f0e8bfff640 1 -- 172.21.15.167:0/3743198298 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e94041020 con 0x7f0e98055d00 2024-05-11T08:06:18.963 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:18.960+0000 7f0e8bfff640 1 -- 172.21.15.167:0/3743198298 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e94032440 con 0x7f0e98055d00 2024-05-11T08:06:18.963 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:18.960+0000 7f0e8bfff640 1 -- 172.21.15.167:0/3743198298 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e94037040 con 0x7f0e98055d00 2024-05-11T08:06:18.964 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:18.960+0000 7f0ea5a72640 1 -- 172.21.15.167:0/3743198298 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e98186460 con 0x7f0e98055d00 2024-05-11T08:06:18.964 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:18.960+0000 7f0ea5a72640 1 -- 172.21.15.167:0/3743198298 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e98186a20 con 0x7f0e98055d00 2024-05-11T08:06:18.965 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:18.960+0000 7f4f8cb38640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4f70004800 0x7f4f8815cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:18.965 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:18.960+0000 7f4f8cb38640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4f8815f060 0x7f4f88162c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:18.965 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:18.961+0000 7f4f8659c640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4f70004800 0x7f4f8815cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:18.965 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:18.961+0000 7f4f8659c640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4f70004800 0x7f4f8815cf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.167:37348/0 (socket says 172.21.15.167:37348) 2024-05-11T08:06:18.965 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:18.961+0000 7f4f8659c640 1 -- 172.21.15.167:0/3317991600 learned_addr learned my addr 172.21.15.167:0/3317991600 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:18.965 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:18.961+0000 7f4f8cb38640 1 --2- 172.21.15.167:0/3317991600 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4f88160a00 0x7f4f88160e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:18.965 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:18.961+0000 7f4f86d9d640 1 --2- 172.21.15.167:0/3317991600 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4f88160a00 0x7f4f88160e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:18.966 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:18.962+0000 7f4f8cb38640 1 -- 172.21.15.167:0/3317991600 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4f88146e90 con 0x7f4f88160a00 2024-05-11T08:06:18.966 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:18.962+0000 7f4f8cb38640 1 -- 172.21.15.167:0/3317991600 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4f88072d30 con 0x7f4f8815f060 2024-05-11T08:06:18.966 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:18.962+0000 7f4f8cb38640 1 -- 172.21.15.167:0/3317991600 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f4f88072340 con 0x7f4f70004800 2024-05-11T08:06:18.966 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:18.962+0000 7f4f8659c640 1 -- 172.21.15.167:0/3317991600 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4f8815f060 msgr2=0x7f4f88162c20 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-11T08:06:18.966 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:18.963+0000 7f0e8bfff640 1 -- 172.21.15.167:0/3743198298 <== mon.2 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f0e94044020 con 0x7f0e98055d00 2024-05-11T08:06:18.967 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:18.962+0000 7f4f8659c640 1 --2- 172.21.15.167:0/3317991600 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4f8815f060 0x7f4f88162c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:18.967 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:18.962+0000 7f4f8659c640 1 -- 172.21.15.167:0/3317991600 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4f88160a00 msgr2=0x7f4f88160e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:18.967 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:18.962+0000 7f4f8659c640 1 --2- 172.21.15.167:0/3317991600 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4f88160a00 0x7f4f88160e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:18.967 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:18.963+0000 7f4f8659c640 1 -- 172.21.15.167:0/3317991600 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f8815d4a0 con 0x7f4f70004800 2024-05-11T08:06:18.967 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:18.963+0000 7f4f86d9d640 1 --2- 172.21.15.167:0/3317991600 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4f88160a00 0x7f4f88160e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T08:06:18.968 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:18.963+0000 7f4f8659c640 1 --2- 172.21.15.167:0/3317991600 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4f70004800 0x7f4f8815cf60 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f4f7800c7a0 tx=0x7f4f7800cc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:18.968 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:18.964+0000 7f0e8bfff640 1 --2- 172.21.15.167:0/3743198298 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f0e900764e0 0x7f0e900789a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:18.968 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:18.964+0000 7f0e8bfff640 1 -- 172.21.15.167:0/3743198298 <== mon.2 v2:172.21.15.204:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f0e940b8260 con 0x7f0e98055d00 2024-05-11T08:06:18.968 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:18.964+0000 7f0e9f7fe640 1 --2- 172.21.15.167:0/3743198298 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f0e900764e0 0x7f0e900789a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:18.968 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:18.965+0000 7f0ea5a72640 1 -- 172.21.15.167:0/3743198298 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e80005290 con 0x7f0e98055d00 2024-05-11T08:06:18.969 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:18.964+0000 7f4f76ffd640 1 -- 172.21.15.167:0/3317991600 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f78018070 con 0x7f4f70004800 2024-05-11T08:06:18.969 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:18.965+0000 7f4f76ffd640 1 -- 172.21.15.167:0/3317991600 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f780074e0 con 0x7f4f70004800 2024-05-11T08:06:18.969 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:18.965+0000 7f4f76ffd640 1 -- 172.21.15.167:0/3317991600 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f78004a20 con 0x7f4f70004800 2024-05-11T08:06:18.969 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:18.965+0000 7f4f8cb38640 1 -- 172.21.15.167:0/3317991600 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f8815fc40 con 0x7f4f70004800 2024-05-11T08:06:18.969 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:18.965+0000 7f4f8cb38640 1 -- 172.21.15.167:0/3317991600 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f88160200 con 0x7f4f70004800 2024-05-11T08:06:18.969 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:18.965+0000 7f0e9f7fe640 1 --2- 172.21.15.167:0/3743198298 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f0e900764e0 0x7f0e900789a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0e8c006fd0 tx=0x7f0e8c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:18.970 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:18.966+0000 7f4f74ff9640 1 -- 172.21.15.167:0/3317991600 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f60005290 con 0x7f4f70004800 2024-05-11T08:06:18.970 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:18.968+0000 7f4f76ffd640 1 -- 172.21.15.167:0/3317991600 <== mon.2 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f4f7801b070 con 0x7f4f70004800 2024-05-11T08:06:18.970 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:18.968+0000 7f4f76ffd640 1 --2- 172.21.15.167:0/3317991600 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f4f68076320 0x7f4f680787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:18.971 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:18.969+0000 7f4f85d9b640 1 --2- 172.21.15.167:0/3317991600 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f4f68076320 0x7f4f680787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:18.971 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:18.969+0000 7f4f85d9b640 1 --2- 172.21.15.167:0/3317991600 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f4f68076320 0x7f4f680787e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f4f80035000 tx=0x7f4f8000b030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:18.971 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:18.970+0000 7f4f76ffd640 1 -- 172.21.15.167:0/3317991600 <== mon.2 v2:172.21.15.204:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f4f7805c1c0 con 0x7f4f70004800 2024-05-11T08:06:18.972 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:18.971+0000 7f0e8bfff640 1 -- 172.21.15.167:0/3743198298 <== mon.2 v2:172.21.15.204:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f0e9404b030 con 0x7f0e98055d00 2024-05-11T08:06:18.977 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:18.975+0000 7f4f76ffd640 1 -- 172.21.15.167:0/3317991600 <== mon.2 v2:172.21.15.204:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f4f8815d4a0 con 0x7f4f70004800 2024-05-11T08:06:19.004 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.002+0000 7f42eb5af640 1 Processor -- start 2024-05-11T08:06:19.004 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.003+0000 7fb790aeb640 1 Processor -- start 2024-05-11T08:06:19.004 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.003+0000 7fb790aeb640 1 -- start start 2024-05-11T08:06:19.005 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.003+0000 7f42eb5af640 1 -- start start 2024-05-11T08:06:19.006 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.004+0000 7fea912fc640 1 Processor -- start 2024-05-11T08:06:19.006 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.004+0000 7f42eb5af640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f42e4161320 0x7f42e40b4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:19.006 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.004+0000 7f42eb5af640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f42e40b6dc0 0x7f42e406d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:19.006 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.004+0000 7f42eb5af640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42e40cdd80 con 0x7f42e41646a0 2024-05-11T08:06:19.006 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.004+0000 7f42eb5af640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f42e406cbd0 con 0x7f42e40b6dc0 2024-05-11T08:06:19.006 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.004+0000 7f42eb5af640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f42e406c580 con 0x7f42e4161320 2024-05-11T08:06:19.007 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.005+0000 7f42e8b4a640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f42e41646a0 0x7f42e40b66b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:60870/0 (socket says 172.21.15.167:60870) 2024-05-11T08:06:19.007 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.005+0000 7f42e8b4a640 1 -- 172.21.15.167:0/3141472842 learned_addr learned my addr 172.21.15.167:0/3141472842 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T08:06:19.007 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.004+0000 7fb790aeb640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fb78c073050 0x7fb78c0b3be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:19.007 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.004+0000 7fb790aeb640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb78c15eb20 con 0x7fb78c0ce150 2024-05-11T08:06:19.007 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.004+0000 7fb790aeb640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb78c06e300 con 0x7fb78c0b4120 2024-05-11T08:06:19.007 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.004+0000 7fb790aeb640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fb78c06e5e0 con 0x7fb78c073050 2024-05-11T08:06:19.007 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.005+0000 7fb78b7fe640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7fb78c0ce150 0x7fb78c15e410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:60854/0 (socket says 172.21.15.167:60854) 2024-05-11T08:06:19.007 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.005+0000 7fb78b7fe640 1 -- 172.21.15.167:0/649807315 learned_addr learned my addr 172.21.15.167:0/649807315 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T08:06:19.008 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.004+0000 7fea912fc640 1 -- start start 2024-05-11T08:06:19.008 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.005+0000 7fea912fc640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fea840e30b0 0x7fea84057d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:19.008 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.005+0000 7fea912fc640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fea8403e830 0x7fea84053be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:19.008 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.005+0000 7fea912fc640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea840582c0 con 0x7fea84054120 2024-05-11T08:06:19.008 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.005+0000 7fea912fc640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7fea84058560 con 0x7fea8403e830 2024-05-11T08:06:19.008 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.005+0000 7fea912fc640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fea84058840 con 0x7fea840e30b0 2024-05-11T08:06:19.008 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.005+0000 7f42e934b640 1 --2- 172.21.15.167:0/3141472842 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f42e4161320 0x7f42e40b4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:19.008 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.005+0000 7f42d7fff640 1 -- 172.21.15.167:0/3141472842 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2612936452 0 0) 0x7f42e40cdd80 con 0x7f42e41646a0 2024-05-11T08:06:19.009 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.005+0000 7fb78a7fc640 1 -- 172.21.15.167:0/649807315 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 156654337 0 0) 0x7fb78c15eb20 con 0x7fb78c0ce150 2024-05-11T08:06:19.009 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.005+0000 7fea8b7fe640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7fea84054120 0x7fea840e6c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:60878/0 (socket says 172.21.15.167:60878) 2024-05-11T08:06:19.009 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.005+0000 7fb78a7fc640 1 -- 172.21.15.167:0/649807315 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb774003520 con 0x7fb78c0ce150 2024-05-11T08:06:19.010 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.005+0000 7fea8b7fe640 1 -- 172.21.15.167:0/3381229062 learned_addr learned my addr 172.21.15.167:0/3381229062 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T08:06:19.010 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.005+0000 7f84578bd640 1 Processor -- start 2024-05-11T08:06:19.010 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.005+0000 7fea8a7fc640 1 --2- 172.21.15.167:0/3381229062 >> v2:172.21.15.167:3300/0 conn(0x7fea8403e830 0x7fea84053be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:19.011 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.006+0000 7f84578bd640 1 -- start start 2024-05-11T08:06:19.011 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.006+0000 7fea8affd640 1 --2- 172.21.15.167:0/3381229062 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fea840e30b0 0x7fea84057d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:19.011 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.006+0000 7fea8a7fc640 1 -- 172.21.15.167:0/3381229062 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fea840e30b0 msgr2=0x7fea84057d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:19.011 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.006+0000 7fea8a7fc640 1 --2- 172.21.15.167:0/3381229062 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fea840e30b0 0x7fea84057d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:19.012 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.006+0000 7fea8a7fc640 1 -- 172.21.15.167:0/3381229062 >> v1:172.21.15.167:6789/0 conn(0x7fea84054120 legacy=0x7fea840e6c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:19.012 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.006+0000 7fea8a7fc640 1 -- 172.21.15.167:0/3381229062 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea84058b20 con 0x7fea8403e830 2024-05-11T08:06:19.012 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.006+0000 7fb78affd640 1 --2- 172.21.15.167:0/649807315 >> v2:172.21.15.204:3300/0 conn(0x7fb78c073050 0x7fb78c0b3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:19.013 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.005+0000 7f42d7fff640 1 -- 172.21.15.167:0/3141472842 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f42d0003520 con 0x7f42e41646a0 2024-05-11T08:06:19.013 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.006+0000 7f42e9b4c640 1 --2- 172.21.15.167:0/3141472842 >> v2:172.21.15.204:3300/0 conn(0x7f42e40b6dc0 0x7f42e406d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:19.013 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.006+0000 7f42d7fff640 1 -- 172.21.15.167:0/3141472842 <== mon.0 v1:172.21.15.167:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1820635196 0 0) 0x7f42d0003520 con 0x7f42e41646a0 2024-05-11T08:06:19.013 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.006+0000 7f42d7fff640 1 -- 172.21.15.167:0/3141472842 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f42e4161320 msgr2=0x7f42e40b4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:19.013 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.006+0000 7f42d7fff640 1 --2- 172.21.15.167:0/3141472842 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f42e4161320 0x7f42e40b4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:19.013 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.006+0000 7f42d7fff640 1 -- 172.21.15.167:0/3141472842 >> v2:172.21.15.204:3300/0 conn(0x7f42e40b6dc0 msgr2=0x7f42e406d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:19.013 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.006+0000 7f42d7fff640 1 --2- 172.21.15.167:0/3141472842 >> v2:172.21.15.204:3300/0 conn(0x7f42e40b6dc0 0x7f42e406d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:19.014 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.006+0000 7f42d7fff640 1 -- 172.21.15.167:0/3141472842 --> v1:172.21.15.167:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42e406c860 con 0x7f42e41646a0 2024-05-11T08:06:19.014 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.006+0000 7f42e9b4c640 1 --2- 172.21.15.167:0/3141472842 >> v2:172.21.15.204:3300/0 conn(0x7f42e40b6dc0 0x7f42e406d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T08:06:19.014 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.007+0000 7f42d7fff640 1 -- 172.21.15.167:0/3141472842 <== mon.0 v1:172.21.15.167:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3602277846 0 0) 0x7f42d80016f0 con 0x7f42e41646a0 2024-05-11T08:06:19.014 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.007+0000 7f42d7fff640 1 -- 172.21.15.167:0/3141472842 >> v1:172.21.15.167:6789/0 conn(0x7f42e41646a0 legacy=0x7f42e40b66b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:19.014 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.007+0000 7f42d7fff640 1 --2- 172.21.15.167:0/3141472842 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f42d0003ad0 0x7f42d0003eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:19.014 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.007+0000 7f42d7fff640 1 --2- 172.21.15.167:0/3141472842 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f42d0004800 0x7f42d0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:19.014 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.007+0000 7f42d7fff640 1 --2- 172.21.15.167:0/3141472842 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f42d0005210 0x7f42d000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:19.015 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.007+0000 7f84578bd640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f845015fee0 0x7f84500b56a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:19.015 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.007+0000 7f84578bd640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84500cfba0 con 0x7f84500d0960 2024-05-11T08:06:19.015 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.007+0000 7f84578bd640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84500cfe80 con 0x7f845006c450 2024-05-11T08:06:19.015 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.007+0000 7f84578bd640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f84500d0160 con 0x7f845015fee0 2024-05-11T08:06:19.016 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.007+0000 7fea8a7fc640 1 --2- 172.21.15.167:0/3381229062 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea8403e830 0x7fea84053be0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fea8c02e140 tx=0x7fea8c06a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4714dbc2387d1b96 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:19.016 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.007+0000 7f42e9b4c640 1 --2- 172.21.15.167:0/3141472842 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f42d0005210 0x7f42d000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:19.016 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.007+0000 7f42e934b640 1 --2- 172.21.15.167:0/3141472842 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f42d0003ad0 0x7f42d0003eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:19.016 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.007+0000 7f42d7fff640 1 -- 172.21.15.167:0/3141472842 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f42e406cbd0 con 0x7f42d0005210 2024-05-11T08:06:19.016 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.007+0000 7f42d7fff640 1 -- 172.21.15.167:0/3141472842 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f42e406c580 con 0x7f42d0003ad0 2024-05-11T08:06:19.017 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.007+0000 7f8454e58640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f845015fee0 0x7f84500b56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:19.017 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.007+0000 7f8455e5a640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f845006c450 0x7f845015f300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.167:45734/0 (socket says 172.21.15.167:45734) 2024-05-11T08:06:19.017 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.007+0000 7fc8690f8640 1 Processor -- start 2024-05-11T08:06:19.018 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.007+0000 7fea89ffb640 1 -- 172.21.15.167:0/3381229062 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea8c0b2070 con 0x7fea8403e830 2024-05-11T08:06:19.019 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.007+0000 7f8455e5a640 1 -- 172.21.15.167:0/3773561705 learned_addr learned my addr 172.21.15.167:0/3773561705 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T08:06:19.019 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.007+0000 7f42d7fff640 1 -- 172.21.15.167:0/3141472842 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f42d000c1c0 con 0x7f42d0004800 2024-05-11T08:06:19.019 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.008+0000 7fea89ffb640 1 -- 172.21.15.167:0/3381229062 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea8c0ab810 con 0x7fea8403e830 2024-05-11T08:06:19.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.008+0000 7f8447fff640 1 -- 172.21.15.167:0/3773561705 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1519539900 0 0) 0x7f84500cfe80 con 0x7f845006c450 2024-05-11T08:06:19.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.008+0000 7f8447fff640 1 -- 172.21.15.167:0/3773561705 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f843c003520 con 0x7f845006c450 2024-05-11T08:06:19.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.008+0000 7f42e9b4c640 1 -- 172.21.15.167:0/3141472842 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f42d0004800 msgr2=0x7f42d0004c80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-11T08:06:19.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.008+0000 7f42e9b4c640 1 --2- 172.21.15.167:0/3141472842 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f42d0004800 0x7f42d0004c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:19.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.008+0000 7f42e9b4c640 1 -- 172.21.15.167:0/3141472842 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f42d0003ad0 msgr2=0x7f42d0003eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:19.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.008+0000 7f42e9b4c640 1 --2- 172.21.15.167:0/3141472842 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f42d0003ad0 0x7f42d0003eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:19.021 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.007+0000 7fb78affd640 1 -- 172.21.15.167:0/649807315 >> v1:172.21.15.204:6789/0 conn(0x7fb78c0b4120 legacy=0x7fb78c06d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:19.021 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.007+0000 7fb78affd640 1 -- 172.21.15.167:0/649807315 >> v1:172.21.15.167:6789/0 conn(0x7fb78c0ce150 legacy=0x7fb78c15e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:19.021 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.007+0000 7fb78affd640 1 -- 172.21.15.167:0/649807315 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb78c06e8c0 con 0x7fb78c073050 2024-05-11T08:06:19.021 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.008+0000 7f42e9b4c640 1 -- 172.21.15.167:0/3141472842 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42d0003520 con 0x7f42d0005210 2024-05-11T08:06:19.022 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.008+0000 7f8454e58640 1 -- 172.21.15.167:0/3773561705 >> v1:172.21.15.204:6789/0 conn(0x7f845006c450 legacy=0x7f845015f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:19.022 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.008+0000 7f8454e58640 1 -- 172.21.15.167:0/3773561705 >> v1:172.21.15.167:6789/0 conn(0x7f84500d0960 legacy=0x7f84500cde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:19.022 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.008+0000 7f8454e58640 1 -- 172.21.15.167:0/3773561705 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84500d0440 con 0x7f845015fee0 2024-05-11T08:06:19.022 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.008+0000 7fb78affd640 1 --2- 172.21.15.167:0/649807315 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb78c073050 0x7fb78c0b3be0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb780009720 tx=0x7fb78002a150 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f3cbb141b49f31f server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:19.023 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.008+0000 7fea89ffb640 1 -- 172.21.15.167:0/3381229062 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea8c0abc80 con 0x7fea8403e830 2024-05-11T08:06:19.023 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.008+0000 7fc8690f8640 1 -- start start 2024-05-11T08:06:19.024 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.009+0000 7fb78a7fc640 1 -- 172.21.15.167:0/649807315 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb780037070 con 0x7fb78c073050 2024-05-11T08:06:19.024 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.009+0000 7fb78a7fc640 1 -- 172.21.15.167:0/649807315 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb780032440 con 0x7fb78c073050 2024-05-11T08:06:19.024 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.009+0000 7fb78a7fc640 1 -- 172.21.15.167:0/649807315 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb78002ae60 con 0x7fb78c073050 2024-05-11T08:06:19.025 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.009+0000 7fea912fc640 1 -- 172.21.15.167:0/3381229062 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea8403e830 msgr2=0x7fea84053be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:19.025 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.009+0000 7fea912fc640 1 --2- 172.21.15.167:0/3381229062 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea8403e830 0x7fea84053be0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fea8c02e140 tx=0x7fea8c06a230 comp rx=0 tx=0).stop 2024-05-11T08:06:19.025 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.009+0000 7f42e9b4c640 1 --2- 172.21.15.167:0/3141472842 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f42d0005210 0x7f42d000b9c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f42e406e840 tx=0x7f42d0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a6f200fc18833dd server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:19.026 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.009+0000 7fc8690f8640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc864155c00 0x7fc8640cf210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:19.026 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.009+0000 7fc8690f8640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc86415bee0 con 0x7fc864155820 2024-05-11T08:06:19.026 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.009+0000 7fc8690f8640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc86415c1c0 con 0x7fc864073370 2024-05-11T08:06:19.027 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.009+0000 7fc8690f8640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fc8640b45f0 con 0x7fc864155c00 2024-05-11T08:06:19.027 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.010+0000 7fea912fc640 1 -- 172.21.15.167:0/3381229062 shutdown_connections 2024-05-11T08:06:19.027 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.010+0000 7fea912fc640 1 --2- 172.21.15.167:0/3381229062 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea8403e830 0x7fea84053be0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:19.027 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.010+0000 7fea912fc640 1 --2- 172.21.15.167:0/3381229062 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fea840e30b0 0x7fea84057d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:19.027 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.010+0000 7fea912fc640 1 -- 172.21.15.167:0/3381229062 >> 172.21.15.167:0/3381229062 conn(0x7fea840559d0 msgr2=0x7fea84055de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:19.028 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.009+0000 7fc862d9d640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7fc864155820 0x7fc86406d460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:60902/0 (socket says 172.21.15.167:60902) 2024-05-11T08:06:19.028 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.009+0000 7fc862d9d640 1 -- 172.21.15.167:0/2133436357 learned_addr learned my addr 172.21.15.167:0/2133436357 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T08:06:19.029 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.010+0000 7fea912fc640 1 -- 172.21.15.167:0/3381229062 shutdown_connections 2024-05-11T08:06:19.029 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.010+0000 7fc85bfff640 1 --2- 172.21.15.167:0/2133436357 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc864155c00 0x7fc8640cf210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:19.030 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.010+0000 7fc86259c640 1 -- 172.21.15.167:0/2133436357 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 101668039 0 0) 0x7fc86415bee0 con 0x7fc864155820 2024-05-11T08:06:19.030 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.010+0000 7fc86259c640 1 -- 172.21.15.167:0/2133436357 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc850003520 con 0x7fc864155820 2024-05-11T08:06:19.030 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.009+0000 7f42d7fff640 1 -- 172.21.15.167:0/3141472842 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42dc009c80 con 0x7f42d0005210 2024-05-11T08:06:19.030 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.009+0000 7f42d7fff640 1 -- 172.21.15.167:0/3141472842 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42dc003b90 con 0x7f42d0005210 2024-05-11T08:06:19.031 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.010+0000 7fc86259c640 1 -- 172.21.15.167:0/2133436357 <== mon.0 v1:172.21.15.167:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3791419950 0 0) 0x7fc850003520 con 0x7fc864155820 2024-05-11T08:06:19.031 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.010+0000 7f42eb5af640 1 -- 172.21.15.167:0/3141472842 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f42d0005210 msgr2=0x7f42d000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:19.031 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.010+0000 7f42eb5af640 1 --2- 172.21.15.167:0/3141472842 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f42d0005210 0x7f42d000b9c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f42e406e840 tx=0x7f42d0005830 comp rx=0 tx=0).stop 2024-05-11T08:06:19.032 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.010+0000 7fb790aeb640 1 -- 172.21.15.167:0/649807315 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb78c073050 msgr2=0x7fb78c0b3be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:19.032 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.010+0000 7fb790aeb640 1 --2- 172.21.15.167:0/649807315 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb78c073050 0x7fb78c0b3be0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb780009720 tx=0x7fb78002a150 comp rx=0 tx=0).stop 2024-05-11T08:06:19.032 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.010+0000 7f8454e58640 1 --2- 172.21.15.167:0/3773561705 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f845015fee0 0x7f84500b56a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f84400097a0 tx=0x7f844002a2c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f5160a419fa248f7 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:19.032 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.010+0000 7f8447fff640 1 -- 172.21.15.167:0/3773561705 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8440037070 con 0x7f845015fee0 2024-05-11T08:06:19.032 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.010+0000 7f8447fff640 1 -- 172.21.15.167:0/3773561705 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8440032640 con 0x7f845015fee0 2024-05-11T08:06:19.033 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.010+0000 7fc86259c640 1 -- 172.21.15.167:0/2133436357 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc864155c00 msgr2=0x7fc8640cf210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:19.033 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.010+0000 7fc86259c640 1 --2- 172.21.15.167:0/2133436357 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc864155c00 0x7fc8640cf210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:19.033 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.010+0000 7fc86259c640 1 -- 172.21.15.167:0/2133436357 >> v1:172.21.15.204:6789/0 conn(0x7fc864073370 legacy=0x7fc8640b4a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:19.033 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.010+0000 7fc86259c640 1 -- 172.21.15.167:0/2133436357 --> v1:172.21.15.167:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc86415c4a0 con 0x7fc864155820 2024-05-11T08:06:19.033 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.010+0000 7fc86259c640 1 -- 172.21.15.167:0/2133436357 <== mon.0 v1:172.21.15.167:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3602277846 0 0) 0x7fc8540016d0 con 0x7fc864155820 2024-05-11T08:06:19.033 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.010+0000 7fc86259c640 1 -- 172.21.15.167:0/2133436357 >> v1:172.21.15.167:6789/0 conn(0x7fc864155820 legacy=0x7fc86406d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:19.033 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.010+0000 7fc86259c640 1 --2- 172.21.15.167:0/2133436357 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc850003a40 0x7fc850003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:19.033 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.010+0000 7fc86259c640 1 --2- 172.21.15.167:0/2133436357 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc850004800 0x7fc850004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:19.033 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.010+0000 7fc86259c640 1 --2- 172.21.15.167:0/2133436357 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc8500051c0 0x7fc8500145c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:19.033 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.010+0000 7fc86259c640 1 -- 172.21.15.167:0/2133436357 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc8640b45f0 con 0x7fc850003a40 2024-05-11T08:06:19.033 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.010+0000 7fc86259c640 1 -- 172.21.15.167:0/2133436357 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc850009bc0 con 0x7fc8500051c0 2024-05-11T08:06:19.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.010+0000 7fc86259c640 1 -- 172.21.15.167:0/2133436357 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fc850014b00 con 0x7fc850004800 2024-05-11T08:06:19.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.011+0000 7fc85bfff640 1 --2- 172.21.15.167:0/2133436357 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc850004800 0x7fc850004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:19.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.010+0000 7fb790aeb640 1 -- 172.21.15.167:0/649807315 shutdown_connections 2024-05-11T08:06:19.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.010+0000 7fb790aeb640 1 --2- 172.21.15.167:0/649807315 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb78c073050 0x7fb78c0b3be0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:19.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.010+0000 7fb790aeb640 1 -- 172.21.15.167:0/649807315 >> 172.21.15.167:0/649807315 conn(0x7fb78c0cd630 msgr2=0x7fb78c0cda40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:19.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.011+0000 7fb790aeb640 1 -- 172.21.15.167:0/649807315 shutdown_connections 2024-05-11T08:06:19.035 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.010+0000 7f42e9b4c640 1 -- 172.21.15.167:0/3141472842 reap_dead start 2024-05-11T08:06:19.035 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.010+0000 7f42eb5af640 1 -- 172.21.15.167:0/3141472842 shutdown_connections 2024-05-11T08:06:19.035 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.010+0000 7f42eb5af640 1 -- 172.21.15.167:0/3141472842 >> 172.21.15.167:0/3141472842 conn(0x7f42e40710a0 msgr2=0x7f42e40b34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:19.035 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.011+0000 7f42eb5af640 1 -- 172.21.15.167:0/3141472842 shutdown_connections 2024-05-11T08:06:19.035 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.011+0000 7f42eb5af640 1 -- 172.21.15.167:0/3141472842 wait complete. 2024-05-11T08:06:19.036 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.011+0000 7f84578bd640 1 -- 172.21.15.167:0/3773561705 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f845015fee0 msgr2=0x7f84500b56a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:19.036 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.011+0000 7f84578bd640 1 --2- 172.21.15.167:0/3773561705 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f845015fee0 0x7f84500b56a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f84400097a0 tx=0x7f844002a2c0 comp rx=0 tx=0).stop 2024-05-11T08:06:19.036 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.012+0000 7f84578bd640 1 -- 172.21.15.167:0/3773561705 shutdown_connections 2024-05-11T08:06:19.036 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.012+0000 7f84578bd640 1 --2- 172.21.15.167:0/3773561705 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f845015fee0 0x7f84500b56a0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:19.036 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.012+0000 7f84578bd640 1 -- 172.21.15.167:0/3773561705 >> 172.21.15.167:0/3773561705 conn(0x7f84500710d0 msgr2=0x7f84500b34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:19.036 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.012+0000 7f84578bd640 1 -- 172.21.15.167:0/3773561705 shutdown_connections 2024-05-11T08:06:19.036 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.012+0000 7f84578bd640 1 -- 172.21.15.167:0/3773561705 wait complete. 2024-05-11T08:06:19.037 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.011+0000 7fc85bfff640 1 -- 172.21.15.167:0/2133436357 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc8500051c0 msgr2=0x7fc8500145c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-11T08:06:19.037 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.011+0000 7fc85bfff640 1 --2- 172.21.15.167:0/2133436357 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc8500051c0 0x7fc8500145c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:19.037 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.011+0000 7fc85bfff640 1 -- 172.21.15.167:0/2133436357 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc850003a40 msgr2=0x7fc850003e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:19.037 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.011+0000 7fc85bfff640 1 --2- 172.21.15.167:0/2133436357 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc850003a40 0x7fc850003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:19.037 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.011+0000 7fc85bfff640 1 -- 172.21.15.167:0/2133436357 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc86415c4a0 con 0x7fc850004800 2024-05-11T08:06:19.037 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.010+0000 7fea912fc640 1 -- 172.21.15.167:0/3381229062 wait complete. 2024-05-11T08:06:19.037 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.012+0000 7fea912fc640 1 Processor -- start 2024-05-11T08:06:19.037 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.013+0000 7f42eb5af640 1 Processor -- start 2024-05-11T08:06:19.038 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.011+0000 7fb790aeb640 1 -- 172.21.15.167:0/649807315 wait complete. 2024-05-11T08:06:19.038 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.012+0000 7fb790aeb640 1 Processor -- start 2024-05-11T08:06:19.038 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.013+0000 7f42eb5af640 1 -- start start 2024-05-11T08:06:19.038 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.013+0000 7fb790aeb640 1 -- start start 2024-05-11T08:06:19.039 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.012+0000 7fea912fc640 1 -- start start 2024-05-11T08:06:19.040 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.013+0000 7fea912fc640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fea8403e830 0x7fea8416eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:19.040 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.013+0000 7fea912fc640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea84054120 0x7fea840dbd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:19.040 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.013+0000 7fea912fc640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fea840e30b0 0x7fea840dc260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:19.040 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.013+0000 7fea912fc640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fea840e3490 con 0x7fea84054120 2024-05-11T08:06:19.040 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.013+0000 7fea912fc640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fea8403bfb0 con 0x7fea840e30b0 2024-05-11T08:06:19.040 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.013+0000 7fea912fc640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fea840078d0 con 0x7fea8403e830 2024-05-11T08:06:19.040 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.014+0000 7fea8a7fc640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea84054120 0x7fea840dbd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:19.040 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.014+0000 7f84578bd640 1 Processor -- start 2024-05-11T08:06:19.041 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.014+0000 7f84578bd640 1 -- start start 2024-05-11T08:06:19.041 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.014+0000 7fea8a7fc640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea84054120 0x7fea840dbd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:54484/0 (socket says 172.21.15.167:54484) 2024-05-11T08:06:19.041 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.014+0000 7fea8a7fc640 1 -- 172.21.15.167:0/1590117397 learned_addr learned my addr 172.21.15.167:0/1590117397 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:19.041 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.014+0000 7f42eb5af640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f42d0005210 0x7f42e41edbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:19.041 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.014+0000 7fb790aeb640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb78c073050 0x7fb78c158f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:19.042 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.014+0000 7f42eb5af640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f42e414ab20 0x7f42e4160310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:19.042 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.014+0000 7fc85bfff640 1 --2- 172.21.15.167:0/2133436357 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc850004800 0x7fc850004c80 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc84c009980 tx=0x7fc850005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3aaa87e638d6fb9b server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:19.042 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.014+0000 7fea8b7fe640 1 --2- 172.21.15.167:0/1590117397 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fea840e30b0 0x7fea840dc260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:19.043 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.014+0000 7f42eb5af640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f42e414b300 0x7f42e414b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:19.043 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.014+0000 7f42e8b4a640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f42e414ab20 0x7f42e4160310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:19.043 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.014+0000 7f42e8b4a640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f42e414ab20 0x7f42e4160310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.167:37416/0 (socket says 172.21.15.167:37416) 2024-05-11T08:06:19.043 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.014+0000 7fb790aeb640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb78c0b4120 0x7fb78c141d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:19.043 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.014+0000 7fb78b7fe640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb78c073050 0x7fb78c158f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:19.043 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.014+0000 7fb78b7fe640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb78c073050 0x7fb78c158f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.167:37408/0 (socket says 172.21.15.167:37408) 2024-05-11T08:06:19.044 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.014+0000 7f42e8b4a640 1 -- 172.21.15.167:0/1465937264 learned_addr learned my addr 172.21.15.167:0/1465937264 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:19.044 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.014+0000 7fb78b7fe640 1 -- 172.21.15.167:0/3945438221 learned_addr learned my addr 172.21.15.167:0/3945438221 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:19.044 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.014+0000 7f42e9b4c640 1 --2- 172.21.15.167:0/1465937264 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f42e414b300 0x7f42e414b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:19.044 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.015+0000 7f42eb5af640 1 -- 172.21.15.167:0/1465937264 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f42e4164b00 con 0x7f42d0005210 2024-05-11T08:06:19.045 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.015+0000 7fb790aeb640 1 --2- 172.21.15.167:0/3945438221 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb78c0ce150 0x7fb78c1563f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:19.045 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.015+0000 7f42eb5af640 1 -- 172.21.15.167:0/1465937264 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f42e40ce060 con 0x7f42e414b300 2024-05-11T08:06:19.045 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.015+0000 7f42eb5af640 1 -- 172.21.15.167:0/1465937264 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f42e40cd930 con 0x7f42e414ab20 2024-05-11T08:06:19.046 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.014+0000 7fea8a7fc640 1 -- 172.21.15.167:0/1590117397 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fea8403e830 msgr2=0x7fea8416eb90 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-11T08:06:19.046 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.014+0000 7fea8a7fc640 1 --2- 172.21.15.167:0/1590117397 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fea8403e830 0x7fea8416eb90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:19.046 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.014+0000 7fea8a7fc640 1 -- 172.21.15.167:0/1590117397 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fea840e30b0 msgr2=0x7fea840dc260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:19.046 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.014+0000 7fea8a7fc640 1 --2- 172.21.15.167:0/1590117397 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fea840e30b0 0x7fea840dc260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:19.046 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.014+0000 7fea8a7fc640 1 -- 172.21.15.167:0/1590117397 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea8416de10 con 0x7fea84054120 2024-05-11T08:06:19.046 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.015+0000 7fb790aeb640 1 -- 172.21.15.167:0/3945438221 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb78c073850 con 0x7fb78c0b4120 2024-05-11T08:06:19.047 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.015+0000 7f42e934b640 1 --2- 172.21.15.167:0/1465937264 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f42d0005210 0x7f42e41edbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:19.047 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.015+0000 7fb78bfff640 1 --2- 172.21.15.167:0/3945438221 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb78c0ce150 0x7fb78c1563f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:19.047 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.015+0000 7fea8a7fc640 1 --2- 172.21.15.167:0/1590117397 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea84054120 0x7fea840dbd20 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fea8c051d40 tx=0x7fea8c06b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:19.049 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.015+0000 7fb78affd640 1 --2- 172.21.15.167:0/3945438221 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb78c0b4120 0x7fb78c141d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:19.050 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.015+0000 7f42e8b4a640 1 -- 172.21.15.167:0/1465937264 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f42e414b300 msgr2=0x7f42e414b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:19.050 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.015+0000 7fb790aeb640 1 -- 172.21.15.167:0/3945438221 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb78c06dd30 con 0x7fb78c0ce150 2024-05-11T08:06:19.050 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.015+0000 7fb790aeb640 1 -- 172.21.15.167:0/3945438221 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fb78c0739d0 con 0x7fb78c073050 2024-05-11T08:06:19.050 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.015+0000 7f84578bd640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f845006c450 0x7f845014b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:19.051 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.015+0000 7f42e8b4a640 1 --2- 172.21.15.167:0/1465937264 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f42e414b300 0x7f42e414b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:19.051 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.015+0000 7f84578bd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f84500d0960 0x7f845014c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:19.051 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.015+0000 7fc86259c640 1 -- 172.21.15.167:0/2133436357 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc84c036070 con 0x7fc850004800 2024-05-11T08:06:19.051 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.015+0000 7fc86259c640 1 -- 172.21.15.167:0/2133436357 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc84c031400 con 0x7fc850004800 2024-05-11T08:06:19.051 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.015+0000 7fc86259c640 1 -- 172.21.15.167:0/2133436357 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc84c009650 con 0x7fc850004800 2024-05-11T08:06:19.052 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.015+0000 7f84578bd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8450151430 0x7f8450149b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:19.052 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.015+0000 7f42e8b4a640 1 -- 172.21.15.167:0/1465937264 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f42d0005210 msgr2=0x7f42e41edbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:19.052 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.015+0000 7f42e8b4a640 1 --2- 172.21.15.167:0/1465937264 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f42d0005210 0x7f42e41edbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:19.052 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.015+0000 7f84578bd640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f845006c8e0 con 0x7f8450151430 2024-05-11T08:06:19.053 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.015+0000 7f84578bd640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f84500cf850 con 0x7f84500d0960 2024-05-11T08:06:19.053 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.015+0000 7f84578bd640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f845006ca60 con 0x7f845006c450 2024-05-11T08:06:19.053 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.015+0000 7f42e8b4a640 1 -- 172.21.15.167:0/1465937264 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42e4160b40 con 0x7f42e414ab20 2024-05-11T08:06:19.053 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.015+0000 7f8454e58640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f84500d0960 0x7f845014c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:19.053 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.016+0000 7f8455e5a640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8450151430 0x7f8450149b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:19.053 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.016+0000 7f8454e58640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f84500d0960 0x7f845014c8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.167:37390/0 (socket says 172.21.15.167:37390) 2024-05-11T08:06:19.054 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.016+0000 7f8454e58640 1 -- 172.21.15.167:0/3425846513 learned_addr learned my addr 172.21.15.167:0/3425846513 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:19.054 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.016+0000 7f42e8b4a640 1 --2- 172.21.15.167:0/1465937264 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f42e414ab20 0x7f42e4160310 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f42d800c6e0 tx=0x7f42d800cba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:19.054 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.016+0000 7fb78bfff640 1 -- 172.21.15.167:0/3945438221 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb78c073050 msgr2=0x7fb78c158f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:19.055 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.016+0000 7fb78bfff640 1 --2- 172.21.15.167:0/3945438221 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb78c073050 0x7fb78c158f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:19.055 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.016+0000 7fb78bfff640 1 -- 172.21.15.167:0/3945438221 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb78c0b4120 msgr2=0x7fb78c141d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:19.055 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.016+0000 7fb78bfff640 1 --2- 172.21.15.167:0/3945438221 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb78c0b4120 0x7fb78c141d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:19.055 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.016+0000 7f8455e5a640 1 -- 172.21.15.167:0/3425846513 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f845006c450 msgr2=0x7f845014b8e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-11T08:06:19.055 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.016+0000 7f8455e5a640 1 --2- 172.21.15.167:0/3425846513 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f845006c450 0x7f845014b8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:19.055 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.016+0000 7f8455e5a640 1 -- 172.21.15.167:0/3425846513 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f84500d0960 msgr2=0x7f845014c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:19.055 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.016+0000 7f8455e5a640 1 --2- 172.21.15.167:0/3425846513 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f84500d0960 0x7f845014c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:19.055 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.016+0000 7f8455e5a640 1 -- 172.21.15.167:0/3425846513 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8450148dd0 con 0x7f8450151430 2024-05-11T08:06:19.056 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.016+0000 7fb78bfff640 1 -- 172.21.15.167:0/3945438221 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb78c0d0290 con 0x7fb78c0ce150 2024-05-11T08:06:19.056 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.016+0000 7fb78bfff640 1 --2- 172.21.15.167:0/3945438221 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb78c0ce150 0x7fb78c1563f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb78400c7f0 tx=0x7fb78400ccb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:19.056 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.016+0000 7f8455e5a640 1 --2- 172.21.15.167:0/3425846513 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8450151430 0x7f8450149b90 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f844c00c3f0 tx=0x7f844c00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:19.056 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.016+0000 7f62f5445640 1 Processor -- start 2024-05-11T08:06:19.057 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.017+0000 7fb78affd640 1 --2- 172.21.15.167:0/3945438221 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb78c0b4120 0x7fb78c141d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T08:06:19.057 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.017+0000 7f8445ffb640 1 -- 172.21.15.167:0/3425846513 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f844c017070 con 0x7f8450151430 2024-05-11T08:06:19.057 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.017+0000 7f8445ffb640 1 -- 172.21.15.167:0/3425846513 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f844c005ce0 con 0x7f8450151430 2024-05-11T08:06:19.057 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.017+0000 7f8445ffb640 1 -- 172.21.15.167:0/3425846513 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f844c0072d0 con 0x7f8450151430 2024-05-11T08:06:19.057 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.016+0000 7fea737fe640 1 -- 172.21.15.167:0/1590117397 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea8c0b2040 con 0x7fea84054120 2024-05-11T08:06:19.058 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.016+0000 7fea737fe640 1 -- 172.21.15.167:0/1590117397 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea8c0adcb0 con 0x7fea84054120 2024-05-11T08:06:19.058 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.017+0000 7f84578bd640 1 -- 172.21.15.167:0/3425846513 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84501490b0 con 0x7f8450151430 2024-05-11T08:06:19.058 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.017+0000 7fea737fe640 1 -- 172.21.15.167:0/1590117397 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea8c0ade60 con 0x7fea84054120 2024-05-11T08:06:19.059 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.017+0000 7fb773fff640 1 -- 172.21.15.167:0/3945438221 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb784017070 con 0x7fb78c0ce150 2024-05-11T08:06:19.059 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.017+0000 7fea912fc640 1 -- 172.21.15.167:0/1590117397 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea8416d8e0 con 0x7fea84054120 2024-05-11T08:06:19.059 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.017+0000 7fea912fc640 1 -- 172.21.15.167:0/1590117397 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea8416e390 con 0x7fea84054120 2024-05-11T08:06:19.059 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.017+0000 7f84578bd640 1 -- 172.21.15.167:0/3425846513 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f845014ce10 con 0x7f8450151430 2024-05-11T08:06:19.060 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.016+0000 7f42d5ffb640 1 -- 172.21.15.167:0/1465937264 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42d8004da0 con 0x7f42e414ab20 2024-05-11T08:06:19.060 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.016+0000 7f42d5ffb640 1 -- 172.21.15.167:0/1465937264 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42d800a640 con 0x7f42e414ab20 2024-05-11T08:06:19.060 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.016+0000 7f42d5ffb640 1 -- 172.21.15.167:0/1465937264 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42d800aa30 con 0x7f42e414ab20 2024-05-11T08:06:19.060 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.017+0000 7f42eb5af640 1 -- 172.21.15.167:0/1465937264 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42e415d740 con 0x7f42e414ab20 2024-05-11T08:06:19.061 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.018+0000 7f42eb5af640 1 -- 172.21.15.167:0/1465937264 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f42e415dd00 con 0x7f42e414ab20 2024-05-11T08:06:19.061 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.017+0000 7fc8690f8640 1 -- 172.21.15.167:0/2133436357 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc850004800 msgr2=0x7fc850004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:19.061 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.017+0000 7fc8690f8640 1 --2- 172.21.15.167:0/2133436357 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc850004800 0x7fc850004c80 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc84c009980 tx=0x7fc850005830 comp rx=0 tx=0).stop 2024-05-11T08:06:19.061 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.017+0000 7fc86359e640 1 -- 172.21.15.167:0/2133436357 reap_dead start 2024-05-11T08:06:19.062 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.018+0000 7fb773fff640 1 -- 172.21.15.167:0/3945438221 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb784003da0 con 0x7fb78c0ce150 2024-05-11T08:06:19.062 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.018+0000 7fb773fff640 1 -- 172.21.15.167:0/3945438221 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb784005290 con 0x7fb78c0ce150 2024-05-11T08:06:19.062 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.017+0000 7f62f5445640 1 -- start start 2024-05-11T08:06:19.063 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.018+0000 7fc8690f8640 1 -- 172.21.15.167:0/2133436357 shutdown_connections 2024-05-11T08:06:19.063 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.018+0000 7fc8690f8640 1 -- 172.21.15.167:0/2133436357 >> 172.21.15.167:0/2133436357 conn(0x7fc8640cd8f0 msgr2=0x7fc8640cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:19.063 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.018+0000 7fc8690f8640 1 -- 172.21.15.167:0/2133436357 shutdown_connections 2024-05-11T08:06:19.064 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.018+0000 7fb790aeb640 1 -- 172.21.15.167:0/3945438221 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb78c1554a0 con 0x7fb78c0ce150 2024-05-11T08:06:19.064 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.019+0000 7fb790aeb640 1 -- 172.21.15.167:0/3945438221 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb78c06f8f0 con 0x7fb78c0ce150 2024-05-11T08:06:19.064 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.019+0000 7fea717fa640 1 -- 172.21.15.167:0/1590117397 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea840057d0 con 0x7fea84054120 2024-05-11T08:06:19.065 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.018+0000 7fc8690f8640 1 -- 172.21.15.167:0/2133436357 wait complete. 2024-05-11T08:06:19.065 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.021+0000 7fc8690f8640 1 Processor -- start 2024-05-11T08:06:19.065 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.018+0000 7f62f5445640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f62f006d460 0x7f62f006d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:19.065 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.018+0000 7f62f5445640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f62f00b47d0 0x7f62f00b4c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:19.066 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.018+0000 7f62f5445640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f62f00cfc10 con 0x7f62f014c8a0 2024-05-11T08:06:19.066 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.018+0000 7f62f5445640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f62f006de00 con 0x7f62f00b47d0 2024-05-11T08:06:19.066 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.018+0000 7f62f5445640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f62f00cfef0 con 0x7f62f006d460 2024-05-11T08:06:19.066 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.019+0000 7f62eeffd640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f62f014c8a0 0x7f62f00cf1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:60910/0 (socket says 172.21.15.167:60910) 2024-05-11T08:06:19.066 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.019+0000 7f62eeffd640 1 -- 172.21.15.167:0/1864611829 learned_addr learned my addr 172.21.15.167:0/1864611829 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T08:06:19.066 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.020+0000 7f62ef7fe640 1 --2- 172.21.15.167:0/1864611829 >> v2:172.21.15.167:3300/0 conn(0x7f62f00b47d0 0x7f62f00b4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:19.066 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.020+0000 7f62edffb640 1 -- 172.21.15.167:0/1864611829 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2131884011 0 0) 0x7f62f00cfc10 con 0x7f62f014c8a0 2024-05-11T08:06:19.067 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.020+0000 7f42d5ffb640 1 -- 172.21.15.167:0/1465937264 <== mon.2 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f42d8011070 con 0x7f42e414ab20 2024-05-11T08:06:19.067 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.020+0000 7f42d5ffb640 1 --2- 172.21.15.167:0/1465937264 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f42c80763d0 0x7f42c8078890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:19.067 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.021+0000 7f42d5ffb640 1 -- 172.21.15.167:0/1465937264 <== mon.2 v2:172.21.15.204:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f42d80975d0 con 0x7f42e414ab20 2024-05-11T08:06:19.068 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.020+0000 7f62edffb640 1 -- 172.21.15.167:0/1864611829 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f62d8003520 con 0x7f62f014c8a0 2024-05-11T08:06:19.068 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.020+0000 7f62ef7fe640 1 -- 172.21.15.167:0/1864611829 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f62f006d460 msgr2=0x7f62f006d8c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-11T08:06:19.068 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.020+0000 7f62ef7fe640 1 --2- 172.21.15.167:0/1864611829 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f62f006d460 0x7f62f006d8c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:19.068 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.020+0000 7f62ef7fe640 1 -- 172.21.15.167:0/1864611829 >> v1:172.21.15.167:6789/0 conn(0x7f62f014c8a0 legacy=0x7f62f00cf1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:19.068 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.020+0000 7f62ef7fe640 1 -- 172.21.15.167:0/1864611829 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62f00d01d0 con 0x7f62f00b47d0 2024-05-11T08:06:19.068 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.021+0000 7f84578bd640 1 -- 172.21.15.167:0/3425846513 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8450069bc0 con 0x7f8450151430 2024-05-11T08:06:19.069 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.021+0000 7fb773fff640 1 -- 172.21.15.167:0/3945438221 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fb78401a070 con 0x7fb78c0ce150 2024-05-11T08:06:19.069 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.022+0000 7fb773fff640 1 --2- 172.21.15.167:0/3945438221 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fb7780763a0 0x7fb778078860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:19.069 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.021+0000 7fea737fe640 1 -- 172.21.15.167:0/1590117397 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fea8c0b8670 con 0x7fea84054120 2024-05-11T08:06:19.069 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.022+0000 7fea737fe640 1 --2- 172.21.15.167:0/1590117397 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fea780763a0 0x7fea78078860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:19.070 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.022+0000 7fb773fff640 1 -- 172.21.15.167:0/3945438221 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fb784096fd0 con 0x7fb78c0ce150 2024-05-11T08:06:19.070 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.022+0000 7f8445ffb640 1 -- 172.21.15.167:0/3425846513 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f844c01a070 con 0x7f8450151430 2024-05-11T08:06:19.070 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.022+0000 7fb78b7fe640 1 --2- 172.21.15.167:0/3945438221 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fb7780763a0 0x7fb778078860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:19.071 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.022+0000 7fea737fe640 1 -- 172.21.15.167:0/1590117397 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fea8c134c50 con 0x7fea84054120 2024-05-11T08:06:19.071 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.022+0000 7f42e934b640 1 --2- 172.21.15.167:0/1465937264 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f42c80763d0 0x7f42c8078890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:19.071 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.022+0000 7fb790aeb640 1 -- 172.21.15.167:0/3945438221 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb78c0694b0 con 0x7fb78c0ce150 2024-05-11T08:06:19.072 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.021+0000 7fc8690f8640 1 -- start start 2024-05-11T08:06:19.072 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.023+0000 7fea8affd640 1 --2- 172.21.15.167:0/1590117397 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fea780763a0 0x7fea78078860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:19.073 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.023+0000 7f42e934b640 1 --2- 172.21.15.167:0/1465937264 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f42c80763d0 0x7f42c8078890 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f42e0006fd0 tx=0x7f42e0008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:19.073 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.023+0000 7fb78b7fe640 1 --2- 172.21.15.167:0/3945438221 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fb7780763a0 0x7fb778078860 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb77c005880 tx=0x7fb77c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:19.073 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.022+0000 7f62ef7fe640 1 --2- 172.21.15.167:0/1864611829 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f62f00b47d0 0x7f62f00b4c40 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f62e80097a0 tx=0x7f62e802a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99f8c63dcf5d7167 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:19.074 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.022+0000 7fc8690f8640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc86415ac40 0x7fc8641f1880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:19.074 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.022+0000 7fc8690f8640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc86414dd00 0x7fc8641f4460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:19.074 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.022+0000 7fc8690f8640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc86415b020 0x7fc8640b1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:19.074 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.022+0000 7fc8690f8640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc86406dcf0 con 0x7fc86415ac40 2024-05-11T08:06:19.074 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.022+0000 7fc8690f8640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc86406db70 con 0x7fc86414dd00 2024-05-11T08:06:19.074 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.022+0000 7fc8690f8640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fc8640ce3e0 con 0x7fc86415b020 2024-05-11T08:06:19.074 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.022+0000 7fc86359e640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc86415b020 0x7fc8640b1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:19.074 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.023+0000 7fc86359e640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc86415b020 0x7fc8640b1ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.167:37434/0 (socket says 172.21.15.167:37434) 2024-05-11T08:06:19.074 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.023+0000 7fc86359e640 1 -- 172.21.15.167:0/767789219 learned_addr learned my addr 172.21.15.167:0/767789219 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:19.074 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.023+0000 7fc862d9d640 1 --2- 172.21.15.167:0/767789219 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc86415ac40 0x7fc8641f1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:19.074 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.023+0000 7fc862d9d640 1 -- 172.21.15.167:0/767789219 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc86415b020 msgr2=0x7fc8640b1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:19.075 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.023+0000 7fc862d9d640 1 --2- 172.21.15.167:0/767789219 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc86415b020 0x7fc8640b1ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:19.075 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.023+0000 7fc862d9d640 1 -- 172.21.15.167:0/767789219 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc86414dd00 msgr2=0x7fc8641f4460 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-11T08:06:19.075 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.023+0000 7fc862d9d640 1 --2- 172.21.15.167:0/767789219 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc86414dd00 0x7fc8641f4460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:19.075 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.023+0000 7fc862d9d640 1 -- 172.21.15.167:0/767789219 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8641f1dc0 con 0x7fc86415ac40 2024-05-11T08:06:19.075 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.023+0000 7fc862d9d640 1 --2- 172.21.15.167:0/767789219 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc86415ac40 0x7fc8641f1880 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fc85400ca70 tx=0x7fc85400cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:19.075 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.023+0000 7f42eb5af640 1 -- 172.21.15.167:0/1465937264 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42e4069bc0 con 0x7f42e414ab20 2024-05-11T08:06:19.076 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.023+0000 7f8445ffb640 1 --2- 172.21.15.167:0/3425846513 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f84340763a0 0x7f8434078860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:19.076 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.023+0000 7f8445ffb640 1 -- 172.21.15.167:0/3425846513 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f844c096bf0 con 0x7f8450151430 2024-05-11T08:06:19.076 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.023+0000 7f8455659640 1 --2- 172.21.15.167:0/3425846513 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f84340763a0 0x7f8434078860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:19.076 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.024+0000 7fea8affd640 1 --2- 172.21.15.167:0/1590117397 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fea780763a0 0x7fea78078860 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fea7c009870 tx=0x7fea7c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:19.077 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.025+0000 7f8455659640 1 --2- 172.21.15.167:0/3425846513 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f84340763a0 0x7f8434078860 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f8448001be0 tx=0x7f8448007cf0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:19.077 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.026+0000 7f62edffb640 1 -- 172.21.15.167:0/1864611829 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62e8037070 con 0x7f62f00b47d0 2024-05-11T08:06:19.078 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.026+0000 7fc85b7fe640 1 -- 172.21.15.167:0/767789219 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc854018070 con 0x7fc86415ac40 2024-05-11T08:06:19.078 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.026+0000 7fc8690f8640 1 -- 172.21.15.167:0/767789219 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8641f4dd0 con 0x7fc86415ac40 2024-05-11T08:06:19.078 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.026+0000 7fc8690f8640 1 -- 172.21.15.167:0/767789219 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8641f1080 con 0x7fc86415ac40 2024-05-11T08:06:19.078 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.026+0000 7fea737fe640 1 -- 172.21.15.167:0/1590117397 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fea8c0fe250 con 0x7fea84054120 2024-05-11T08:06:19.079 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.027+0000 7fc8690f8640 1 -- 172.21.15.167:0/767789219 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8641f0e70 con 0x7fc86415ac40 2024-05-11T08:06:19.079 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.028+0000 7fb773fff640 1 -- 172.21.15.167:0/3945438221 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fb784060010 con 0x7fb78c0ce150 2024-05-11T08:06:19.079 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.028+0000 7f42d5ffb640 1 -- 172.21.15.167:0/1465937264 <== mon.2 v2:172.21.15.204:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f42d8060bd0 con 0x7f42e414ab20 2024-05-11T08:06:19.080 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.031+0000 7fc85b7fe640 1 -- 172.21.15.167:0/767789219 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc854004cb0 con 0x7fc86415ac40 2024-05-11T08:06:19.080 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.031+0000 7fc85b7fe640 1 -- 172.21.15.167:0/767789219 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc854022060 con 0x7fc86415ac40 2024-05-11T08:06:19.080 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.031+0000 7f8445ffb640 1 -- 172.21.15.167:0/3425846513 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f844c05fba0 con 0x7f8450151430 2024-05-11T08:06:19.080 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.031+0000 7f62edffb640 1 -- 172.21.15.167:0/1864611829 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62e8032440 con 0x7f62f00b47d0 2024-05-11T08:06:19.081 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.031+0000 7f62edffb640 1 -- 172.21.15.167:0/1864611829 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62e80327f0 con 0x7f62f00b47d0 2024-05-11T08:06:19.081 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.033+0000 7f62f5445640 1 -- 172.21.15.167:0/1864611829 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f62f00b47d0 msgr2=0x7f62f00b4c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:19.081 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.033+0000 7f62f5445640 1 --2- 172.21.15.167:0/1864611829 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f62f00b47d0 0x7f62f00b4c40 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f62e80097a0 tx=0x7f62e802a1c0 comp rx=0 tx=0).stop 2024-05-11T08:06:19.081 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.033+0000 7f62f5445640 1 -- 172.21.15.167:0/1864611829 shutdown_connections 2024-05-11T08:06:19.081 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.033+0000 7f62f5445640 1 --2- 172.21.15.167:0/1864611829 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f62f00b47d0 0x7f62f00b4c40 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:19.081 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.033+0000 7f62f5445640 1 --2- 172.21.15.167:0/1864611829 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f62f006d460 0x7f62f006d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:19.081 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.033+0000 7f62f5445640 1 -- 172.21.15.167:0/1864611829 >> 172.21.15.167:0/1864611829 conn(0x7f62f00cd8f0 msgr2=0x7f62f00cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:19.081 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.035+0000 7f62f5445640 1 -- 172.21.15.167:0/1864611829 shutdown_connections 2024-05-11T08:06:19.082 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.034+0000 7fc85b7fe640 1 -- 172.21.15.167:0/767789219 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fc85401b070 con 0x7fc86415ac40 2024-05-11T08:06:19.082 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.034+0000 7fc85b7fe640 1 --2- 172.21.15.167:0/767789219 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fc8440762d0 0x7fc844078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:19.082 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.035+0000 7fc85b7fe640 1 -- 172.21.15.167:0/767789219 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fc8540970f0 con 0x7fc86415ac40 2024-05-11T08:06:19.082 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.037+0000 7fc85bfff640 1 --2- 172.21.15.167:0/767789219 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fc8440762d0 0x7fc844078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:19.082 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.038+0000 7fc85bfff640 1 --2- 172.21.15.167:0/767789219 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fc8440762d0 0x7fc844078790 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc84c033000 tx=0x7fc84c008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:19.083 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.035+0000 7f62f5445640 1 -- 172.21.15.167:0/1864611829 wait complete. 2024-05-11T08:06:19.084 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.037+0000 7f62f5445640 1 Processor -- start 2024-05-11T08:06:19.084 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.038+0000 7f62f5445640 1 -- start start 2024-05-11T08:06:19.084 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.040+0000 7f62f5445640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f62f006d460 0x7f62f00b0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:19.084 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.040+0000 7f62f5445640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f62f00b47d0 0x7f62f013e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:19.084 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.040+0000 7f62f5445640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f62f014c8a0 0x7f62f013ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:19.084 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.040+0000 7f62f5445640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f62f00cf8d0 con 0x7f62f00b47d0 2024-05-11T08:06:19.084 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.040+0000 7f62f5445640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f62f00b3fa0 con 0x7f62f014c8a0 2024-05-11T08:06:19.084 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.040+0000 7f62f5445640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f62f006fd30 con 0x7f62f006d460 2024-05-11T08:06:19.084 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.041+0000 7f62ee7fc640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f62f00b47d0 0x7f62f013e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:19.084 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.041+0000 7f62ee7fc640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f62f00b47d0 0x7f62f013e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:54542/0 (socket says 172.21.15.167:54542) 2024-05-11T08:06:19.084 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.041+0000 7f62ef7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f62f014c8a0 0x7f62f013ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:19.085 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.041+0000 7fc85b7fe640 1 -- 172.21.15.167:0/767789219 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fc8641f1dc0 con 0x7fc86415ac40 2024-05-11T08:06:19.087 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.041+0000 7f62ef7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f62f014c8a0 0x7f62f013ea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.167:37396/0 (socket says 172.21.15.167:37396) 2024-05-11T08:06:19.087 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.041+0000 7f62ef7fe640 1 -- 172.21.15.167:0/684072091 learned_addr learned my addr 172.21.15.167:0/684072091 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:19.087 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.042+0000 7f62ef7fe640 1 -- 172.21.15.167:0/684072091 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f62f006d460 msgr2=0x7f62f00b0ed0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-11T08:06:19.087 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.042+0000 7f62ef7fe640 1 --2- 172.21.15.167:0/684072091 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f62f006d460 0x7f62f00b0ed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:19.087 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.042+0000 7f62ef7fe640 1 -- 172.21.15.167:0/684072091 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f62f00b47d0 msgr2=0x7f62f013e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:19.087 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.042+0000 7f62ef7fe640 1 --2- 172.21.15.167:0/684072091 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f62f00b47d0 0x7f62f013e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:19.087 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.042+0000 7f62ef7fe640 1 -- 172.21.15.167:0/684072091 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62f0161400 con 0x7f62f014c8a0 2024-05-11T08:06:19.087 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.042+0000 7f62ee7fc640 1 --2- 172.21.15.167:0/684072091 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f62f00b47d0 0x7f62f013e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T08:06:19.088 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.042+0000 7f62ef7fe640 1 --2- 172.21.15.167:0/684072091 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f62f014c8a0 0x7f62f013ea80 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f62e8035040 tx=0x7f62e80046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:19.088 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.043+0000 7f62d77fe640 1 -- 172.21.15.167:0/684072091 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62e8037050 con 0x7f62f014c8a0 2024-05-11T08:06:19.088 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.043+0000 7f62f5445640 1 -- 172.21.15.167:0/684072091 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62f00ac2e0 con 0x7f62f014c8a0 2024-05-11T08:06:19.088 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.043+0000 7f62f5445640 1 -- 172.21.15.167:0/684072091 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62f0160c00 con 0x7f62f014c8a0 2024-05-11T08:06:19.088 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.043+0000 7f62d77fe640 1 -- 172.21.15.167:0/684072091 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62e8043070 con 0x7f62f014c8a0 2024-05-11T08:06:19.088 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.043+0000 7f62d77fe640 1 -- 172.21.15.167:0/684072091 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62e80044e0 con 0x7f62f014c8a0 2024-05-11T08:06:19.088 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.046+0000 7f62d77fe640 1 -- 172.21.15.167:0/684072091 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f62e802ad10 con 0x7f62f014c8a0 2024-05-11T08:06:19.088 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.047+0000 7f62d77fe640 1 --2- 172.21.15.167:0/684072091 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f62dc076350 0x7f62dc078810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:19.088 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.048+0000 7f62eeffd640 1 --2- 172.21.15.167:0/684072091 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f62dc076350 0x7f62dc078810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:19.088 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.048+0000 7f62d77fe640 1 -- 172.21.15.167:0/684072091 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f62e807d710 con 0x7f62f014c8a0 2024-05-11T08:06:19.088 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.049+0000 7f62f5445640 1 -- 172.21.15.167:0/684072091 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62f0069590 con 0x7f62f014c8a0 2024-05-11T08:06:19.088 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.050+0000 7f62eeffd640 1 --2- 172.21.15.167:0/684072091 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f62dc076350 0x7f62dc078810 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f62e40014f0 tx=0x7f62e4007c90 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:19.088 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.054+0000 7f62d77fe640 1 -- 172.21.15.167:0/684072091 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f62e8081820 con 0x7f62f014c8a0 2024-05-11T08:06:19.389 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.387+0000 7f0ea5a72640 1 -- 172.21.15.167:0/3743198298 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f0e80005580 con 0x7f0e98055d00 2024-05-11T08:06:19.389 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.388+0000 7f0e8bfff640 1 -- 172.21.15.167:0/3743198298 <== mon.2 v2:172.21.15.204:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f0e94081860 con 0x7f0e98055d00 2024-05-11T08:06:19.389 INFO:teuthology.orchestra.run.smithi167.stdout:64424509450 2024-05-11T08:06:19.393 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.391+0000 7f0e89ffb640 1 -- 172.21.15.167:0/3743198298 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f0e900764e0 msgr2=0x7f0e900789a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:19.393 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.392+0000 7f0e89ffb640 1 --2- 172.21.15.167:0/3743198298 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f0e900764e0 0x7f0e900789a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0e8c006fd0 tx=0x7f0e8c008040 comp rx=0 tx=0).stop 2024-05-11T08:06:19.393 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.392+0000 7f0e89ffb640 1 -- 172.21.15.167:0/3743198298 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0e98055d00 msgr2=0x7f0e9804b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:19.393 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.392+0000 7f0e89ffb640 1 --2- 172.21.15.167:0/3743198298 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0e98055d00 0x7f0e9804b890 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f0e94009870 tx=0x7f0e940046c0 comp rx=0 tx=0).stop 2024-05-11T08:06:19.393 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.392+0000 7f0e89ffb640 1 -- 172.21.15.167:0/3743198298 shutdown_connections 2024-05-11T08:06:19.394 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.392+0000 7f0e89ffb640 1 --2- 172.21.15.167:0/3743198298 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f0e900764e0 0x7f0e900789a0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:19.394 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.392+0000 7f0e89ffb640 1 --2- 172.21.15.167:0/3743198298 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0e980df7f0 0x7f0e9804d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:19.394 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.392+0000 7f0e89ffb640 1 --2- 172.21.15.167:0/3743198298 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0e98055d00 0x7f0e9804b890 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:19.394 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.392+0000 7f0e89ffb640 1 --2- 172.21.15.167:0/3743198298 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0e98006210 0x7f0e98185b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:19.394 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.392+0000 7f0e89ffb640 1 -- 172.21.15.167:0/3743198298 >> 172.21.15.167:0/3743198298 conn(0x7f0e9801e6b0 msgr2=0x7f0e9800b060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:19.395 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.393+0000 7f0e89ffb640 1 -- 172.21.15.167:0/3743198298 shutdown_connections 2024-05-11T08:06:19.395 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.393+0000 7f0e89ffb640 1 -- 172.21.15.167:0/3743198298 wait complete. 2024-05-11T08:06:19.396 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.395+0000 7f4f74ff9640 1 -- 172.21.15.167:0/3317991600 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f4f60005580 con 0x7f4f70004800 2024-05-11T08:06:19.401 INFO:teuthology.orchestra.run.smithi167.stdout:64424509450 2024-05-11T08:06:19.401 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.396+0000 7f4f76ffd640 1 -- 172.21.15.167:0/3317991600 <== mon.2 v2:172.21.15.204:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f4f7805ff30 con 0x7f4f70004800 2024-05-11T08:06:19.402 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.400+0000 7f4f8cb38640 1 -- 172.21.15.167:0/3317991600 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f4f68076320 msgr2=0x7f4f680787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:19.402 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.400+0000 7f4f8cb38640 1 --2- 172.21.15.167:0/3317991600 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f4f68076320 0x7f4f680787e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f4f80035000 tx=0x7f4f8000b030 comp rx=0 tx=0).stop 2024-05-11T08:06:19.403 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.400+0000 7f4f8cb38640 1 -- 172.21.15.167:0/3317991600 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4f70004800 msgr2=0x7f4f8815cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:19.403 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.400+0000 7f4f8cb38640 1 --2- 172.21.15.167:0/3317991600 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4f70004800 0x7f4f8815cf60 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f4f7800c7a0 tx=0x7f4f7800cc60 comp rx=0 tx=0).stop 2024-05-11T08:06:19.403 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.400+0000 7f4f8cb38640 1 -- 172.21.15.167:0/3317991600 shutdown_connections 2024-05-11T08:06:19.403 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.400+0000 7f4f8cb38640 1 --2- 172.21.15.167:0/3317991600 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f4f68076320 0x7f4f680787e0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:19.403 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.400+0000 7f4f8cb38640 1 --2- 172.21.15.167:0/3317991600 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4f88160a00 0x7f4f88160e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:19.403 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.401+0000 7f4f8cb38640 1 --2- 172.21.15.167:0/3317991600 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4f8815f060 0x7f4f88162c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:19.403 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.401+0000 7f4f8cb38640 1 --2- 172.21.15.167:0/3317991600 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4f70004800 0x7f4f8815cf60 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:19.403 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.401+0000 7f4f8cb38640 1 -- 172.21.15.167:0/3317991600 >> 172.21.15.167:0/3317991600 conn(0x7f4f880cd8f0 msgr2=0x7f4f880b2130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:19.403 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.401+0000 7f4f8cb38640 1 -- 172.21.15.167:0/3317991600 shutdown_connections 2024-05-11T08:06:19.403 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.401+0000 7f4f8cb38640 1 -- 172.21.15.167:0/3317991600 wait complete. 2024-05-11T08:06:19.412 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509450 got 64424509450 for osd.7 2024-05-11T08:06:19.412 DEBUG:teuthology.parallel:result is None 2024-05-11T08:06:19.419 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509450 got 64424509450 for osd.5 2024-05-11T08:06:19.419 DEBUG:teuthology.parallel:result is None 2024-05-11T08:06:19.430 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.429+0000 7fea717fa640 1 -- 172.21.15.167:0/1590117397 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fea8400df30 con 0x7fea84054120 2024-05-11T08:06:19.432 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.430+0000 7fea737fe640 1 -- 172.21.15.167:0/1590117397 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fea8c0fdb50 con 0x7fea84054120 2024-05-11T08:06:19.432 INFO:teuthology.orchestra.run.smithi167.stdout:64424509451 2024-05-11T08:06:19.434 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.433+0000 7fea717fa640 1 -- 172.21.15.167:0/1590117397 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fea780763a0 msgr2=0x7fea78078860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:19.434 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.433+0000 7fea717fa640 1 --2- 172.21.15.167:0/1590117397 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fea780763a0 0x7fea78078860 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fea7c009870 tx=0x7fea7c008040 comp rx=0 tx=0).stop 2024-05-11T08:06:19.435 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.434+0000 7fea717fa640 1 -- 172.21.15.167:0/1590117397 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea84054120 msgr2=0x7fea840dbd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:19.436 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.434+0000 7fea717fa640 1 --2- 172.21.15.167:0/1590117397 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea84054120 0x7fea840dbd20 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fea8c051d40 tx=0x7fea8c06b820 comp rx=0 tx=0).stop 2024-05-11T08:06:19.436 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.434+0000 7fea717fa640 1 -- 172.21.15.167:0/1590117397 shutdown_connections 2024-05-11T08:06:19.436 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.434+0000 7fea717fa640 1 --2- 172.21.15.167:0/1590117397 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fea780763a0 0x7fea78078860 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:19.436 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.434+0000 7fea717fa640 1 --2- 172.21.15.167:0/1590117397 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fea840e30b0 0x7fea840dc260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:19.436 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.434+0000 7fea717fa640 1 --2- 172.21.15.167:0/1590117397 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea84054120 0x7fea840dbd20 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:19.436 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.434+0000 7fea717fa640 1 --2- 172.21.15.167:0/1590117397 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fea8403e830 0x7fea8416eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:19.436 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.434+0000 7fea717fa640 1 -- 172.21.15.167:0/1590117397 >> 172.21.15.167:0/1590117397 conn(0x7fea840559d0 msgr2=0x7fea8403a1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:19.436 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.434+0000 7fea717fa640 1 -- 172.21.15.167:0/1590117397 shutdown_connections 2024-05-11T08:06:19.436 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.434+0000 7fea717fa640 1 -- 172.21.15.167:0/1590117397 wait complete. 2024-05-11T08:06:19.446 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509451 got 64424509451 for osd.3 2024-05-11T08:06:19.446 DEBUG:teuthology.parallel:result is None 2024-05-11T08:06:19.452 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.451+0000 7f62f5445640 1 -- 172.21.15.167:0/684072091 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f62f0069c60 con 0x7f62f014c8a0 2024-05-11T08:06:19.453 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.452+0000 7f62d77fe640 1 -- 172.21.15.167:0/684072091 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f62e8081820 con 0x7f62f014c8a0 2024-05-11T08:06:19.453 INFO:teuthology.orchestra.run.smithi167.stdout:64424509450 2024-05-11T08:06:19.455 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.454+0000 7f62f5445640 1 -- 172.21.15.167:0/684072091 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f62dc076350 msgr2=0x7f62dc078810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:19.455 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.454+0000 7f62f5445640 1 --2- 172.21.15.167:0/684072091 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f62dc076350 0x7f62dc078810 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f62e40014f0 tx=0x7f62e4007c90 comp rx=0 tx=0).stop 2024-05-11T08:06:19.455 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.454+0000 7f62f5445640 1 -- 172.21.15.167:0/684072091 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f62f014c8a0 msgr2=0x7f62f013ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:19.455 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.454+0000 7f62f5445640 1 --2- 172.21.15.167:0/684072091 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f62f014c8a0 0x7f62f013ea80 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f62e8035040 tx=0x7f62e80046c0 comp rx=0 tx=0).stop 2024-05-11T08:06:19.455 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.454+0000 7f62f5445640 1 -- 172.21.15.167:0/684072091 shutdown_connections 2024-05-11T08:06:19.455 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.454+0000 7f62f5445640 1 --2- 172.21.15.167:0/684072091 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f62dc076350 0x7f62dc078810 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:19.455 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.454+0000 7f62f5445640 1 --2- 172.21.15.167:0/684072091 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f62f014c8a0 0x7f62f013ea80 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:19.455 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.454+0000 7f62f5445640 1 --2- 172.21.15.167:0/684072091 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f62f00b47d0 0x7f62f013e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:19.455 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.454+0000 7f62f5445640 1 --2- 172.21.15.167:0/684072091 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f62f006d460 0x7f62f00b0ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:19.456 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.454+0000 7f62f5445640 1 -- 172.21.15.167:0/684072091 >> 172.21.15.167:0/684072091 conn(0x7f62f00cd8f0 msgr2=0x7f62f006cc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:19.457 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.454+0000 7f62f5445640 1 -- 172.21.15.167:0/684072091 shutdown_connections 2024-05-11T08:06:19.457 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.454+0000 7f62f5445640 1 -- 172.21.15.167:0/684072091 wait complete. 2024-05-11T08:06:19.466 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509450 got 64424509450 for osd.0 2024-05-11T08:06:19.466 DEBUG:teuthology.parallel:result is None 2024-05-11T08:06:19.468 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.467+0000 7f84578bd640 1 -- 172.21.15.167:0/3425846513 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f84500b4d60 con 0x7f8450151430 2024-05-11T08:06:19.468 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.467+0000 7f8445ffb640 1 -- 172.21.15.167:0/3425846513 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f844c063ea0 con 0x7f8450151430 2024-05-11T08:06:19.468 INFO:teuthology.orchestra.run.smithi167.stdout:55834574859 2024-05-11T08:06:19.472 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.470+0000 7f84578bd640 1 -- 172.21.15.167:0/3425846513 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f84340763a0 msgr2=0x7f8434078860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:19.472 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.470+0000 7f84578bd640 1 --2- 172.21.15.167:0/3425846513 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f84340763a0 0x7f8434078860 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f8448001be0 tx=0x7f8448007cf0 comp rx=0 tx=0).stop 2024-05-11T08:06:19.472 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.470+0000 7f84578bd640 1 -- 172.21.15.167:0/3425846513 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8450151430 msgr2=0x7f8450149b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:19.472 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.470+0000 7f84578bd640 1 --2- 172.21.15.167:0/3425846513 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8450151430 0x7f8450149b90 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f844c00c3f0 tx=0x7f844c00c8b0 comp rx=0 tx=0).stop 2024-05-11T08:06:19.472 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.470+0000 7f84578bd640 1 -- 172.21.15.167:0/3425846513 shutdown_connections 2024-05-11T08:06:19.472 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.470+0000 7f84578bd640 1 --2- 172.21.15.167:0/3425846513 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f84340763a0 0x7f8434078860 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:19.472 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.471+0000 7f84578bd640 1 --2- 172.21.15.167:0/3425846513 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8450151430 0x7f8450149b90 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:19.472 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.471+0000 7f84578bd640 1 --2- 172.21.15.167:0/3425846513 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f84500d0960 0x7f845014c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:19.472 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.471+0000 7f84578bd640 1 --2- 172.21.15.167:0/3425846513 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f845006c450 0x7f845014b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:19.473 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.471+0000 7f84578bd640 1 -- 172.21.15.167:0/3425846513 >> 172.21.15.167:0/3425846513 conn(0x7f84500710d0 msgr2=0x7f845015d0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:19.473 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.471+0000 7f84578bd640 1 -- 172.21.15.167:0/3425846513 shutdown_connections 2024-05-11T08:06:19.473 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.471+0000 7f84578bd640 1 -- 172.21.15.167:0/3425846513 wait complete. 2024-05-11T08:06:19.474 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.474+0000 7f42eb5af640 1 -- 172.21.15.167:0/1465937264 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f42e4166750 con 0x7f42e414ab20 2024-05-11T08:06:19.474 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.474+0000 7fb790aeb640 1 -- 172.21.15.167:0/3945438221 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fb78c06f0f0 con 0x7fb78c0ce150 2024-05-11T08:06:19.475 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.475+0000 7f42d5ffb640 1 -- 172.21.15.167:0/1465937264 <== mon.2 v2:172.21.15.204:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f42d80604d0 con 0x7f42e414ab20 2024-05-11T08:06:19.475 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.475+0000 7fb773fff640 1 -- 172.21.15.167:0/3945438221 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fb784064310 con 0x7fb78c0ce150 2024-05-11T08:06:19.475 INFO:teuthology.orchestra.run.smithi167.stdout:64424509450 2024-05-11T08:06:19.476 INFO:teuthology.orchestra.run.smithi167.stdout:64424509451 2024-05-11T08:06:19.478 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.478+0000 7fb790aeb640 1 -- 172.21.15.167:0/3945438221 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fb7780763a0 msgr2=0x7fb778078860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:19.478 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.478+0000 7fb790aeb640 1 --2- 172.21.15.167:0/3945438221 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fb7780763a0 0x7fb778078860 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb77c005880 tx=0x7fb77c009040 comp rx=0 tx=0).stop 2024-05-11T08:06:19.478 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.478+0000 7fb790aeb640 1 -- 172.21.15.167:0/3945438221 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb78c0ce150 msgr2=0x7fb78c1563f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:19.479 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.478+0000 7fb790aeb640 1 --2- 172.21.15.167:0/3945438221 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb78c0ce150 0x7fb78c1563f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb78400c7f0 tx=0x7fb78400ccb0 comp rx=0 tx=0).stop 2024-05-11T08:06:19.479 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.478+0000 7f42eb5af640 1 -- 172.21.15.167:0/1465937264 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f42c80763d0 msgr2=0x7f42c8078890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:19.479 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.478+0000 7f42eb5af640 1 --2- 172.21.15.167:0/1465937264 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f42c80763d0 0x7f42c8078890 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f42e0006fd0 tx=0x7f42e0008040 comp rx=0 tx=0).stop 2024-05-11T08:06:19.479 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.478+0000 7f42eb5af640 1 -- 172.21.15.167:0/1465937264 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f42e414ab20 msgr2=0x7f42e4160310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:19.479 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.478+0000 7f42eb5af640 1 --2- 172.21.15.167:0/1465937264 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f42e414ab20 0x7f42e4160310 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f42d800c6e0 tx=0x7f42d800cba0 comp rx=0 tx=0).stop 2024-05-11T08:06:19.480 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.478+0000 7fb790aeb640 1 -- 172.21.15.167:0/3945438221 shutdown_connections 2024-05-11T08:06:19.480 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.478+0000 7fb790aeb640 1 --2- 172.21.15.167:0/3945438221 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fb7780763a0 0x7fb778078860 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:19.480 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.478+0000 7fb790aeb640 1 --2- 172.21.15.167:0/3945438221 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb78c0ce150 0x7fb78c1563f0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:19.480 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.478+0000 7fb790aeb640 1 --2- 172.21.15.167:0/3945438221 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb78c0b4120 0x7fb78c141d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:19.480 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.478+0000 7fb790aeb640 1 --2- 172.21.15.167:0/3945438221 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb78c073050 0x7fb78c158f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:19.480 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.478+0000 7fb790aeb640 1 -- 172.21.15.167:0/3945438221 >> 172.21.15.167:0/3945438221 conn(0x7fb78c0cd630 msgr2=0x7fb78c15c500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:19.480 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.479+0000 7f42eb5af640 1 -- 172.21.15.167:0/1465937264 shutdown_connections 2024-05-11T08:06:19.480 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.479+0000 7f42eb5af640 1 --2- 172.21.15.167:0/1465937264 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f42c80763d0 0x7f42c8078890 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:19.481 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.479+0000 7f42eb5af640 1 --2- 172.21.15.167:0/1465937264 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f42e414b300 0x7f42e414b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:19.481 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.479+0000 7f42eb5af640 1 --2- 172.21.15.167:0/1465937264 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f42e414ab20 0x7f42e4160310 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:19.481 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.479+0000 7f42eb5af640 1 --2- 172.21.15.167:0/1465937264 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f42d0005210 0x7f42e41edbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:19.481 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.479+0000 7f42eb5af640 1 -- 172.21.15.167:0/1465937264 >> 172.21.15.167:0/1465937264 conn(0x7f42e40710a0 msgr2=0x7f42e40728e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:19.481 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.479+0000 7fb790aeb640 1 -- 172.21.15.167:0/3945438221 shutdown_connections 2024-05-11T08:06:19.481 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.479+0000 7fb790aeb640 1 -- 172.21.15.167:0/3945438221 wait complete. 2024-05-11T08:06:19.482 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.479+0000 7f42eb5af640 1 -- 172.21.15.167:0/1465937264 shutdown_connections 2024-05-11T08:06:19.482 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.479+0000 7f42eb5af640 1 -- 172.21.15.167:0/1465937264 wait complete. 2024-05-11T08:06:19.485 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.485+0000 7fc8690f8640 1 -- 172.21.15.167:0/767789219 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fc864069590 con 0x7fc86415ac40 2024-05-11T08:06:19.485 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.485+0000 7fc85b7fe640 1 -- 172.21.15.167:0/767789219 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc854060130 con 0x7fc86415ac40 2024-05-11T08:06:19.485 INFO:teuthology.orchestra.run.smithi167.stdout:64424509450 2024-05-11T08:06:19.491 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.488+0000 7fc8690f8640 1 -- 172.21.15.167:0/767789219 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fc8440762d0 msgr2=0x7fc844078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:19.491 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.488+0000 7fc8690f8640 1 --2- 172.21.15.167:0/767789219 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fc8440762d0 0x7fc844078790 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc84c033000 tx=0x7fc84c008030 comp rx=0 tx=0).stop 2024-05-11T08:06:19.491 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.488+0000 7fc8690f8640 1 -- 172.21.15.167:0/767789219 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc86415ac40 msgr2=0x7fc8641f1880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:19.491 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.488+0000 7fc8690f8640 1 --2- 172.21.15.167:0/767789219 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc86415ac40 0x7fc8641f1880 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fc85400ca70 tx=0x7fc85400cf30 comp rx=0 tx=0).stop 2024-05-11T08:06:19.491 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.488+0000 7fc8690f8640 1 -- 172.21.15.167:0/767789219 shutdown_connections 2024-05-11T08:06:19.491 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.488+0000 7fc8690f8640 1 --2- 172.21.15.167:0/767789219 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fc8440762d0 0x7fc844078790 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:19.491 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.488+0000 7fc8690f8640 1 --2- 172.21.15.167:0/767789219 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc86415b020 0x7fc8640b1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:19.491 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.488+0000 7fc8690f8640 1 --2- 172.21.15.167:0/767789219 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc86414dd00 0x7fc8641f4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:19.491 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.488+0000 7fc8690f8640 1 --2- 172.21.15.167:0/767789219 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc86415ac40 0x7fc8641f1880 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:19.491 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.488+0000 7fc8690f8640 1 -- 172.21.15.167:0/767789219 >> 172.21.15.167:0/767789219 conn(0x7fc8640cd8f0 msgr2=0x7fc86406d220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:19.491 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.488+0000 7fc8690f8640 1 -- 172.21.15.167:0/767789219 shutdown_connections 2024-05-11T08:06:19.492 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.488+0000 7fc8690f8640 1 -- 172.21.15.167:0/767789219 wait complete. 2024-05-11T08:06:19.492 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.2 2024-05-11T08:06:19.493 DEBUG:teuthology.parallel:result is None 2024-05-11T08:06:19.497 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509450 got 64424509450 for osd.6 2024-05-11T08:06:19.497 DEBUG:teuthology.parallel:result is None 2024-05-11T08:06:19.498 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509451 got 64424509451 for osd.1 2024-05-11T08:06:19.498 DEBUG:teuthology.parallel:result is None 2024-05-11T08:06:19.499 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509450 got 64424509450 for osd.4 2024-05-11T08:06:19.499 DEBUG:teuthology.parallel:result is None 2024-05-11T08:06:19.499 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-11T08:06:19.500 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-11T08:06:19.643 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.643+0000 7f589be09640 1 Processor -- start 2024-05-11T08:06:19.643 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.643+0000 7f589be09640 1 -- start start 2024-05-11T08:06:19.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.644+0000 7f589be09640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f589413eeb0 0x7f589406f480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:19.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.644+0000 7f589be09640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f589413f290 0x7f589406f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:19.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.644+0000 7f589be09640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f589415fb90 con 0x7f58940b50a0 2024-05-11T08:06:19.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.644+0000 7f589be09640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f589415d5b0 con 0x7f589413eeb0 2024-05-11T08:06:19.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.644+0000 7f589be09640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f589415fe70 con 0x7f589413f290 2024-05-11T08:06:19.646 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.644+0000 7f5899ba5640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f589413eeb0 0x7f589406f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:19.646 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.644+0000 7f5899ba5640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f589413eeb0 0x7f589406f480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:54548/0 (socket says 172.21.15.167:54548) 2024-05-11T08:06:19.646 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.644+0000 7f5899ba5640 1 -- 172.21.15.167:0/693803297 learned_addr learned my addr 172.21.15.167:0/693803297 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:19.646 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.644+0000 7f58993a4640 1 --2- 172.21.15.167:0/693803297 >> v2:172.21.15.204:3300/0 conn(0x7f589413f290 0x7f589406f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:19.646 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.644+0000 7f5899ba5640 1 -- 172.21.15.167:0/693803297 >> v2:172.21.15.204:3300/0 conn(0x7f589413f290 msgr2=0x7f589406f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:19.646 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.644+0000 7f5899ba5640 1 --2- 172.21.15.167:0/693803297 >> v2:172.21.15.204:3300/0 conn(0x7f589413f290 0x7f589406f9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:19.646 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.644+0000 7f5899ba5640 1 -- 172.21.15.167:0/693803297 >> v1:172.21.15.204:6789/0 conn(0x7f58940b50a0 legacy=0x7f58940735c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:19.646 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.644+0000 7f5899ba5640 1 -- 172.21.15.167:0/693803297 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5894160150 con 0x7f589413eeb0 2024-05-11T08:06:19.647 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.644+0000 7f58993a4640 1 --2- 172.21.15.167:0/693803297 >> v2:172.21.15.204:3300/0 conn(0x7f589413f290 0x7f589406f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T08:06:19.647 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.645+0000 7f5899ba5640 1 --2- 172.21.15.167:0/693803297 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f589413eeb0 0x7f589406f480 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f5894074520 tx=0x7f589002a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dde7e067cd02edb server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:19.647 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.645+0000 7f5898ba3640 1 -- 172.21.15.167:0/693803297 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5890036070 con 0x7f589413eeb0 2024-05-11T08:06:19.647 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.645+0000 7f5898ba3640 1 -- 172.21.15.167:0/693803297 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5890031600 con 0x7f589413eeb0 2024-05-11T08:06:19.647 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.645+0000 7f5898ba3640 1 -- 172.21.15.167:0/693803297 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58900319d0 con 0x7f589413eeb0 2024-05-11T08:06:19.647 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.646+0000 7f589be09640 1 -- 172.21.15.167:0/693803297 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f589413eeb0 msgr2=0x7f589406f480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:19.647 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.646+0000 7f589be09640 1 --2- 172.21.15.167:0/693803297 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f589413eeb0 0x7f589406f480 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f5894074520 tx=0x7f589002a610 comp rx=0 tx=0).stop 2024-05-11T08:06:19.647 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.646+0000 7f589be09640 1 -- 172.21.15.167:0/693803297 shutdown_connections 2024-05-11T08:06:19.647 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.646+0000 7f589be09640 1 --2- 172.21.15.167:0/693803297 >> v2:172.21.15.204:3300/0 conn(0x7f589413f290 0x7f589406f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:19.647 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.646+0000 7f589be09640 1 --2- 172.21.15.167:0/693803297 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f589413eeb0 0x7f589406f480 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:19.647 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.646+0000 7f589be09640 1 -- 172.21.15.167:0/693803297 >> 172.21.15.167:0/693803297 conn(0x7f58940cd8f0 msgr2=0x7f58940cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:19.647 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.646+0000 7f589be09640 1 -- 172.21.15.167:0/693803297 shutdown_connections 2024-05-11T08:06:19.647 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.646+0000 7f589be09640 1 -- 172.21.15.167:0/693803297 wait complete. 2024-05-11T08:06:19.650 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.647+0000 7f589be09640 1 Processor -- start 2024-05-11T08:06:19.650 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.647+0000 7f589be09640 1 -- start start 2024-05-11T08:06:19.650 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.648+0000 7f589be09640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f58940b50a0 0x7f58941f1930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:19.650 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.648+0000 7f589be09640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f589413eeb0 0x7f5894150cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:19.650 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.648+0000 7f589be09640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f589413f290 0x7f5894151230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:19.650 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.648+0000 7f589be09640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5894073cd0 con 0x7f589413eeb0 2024-05-11T08:06:19.650 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.648+0000 7f589be09640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5894160710 con 0x7f589413f290 2024-05-11T08:06:19.650 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.648+0000 7f589be09640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f5894073e50 con 0x7f58940b50a0 2024-05-11T08:06:19.650 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.648+0000 7f58993a4640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f589413eeb0 0x7f5894150cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:19.650 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.648+0000 7f58993a4640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f589413eeb0 0x7f5894150cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:54562/0 (socket says 172.21.15.167:54562) 2024-05-11T08:06:19.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.648+0000 7f58993a4640 1 -- 172.21.15.167:0/2547578252 learned_addr learned my addr 172.21.15.167:0/2547578252 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:19.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.648+0000 7f5899ba5640 1 --2- 172.21.15.167:0/2547578252 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f58940b50a0 0x7f58941f1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:19.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.648+0000 7f589a3a6640 1 --2- 172.21.15.167:0/2547578252 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f589413f290 0x7f5894151230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:19.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.648+0000 7f58993a4640 1 -- 172.21.15.167:0/2547578252 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f58940b50a0 msgr2=0x7f58941f1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:19.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.648+0000 7f58993a4640 1 --2- 172.21.15.167:0/2547578252 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f58940b50a0 0x7f58941f1930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:19.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.648+0000 7f58993a4640 1 -- 172.21.15.167:0/2547578252 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f589413f290 msgr2=0x7f5894151230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:19.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.648+0000 7f58993a4640 1 --2- 172.21.15.167:0/2547578252 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f589413f290 0x7f5894151230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:19.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.648+0000 7f58993a4640 1 -- 172.21.15.167:0/2547578252 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5894151ab0 con 0x7f589413eeb0 2024-05-11T08:06:19.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.648+0000 7f58993a4640 1 --2- 172.21.15.167:0/2547578252 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f589413eeb0 0x7f5894150cf0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f588400c7f0 tx=0x7f588400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:19.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.648+0000 7f588a7fc640 1 -- 172.21.15.167:0/2547578252 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5884015070 con 0x7f589413eeb0 2024-05-11T08:06:19.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.648+0000 7f588a7fc640 1 -- 172.21.15.167:0/2547578252 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5884007820 con 0x7f589413eeb0 2024-05-11T08:06:19.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.648+0000 7f588a7fc640 1 -- 172.21.15.167:0/2547578252 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5884007bd0 con 0x7f589413eeb0 2024-05-11T08:06:19.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.648+0000 7f589be09640 1 -- 172.21.15.167:0/2547578252 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5894130fa0 con 0x7f589413eeb0 2024-05-11T08:06:19.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.649+0000 7f589be09640 1 -- 172.21.15.167:0/2547578252 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5894151d90 con 0x7f589413eeb0 2024-05-11T08:06:19.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.649+0000 7f589be09640 1 -- 172.21.15.167:0/2547578252 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5894069590 con 0x7f589413eeb0 2024-05-11T08:06:19.652 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.650+0000 7f588a7fc640 1 -- 172.21.15.167:0/2547578252 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f5884018070 con 0x7f589413eeb0 2024-05-11T08:06:19.652 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.651+0000 7f588a7fc640 1 --2- 172.21.15.167:0/2547578252 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f58780762d0 0x7f5878078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:19.652 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.651+0000 7f5899ba5640 1 --2- 172.21.15.167:0/2547578252 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f58780762d0 0x7f5878078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:19.652 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.651+0000 7f588a7fc640 1 -- 172.21.15.167:0/2547578252 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f5884097990 con 0x7f589413eeb0 2024-05-11T08:06:19.652 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.651+0000 7f5899ba5640 1 --2- 172.21.15.167:0/2547578252 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f58780762d0 0x7f5878078790 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f5890033000 tx=0x7f5890009000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:19.655 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.655+0000 7f588a7fc640 1 -- 172.21.15.167:0/2547578252 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f5884060960 con 0x7f589413eeb0 2024-05-11T08:06:19.835 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.835+0000 7f589be09640 1 -- 172.21.15.167:0/2547578252 --> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f589406c640 con 0x7f58780762d0 2024-05-11T08:06:19.843 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.843+0000 7f588a7fc640 1 -- 172.21.15.167:0/2547578252 <== mgr.4113 v2:172.21.15.167:6832/253569496 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332523 (secure 0 0 0) 0x7f589406c640 con 0x7f58780762d0 2024-05-11T08:06:19.843 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T08:06:19.848 INFO:teuthology.orchestra.run.smithi167.stderr:dumped all 2024-05-11T08:06:19.850 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.846+0000 7f589be09640 1 -- 172.21.15.167:0/2547578252 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f58780762d0 msgr2=0x7f5878078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:19.850 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.846+0000 7f589be09640 1 --2- 172.21.15.167:0/2547578252 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f58780762d0 0x7f5878078790 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f5890033000 tx=0x7f5890009000 comp rx=0 tx=0).stop 2024-05-11T08:06:19.850 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.846+0000 7f589be09640 1 -- 172.21.15.167:0/2547578252 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f589413eeb0 msgr2=0x7f5894150cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:19.850 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.846+0000 7f589be09640 1 --2- 172.21.15.167:0/2547578252 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f589413eeb0 0x7f5894150cf0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f588400c7f0 tx=0x7f588400ccb0 comp rx=0 tx=0).stop 2024-05-11T08:06:19.850 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.846+0000 7f589be09640 1 -- 172.21.15.167:0/2547578252 shutdown_connections 2024-05-11T08:06:19.850 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.846+0000 7f589be09640 1 --2- 172.21.15.167:0/2547578252 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f58780762d0 0x7f5878078790 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:19.850 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.846+0000 7f589be09640 1 --2- 172.21.15.167:0/2547578252 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f589413f290 0x7f5894151230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:19.850 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.846+0000 7f589be09640 1 --2- 172.21.15.167:0/2547578252 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f589413eeb0 0x7f5894150cf0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:19.850 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.846+0000 7f589be09640 1 --2- 172.21.15.167:0/2547578252 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f58940b50a0 0x7f58941f1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:19.850 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.846+0000 7f589be09640 1 -- 172.21.15.167:0/2547578252 >> 172.21.15.167:0/2547578252 conn(0x7f58940cd8f0 msgr2=0x7f589415e8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:19.850 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.846+0000 7f589be09640 1 -- 172.21.15.167:0/2547578252 shutdown_connections 2024-05-11T08:06:19.850 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:19.846+0000 7f589be09640 1 -- 172.21.15.167:0/2547578252 wait complete. 2024-05-11T08:06:19.858 INFO:teuthology.orchestra.run.smithi167.stdout:{"pg_ready":true,"pg_map":{"version":39,"stamp":"2024-05-11T08:06:17.922016+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":149,"num_write_kb":1646,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":114,"ondisk_log_size":114,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219604,"kb_used_data":3432,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755116,"statfs":{"total":773094113280,"available":772869238784,"internally_reserved":0,"allocated":3514368,"data_stored":2074472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002936"},"pg_stats":[{"pgid":"2.1f","version":"22'4","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.392985+0000","last_change":"2024-05-11T08:05:54.368188+0000","last_active":"2024-05-11T08:06:00.392985+0000","last_peered":"2024-05-11T08:06:00.392985+0000","last_clean":"2024-05-11T08:06:00.392985+0000","last_became_active":"2024-05-11T08:05:54.367881+0000","last_became_peered":"2024-05-11T08:05:54.367881+0000","last_unstale":"2024-05-11T08:06:00.392985+0000","last_undegraded":"2024-05-11T08:06:00.392985+0000","last_fullsized":"2024-05-11T08:06:00.392985+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_clean_scrub_stamp":"2024-05-11T08:05:53.331701+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:22:53.142453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.645941+0000","last_change":"2024-05-11T08:05:55.367955+0000","last_active":"2024-05-11T08:06:01.645941+0000","last_peered":"2024-05-11T08:06:01.645941+0000","last_clean":"2024-05-11T08:06:01.645941+0000","last_became_active":"2024-05-11T08:05:55.366260+0000","last_became_peered":"2024-05-11T08:05:55.366260+0000","last_unstale":"2024-05-11T08:06:01.645941+0000","last_undegraded":"2024-05-11T08:06:01.645941+0000","last_fullsized":"2024-05-11T08:06:01.645941+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:44:43.387850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.647113+0000","last_change":"2024-05-11T08:05:59.558909+0000","last_active":"2024-05-11T08:06:01.647113+0000","last_peered":"2024-05-11T08:06:01.647113+0000","last_clean":"2024-05-11T08:06:01.647113+0000","last_became_active":"2024-05-11T08:05:59.558344+0000","last_became_peered":"2024-05-11T08:05:59.558344+0000","last_unstale":"2024-05-11T08:06:01.647113+0000","last_undegraded":"2024-05-11T08:06:01.647113+0000","last_fullsized":"2024-05-11T08:06:01.647113+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_clean_scrub_stamp":"2024-05-11T08:05:58.374552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:50:45.024927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"22'1","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.646170+0000","last_change":"2024-05-11T08:05:54.371497+0000","last_active":"2024-05-11T08:06:01.646170+0000","last_peered":"2024-05-11T08:06:01.646170+0000","last_clean":"2024-05-11T08:06:01.646170+0000","last_became_active":"2024-05-11T08:05:54.371241+0000","last_became_peered":"2024-05-11T08:05:54.371241+0000","last_unstale":"2024-05-11T08:06:01.646170+0000","last_undegraded":"2024-05-11T08:06:01.646170+0000","last_fullsized":"2024-05-11T08:06:01.646170+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_clean_scrub_stamp":"2024-05-11T08:05:53.331701+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:33:25.907598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.392845+0000","last_change":"2024-05-11T08:05:55.369937+0000","last_active":"2024-05-11T08:06:00.392845+0000","last_peered":"2024-05-11T08:06:00.392845+0000","last_clean":"2024-05-11T08:06:00.392845+0000","last_became_active":"2024-05-11T08:05:55.369476+0000","last_became_peered":"2024-05-11T08:05:55.369476+0000","last_unstale":"2024-05-11T08:06:00.392845+0000","last_undegraded":"2024-05-11T08:06:00.392845+0000","last_fullsized":"2024-05-11T08:06:00.392845+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:16:46.423362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.823013+0000","last_change":"2024-05-11T08:05:59.407301+0000","last_active":"2024-05-11T08:06:00.823013+0000","last_peered":"2024-05-11T08:06:00.823013+0000","last_clean":"2024-05-11T08:06:00.823013+0000","last_became_active":"2024-05-11T08:05:59.406627+0000","last_became_peered":"2024-05-11T08:05:59.406627+0000","last_unstale":"2024-05-11T08:06:00.823013+0000","last_undegraded":"2024-05-11T08:06:00.823013+0000","last_fullsized":"2024-05-11T08:06:00.823013+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_clean_scrub_stamp":"2024-05-11T08:05:58.374552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:41:42.964013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"22'2","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.388522+0000","last_change":"2024-05-11T08:05:54.367998+0000","last_active":"2024-05-11T08:06:00.388522+0000","last_peered":"2024-05-11T08:06:00.388522+0000","last_clean":"2024-05-11T08:06:00.388522+0000","last_became_active":"2024-05-11T08:05:54.367681+0000","last_became_peered":"2024-05-11T08:05:54.367681+0000","last_unstale":"2024-05-11T08:06:00.388522+0000","last_undegraded":"2024-05-11T08:06:00.388522+0000","last_fullsized":"2024-05-11T08:06:00.388522+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_clean_scrub_stamp":"2024-05-11T08:05:53.331701+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:50:17.617231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.389537+0000","last_change":"2024-05-11T08:05:55.379931+0000","last_active":"2024-05-11T08:06:00.389537+0000","last_peered":"2024-05-11T08:06:00.389537+0000","last_clean":"2024-05-11T08:06:00.389537+0000","last_became_active":"2024-05-11T08:05:55.379656+0000","last_became_peered":"2024-05-11T08:05:55.379656+0000","last_unstale":"2024-05-11T08:06:00.389537+0000","last_undegraded":"2024-05-11T08:06:00.389537+0000","last_fullsized":"2024-05-11T08:06:00.389537+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:14:09.302889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.823238+0000","last_change":"2024-05-11T08:05:59.404405+0000","last_active":"2024-05-11T08:06:00.823238+0000","last_peered":"2024-05-11T08:06:00.823238+0000","last_clean":"2024-05-11T08:06:00.823238+0000","last_became_active":"2024-05-11T08:05:59.403407+0000","last_became_peered":"2024-05-11T08:05:59.403407+0000","last_unstale":"2024-05-11T08:06:00.823238+0000","last_undegraded":"2024-05-11T08:06:00.823238+0000","last_fullsized":"2024-05-11T08:06:00.823238+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_clean_scrub_stamp":"2024-05-11T08:05:58.374552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:57:03.981207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"22'2","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.388963+0000","last_change":"2024-05-11T08:05:54.369560+0000","last_active":"2024-05-11T08:06:00.388963+0000","last_peered":"2024-05-11T08:06:00.388963+0000","last_clean":"2024-05-11T08:06:00.388963+0000","last_became_active":"2024-05-11T08:05:54.369150+0000","last_became_peered":"2024-05-11T08:05:54.369150+0000","last_unstale":"2024-05-11T08:06:00.388963+0000","last_undegraded":"2024-05-11T08:06:00.388963+0000","last_fullsized":"2024-05-11T08:06:00.388963+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_clean_scrub_stamp":"2024-05-11T08:05:53.331701+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:17:22.080661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.389718+0000","last_change":"2024-05-11T08:05:55.379813+0000","last_active":"2024-05-11T08:06:00.389718+0000","last_peered":"2024-05-11T08:06:00.389718+0000","last_clean":"2024-05-11T08:06:00.389718+0000","last_became_active":"2024-05-11T08:05:55.379575+0000","last_became_peered":"2024-05-11T08:05:55.379575+0000","last_unstale":"2024-05-11T08:06:00.389718+0000","last_undegraded":"2024-05-11T08:06:00.389718+0000","last_fullsized":"2024-05-11T08:06:00.389718+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:28:24.740747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.823228+0000","last_change":"2024-05-11T08:05:59.412477+0000","last_active":"2024-05-11T08:06:00.823228+0000","last_peered":"2024-05-11T08:06:00.823228+0000","last_clean":"2024-05-11T08:06:00.823228+0000","last_became_active":"2024-05-11T08:05:59.411969+0000","last_became_peered":"2024-05-11T08:05:59.411969+0000","last_unstale":"2024-05-11T08:06:00.823228+0000","last_undegraded":"2024-05-11T08:06:00.823228+0000","last_fullsized":"2024-05-11T08:06:00.823228+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_clean_scrub_stamp":"2024-05-11T08:05:58.374552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:38:06.426124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.645499+0000","last_change":"2024-05-11T08:05:55.376503+0000","last_active":"2024-05-11T08:06:01.645499+0000","last_peered":"2024-05-11T08:06:01.645499+0000","last_clean":"2024-05-11T08:06:01.645499+0000","last_became_active":"2024-05-11T08:05:55.376198+0000","last_became_peered":"2024-05-11T08:05:55.376198+0000","last_unstale":"2024-05-11T08:06:01.645499+0000","last_undegraded":"2024-05-11T08:06:01.645499+0000","last_fullsized":"2024-05-11T08:06:01.645499+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:20:43.832246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"22'2","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.388669+0000","last_change":"2024-05-11T08:05:54.370164+0000","last_active":"2024-05-11T08:06:00.388669+0000","last_peered":"2024-05-11T08:06:00.388669+0000","last_clean":"2024-05-11T08:06:00.388669+0000","last_became_active":"2024-05-11T08:05:54.369893+0000","last_became_peered":"2024-05-11T08:05:54.369893+0000","last_unstale":"2024-05-11T08:06:00.388669+0000","last_undegraded":"2024-05-11T08:06:00.388669+0000","last_fullsized":"2024-05-11T08:06:00.388669+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_clean_scrub_stamp":"2024-05-11T08:05:53.331701+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:53:10.612688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.823618+0000","last_change":"2024-05-11T08:05:59.409212+0000","last_active":"2024-05-11T08:06:00.823618+0000","last_peered":"2024-05-11T08:06:00.823618+0000","last_clean":"2024-05-11T08:06:00.823618+0000","last_became_active":"2024-05-11T08:05:59.408832+0000","last_became_peered":"2024-05-11T08:05:59.408832+0000","last_unstale":"2024-05-11T08:06:00.823618+0000","last_undegraded":"2024-05-11T08:06:00.823618+0000","last_fullsized":"2024-05-11T08:06:00.823618+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_clean_scrub_stamp":"2024-05-11T08:05:58.374552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:45:28.584065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.646456+0000","last_change":"2024-05-11T08:05:55.362770+0000","last_active":"2024-05-11T08:06:01.646456+0000","last_peered":"2024-05-11T08:06:01.646456+0000","last_clean":"2024-05-11T08:06:01.646456+0000","last_became_active":"2024-05-11T08:05:55.362481+0000","last_became_peered":"2024-05-11T08:05:55.362481+0000","last_unstale":"2024-05-11T08:06:01.646456+0000","last_undegraded":"2024-05-11T08:06:01.646456+0000","last_fullsized":"2024-05-11T08:06:01.646456+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:22:00.631022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.645921+0000","last_change":"2024-05-11T08:05:54.369496+0000","last_active":"2024-05-11T08:06:01.645921+0000","last_peered":"2024-05-11T08:06:01.645921+0000","last_clean":"2024-05-11T08:06:01.645921+0000","last_became_active":"2024-05-11T08:05:54.369246+0000","last_became_peered":"2024-05-11T08:05:54.369246+0000","last_unstale":"2024-05-11T08:06:01.645921+0000","last_undegraded":"2024-05-11T08:06:01.645921+0000","last_fullsized":"2024-05-11T08:06:01.645921+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_clean_scrub_stamp":"2024-05-11T08:05:53.331701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:30:23.111497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.823390+0000","last_change":"2024-05-11T08:05:59.403693+0000","last_active":"2024-05-11T08:06:00.823390+0000","last_peered":"2024-05-11T08:06:00.823390+0000","last_clean":"2024-05-11T08:06:00.823390+0000","last_became_active":"2024-05-11T08:05:59.403195+0000","last_became_peered":"2024-05-11T08:05:59.403195+0000","last_unstale":"2024-05-11T08:06:00.823390+0000","last_undegraded":"2024-05-11T08:06:00.823390+0000","last_fullsized":"2024-05-11T08:06:00.823390+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_clean_scrub_stamp":"2024-05-11T08:05:58.374552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:28:28.773313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.645822+0000","last_change":"2024-05-11T08:05:55.366582+0000","last_active":"2024-05-11T08:06:01.645822+0000","last_peered":"2024-05-11T08:06:01.645822+0000","last_clean":"2024-05-11T08:06:01.645822+0000","last_became_active":"2024-05-11T08:05:55.366309+0000","last_became_peered":"2024-05-11T08:05:55.366309+0000","last_unstale":"2024-05-11T08:06:01.645822+0000","last_undegraded":"2024-05-11T08:06:01.645822+0000","last_fullsized":"2024-05-11T08:06:01.645822+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:32:48.010809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.645408+0000","last_change":"2024-05-11T08:05:54.368032+0000","last_active":"2024-05-11T08:06:01.645408+0000","last_peered":"2024-05-11T08:06:01.645408+0000","last_clean":"2024-05-11T08:06:01.645408+0000","last_became_active":"2024-05-11T08:05:54.367739+0000","last_became_peered":"2024-05-11T08:05:54.367739+0000","last_unstale":"2024-05-11T08:06:01.645408+0000","last_undegraded":"2024-05-11T08:06:01.645408+0000","last_fullsized":"2024-05-11T08:06:01.645408+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_clean_scrub_stamp":"2024-05-11T08:05:53.331701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:01:58.573898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.646280+0000","last_change":"2024-05-11T08:05:59.560219+0000","last_active":"2024-05-11T08:06:01.646280+0000","last_peered":"2024-05-11T08:06:01.646280+0000","last_clean":"2024-05-11T08:06:01.646280+0000","last_became_active":"2024-05-11T08:05:59.559821+0000","last_became_peered":"2024-05-11T08:05:59.559821+0000","last_unstale":"2024-05-11T08:06:01.646280+0000","last_undegraded":"2024-05-11T08:06:01.646280+0000","last_fullsized":"2024-05-11T08:06:01.646280+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_clean_scrub_stamp":"2024-05-11T08:05:58.374552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:51:30.882512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.823321+0000","last_change":"2024-05-11T08:05:55.370564+0000","last_active":"2024-05-11T08:06:00.823321+0000","last_peered":"2024-05-11T08:06:00.823321+0000","last_clean":"2024-05-11T08:06:00.823321+0000","last_became_active":"2024-05-11T08:05:55.370277+0000","last_became_peered":"2024-05-11T08:05:55.370277+0000","last_unstale":"2024-05-11T08:06:00.823321+0000","last_undegraded":"2024-05-11T08:06:00.823321+0000","last_fullsized":"2024-05-11T08:06:00.823321+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:30:24.121782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.387998+0000","last_change":"2024-05-11T08:05:54.371153+0000","last_active":"2024-05-11T08:06:00.387998+0000","last_peered":"2024-05-11T08:06:00.387998+0000","last_clean":"2024-05-11T08:06:00.387998+0000","last_became_active":"2024-05-11T08:05:54.369944+0000","last_became_peered":"2024-05-11T08:05:54.369944+0000","last_unstale":"2024-05-11T08:06:00.387998+0000","last_undegraded":"2024-05-11T08:06:00.387998+0000","last_fullsized":"2024-05-11T08:06:00.387998+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_clean_scrub_stamp":"2024-05-11T08:05:53.331701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:30:23.111497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.823584+0000","last_change":"2024-05-11T08:05:59.403650+0000","last_active":"2024-05-11T08:06:00.823584+0000","last_peered":"2024-05-11T08:06:00.823584+0000","last_clean":"2024-05-11T08:06:00.823584+0000","last_became_active":"2024-05-11T08:05:59.403181+0000","last_became_peered":"2024-05-11T08:05:59.403181+0000","last_unstale":"2024-05-11T08:06:00.823584+0000","last_undegraded":"2024-05-11T08:06:00.823584+0000","last_fullsized":"2024-05-11T08:06:00.823584+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_clean_scrub_stamp":"2024-05-11T08:05:58.374552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:51:30.882512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.392942+0000","last_change":"2024-05-11T08:05:55.365766+0000","last_active":"2024-05-11T08:06:00.392942+0000","last_peered":"2024-05-11T08:06:00.392942+0000","last_clean":"2024-05-11T08:06:00.392942+0000","last_became_active":"2024-05-11T08:05:55.365513+0000","last_became_peered":"2024-05-11T08:05:55.365513+0000","last_unstale":"2024-05-11T08:06:00.392942+0000","last_undegraded":"2024-05-11T08:06:00.392942+0000","last_fullsized":"2024-05-11T08:06:00.392942+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:43:37.948904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"22'2","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.646133+0000","last_change":"2024-05-11T08:05:54.368015+0000","last_active":"2024-05-11T08:06:01.646133+0000","last_peered":"2024-05-11T08:06:01.646133+0000","last_clean":"2024-05-11T08:06:01.646133+0000","last_became_active":"2024-05-11T08:05:54.367740+0000","last_became_peered":"2024-05-11T08:05:54.367740+0000","last_unstale":"2024-05-11T08:06:01.646133+0000","last_undegraded":"2024-05-11T08:06:01.646133+0000","last_fullsized":"2024-05-11T08:06:01.646133+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_clean_scrub_stamp":"2024-05-11T08:05:53.331701+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:30:23.111497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.823061+0000","last_change":"2024-05-11T08:05:59.559986+0000","last_active":"2024-05-11T08:06:00.823061+0000","last_peered":"2024-05-11T08:06:00.823061+0000","last_clean":"2024-05-11T08:06:00.823061+0000","last_became_active":"2024-05-11T08:05:59.559594+0000","last_became_peered":"2024-05-11T08:05:59.559594+0000","last_unstale":"2024-05-11T08:06:00.823061+0000","last_undegraded":"2024-05-11T08:06:00.823061+0000","last_fullsized":"2024-05-11T08:06:00.823061+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_clean_scrub_stamp":"2024-05-11T08:05:58.374552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:14:13.335454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.646670+0000","last_change":"2024-05-11T08:05:55.374876+0000","last_active":"2024-05-11T08:06:01.646670+0000","last_peered":"2024-05-11T08:06:01.646670+0000","last_clean":"2024-05-11T08:06:01.646670+0000","last_became_active":"2024-05-11T08:05:55.374617+0000","last_became_peered":"2024-05-11T08:05:55.374617+0000","last_unstale":"2024-05-11T08:06:01.646670+0000","last_undegraded":"2024-05-11T08:06:01.646670+0000","last_fullsized":"2024-05-11T08:06:01.646670+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:41:38.931447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"23'1","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.646291+0000","last_change":"2024-05-11T08:05:54.365008+0000","last_active":"2024-05-11T08:06:01.646291+0000","last_peered":"2024-05-11T08:06:01.646291+0000","last_clean":"2024-05-11T08:06:01.646291+0000","last_became_active":"2024-05-11T08:05:54.364698+0000","last_became_peered":"2024-05-11T08:05:54.364698+0000","last_unstale":"2024-05-11T08:06:01.646291+0000","last_undegraded":"2024-05-11T08:06:01.646291+0000","last_fullsized":"2024-05-11T08:06:01.646291+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_clean_scrub_stamp":"2024-05-11T08:05:53.331701+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:17:22.080661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.645075+0000","last_change":"2024-05-11T08:05:59.562139+0000","last_active":"2024-05-11T08:06:01.645075+0000","last_peered":"2024-05-11T08:06:01.645075+0000","last_clean":"2024-05-11T08:06:01.645075+0000","last_became_active":"2024-05-11T08:05:59.561081+0000","last_became_peered":"2024-05-11T08:05:59.561081+0000","last_unstale":"2024-05-11T08:06:01.645075+0000","last_undegraded":"2024-05-11T08:06:01.645075+0000","last_fullsized":"2024-05-11T08:06:01.645075+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_clean_scrub_stamp":"2024-05-11T08:05:58.374552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:41:33.826242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.645886+0000","last_change":"2024-05-11T08:05:55.371779+0000","last_active":"2024-05-11T08:06:01.645886+0000","last_peered":"2024-05-11T08:06:01.645886+0000","last_clean":"2024-05-11T08:06:01.645886+0000","last_became_active":"2024-05-11T08:05:55.368726+0000","last_became_peered":"2024-05-11T08:05:55.368726+0000","last_unstale":"2024-05-11T08:06:01.645886+0000","last_undegraded":"2024-05-11T08:06:01.645886+0000","last_fullsized":"2024-05-11T08:06:01.645886+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:32:48.010809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.388659+0000","last_change":"2024-05-11T08:05:54.370339+0000","last_active":"2024-05-11T08:06:00.388659+0000","last_peered":"2024-05-11T08:06:00.388659+0000","last_clean":"2024-05-11T08:06:00.388659+0000","last_became_active":"2024-05-11T08:05:54.370026+0000","last_became_peered":"2024-05-11T08:05:54.370026+0000","last_unstale":"2024-05-11T08:06:00.388659+0000","last_undegraded":"2024-05-11T08:06:00.388659+0000","last_fullsized":"2024-05-11T08:06:00.388659+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_clean_scrub_stamp":"2024-05-11T08:05:53.331701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:32:43.591947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.392567+0000","last_change":"2024-05-11T08:05:59.558387+0000","last_active":"2024-05-11T08:06:00.392567+0000","last_peered":"2024-05-11T08:06:00.392567+0000","last_clean":"2024-05-11T08:06:00.392567+0000","last_became_active":"2024-05-11T08:05:59.557980+0000","last_became_peered":"2024-05-11T08:05:59.557980+0000","last_unstale":"2024-05-11T08:06:00.392567+0000","last_undegraded":"2024-05-11T08:06:00.392567+0000","last_fullsized":"2024-05-11T08:06:00.392567+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_clean_scrub_stamp":"2024-05-11T08:05:58.374552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:12:42.864479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.389813+0000","last_change":"2024-05-11T08:05:55.373737+0000","last_active":"2024-05-11T08:06:00.389813+0000","last_peered":"2024-05-11T08:06:00.389813+0000","last_clean":"2024-05-11T08:06:00.389813+0000","last_became_active":"2024-05-11T08:05:55.373459+0000","last_became_peered":"2024-05-11T08:05:55.373459+0000","last_unstale":"2024-05-11T08:06:00.389813+0000","last_undegraded":"2024-05-11T08:06:00.389813+0000","last_fullsized":"2024-05-11T08:06:00.389813+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:51:26.849946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.645985+0000","last_change":"2024-05-11T08:05:54.367679+0000","last_active":"2024-05-11T08:06:01.645985+0000","last_peered":"2024-05-11T08:06:01.645985+0000","last_clean":"2024-05-11T08:06:01.645985+0000","last_became_active":"2024-05-11T08:05:54.367393+0000","last_became_peered":"2024-05-11T08:05:54.367393+0000","last_unstale":"2024-05-11T08:06:01.645985+0000","last_undegraded":"2024-05-11T08:06:01.645985+0000","last_fullsized":"2024-05-11T08:06:01.645985+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_clean_scrub_stamp":"2024-05-11T08:05:53.331701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:50:17.617231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.645628+0000","last_change":"2024-05-11T08:05:59.413104+0000","last_active":"2024-05-11T08:06:01.645628+0000","last_peered":"2024-05-11T08:06:01.645628+0000","last_clean":"2024-05-11T08:06:01.645628+0000","last_became_active":"2024-05-11T08:05:59.412442+0000","last_became_peered":"2024-05-11T08:05:59.412442+0000","last_unstale":"2024-05-11T08:06:01.645628+0000","last_undegraded":"2024-05-11T08:06:01.645628+0000","last_fullsized":"2024-05-11T08:06:01.645628+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_clean_scrub_stamp":"2024-05-11T08:05:58.374552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:57:03.981207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.823506+0000","last_change":"2024-05-11T08:05:55.377393+0000","last_active":"2024-05-11T08:06:00.823506+0000","last_peered":"2024-05-11T08:06:00.823506+0000","last_clean":"2024-05-11T08:06:00.823506+0000","last_became_active":"2024-05-11T08:05:55.377129+0000","last_became_peered":"2024-05-11T08:05:55.377129+0000","last_unstale":"2024-05-11T08:06:00.823506+0000","last_undegraded":"2024-05-11T08:06:00.823506+0000","last_fullsized":"2024-05-11T08:06:00.823506+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:05:55.494912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.389461+0000","last_change":"2024-05-11T08:05:54.361751+0000","last_active":"2024-05-11T08:06:00.389461+0000","last_peered":"2024-05-11T08:06:00.389461+0000","last_clean":"2024-05-11T08:06:00.389461+0000","last_became_active":"2024-05-11T08:05:54.361484+0000","last_became_peered":"2024-05-11T08:05:54.361484+0000","last_unstale":"2024-05-11T08:06:00.389461+0000","last_undegraded":"2024-05-11T08:06:00.389461+0000","last_fullsized":"2024-05-11T08:06:00.389461+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_clean_scrub_stamp":"2024-05-11T08:05:53.331701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:53:10.612688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.645837+0000","last_change":"2024-05-11T08:05:59.409532+0000","last_active":"2024-05-11T08:06:01.645837+0000","last_peered":"2024-05-11T08:06:01.645837+0000","last_clean":"2024-05-11T08:06:01.645837+0000","last_became_active":"2024-05-11T08:05:59.409096+0000","last_became_peered":"2024-05-11T08:05:59.409096+0000","last_unstale":"2024-05-11T08:06:01.645837+0000","last_undegraded":"2024-05-11T08:06:01.645837+0000","last_fullsized":"2024-05-11T08:06:01.645837+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_clean_scrub_stamp":"2024-05-11T08:05:58.374552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:12:42.864479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.645313+0000","last_change":"2024-05-11T08:05:55.368600+0000","last_active":"2024-05-11T08:06:01.645313+0000","last_peered":"2024-05-11T08:06:01.645313+0000","last_clean":"2024-05-11T08:06:01.645313+0000","last_became_active":"2024-05-11T08:05:55.368322+0000","last_became_peered":"2024-05-11T08:05:55.368322+0000","last_unstale":"2024-05-11T08:06:01.645313+0000","last_undegraded":"2024-05-11T08:06:01.645313+0000","last_fullsized":"2024-05-11T08:06:01.645313+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:38:18.066779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.645325+0000","last_change":"2024-05-11T08:05:54.369698+0000","last_active":"2024-05-11T08:06:01.645325+0000","last_peered":"2024-05-11T08:06:01.645325+0000","last_clean":"2024-05-11T08:06:01.645325+0000","last_became_active":"2024-05-11T08:05:54.369387+0000","last_became_peered":"2024-05-11T08:05:54.369387+0000","last_unstale":"2024-05-11T08:06:01.645325+0000","last_undegraded":"2024-05-11T08:06:01.645325+0000","last_fullsized":"2024-05-11T08:06:01.645325+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_clean_scrub_stamp":"2024-05-11T08:05:53.331701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:14:18.956461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.646283+0000","last_change":"2024-05-11T08:05:59.413264+0000","last_active":"2024-05-11T08:06:01.646283+0000","last_peered":"2024-05-11T08:06:01.646283+0000","last_clean":"2024-05-11T08:06:01.646283+0000","last_became_active":"2024-05-11T08:05:59.412704+0000","last_became_peered":"2024-05-11T08:05:59.412704+0000","last_unstale":"2024-05-11T08:06:01.646283+0000","last_undegraded":"2024-05-11T08:06:01.646283+0000","last_fullsized":"2024-05-11T08:06:01.646283+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_clean_scrub_stamp":"2024-05-11T08:05:58.374552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:45:28.584065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.392303+0000","last_change":"2024-05-11T08:05:55.373518+0000","last_active":"2024-05-11T08:06:00.392303+0000","last_peered":"2024-05-11T08:06:00.392303+0000","last_clean":"2024-05-11T08:06:00.392303+0000","last_became_active":"2024-05-11T08:05:55.373067+0000","last_became_peered":"2024-05-11T08:05:55.373067+0000","last_unstale":"2024-05-11T08:06:00.392303+0000","last_undegraded":"2024-05-11T08:06:00.392303+0000","last_fullsized":"2024-05-11T08:06:00.392303+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:35:55.478239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.388011+0000","last_change":"2024-05-11T08:05:54.369563+0000","last_active":"2024-05-11T08:06:00.388011+0000","last_peered":"2024-05-11T08:06:00.388011+0000","last_clean":"2024-05-11T08:06:00.388011+0000","last_became_active":"2024-05-11T08:05:54.369257+0000","last_became_peered":"2024-05-11T08:05:54.369257+0000","last_unstale":"2024-05-11T08:06:00.388011+0000","last_undegraded":"2024-05-11T08:06:00.388011+0000","last_fullsized":"2024-05-11T08:06:00.388011+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_clean_scrub_stamp":"2024-05-11T08:05:53.331701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:05:54.484627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.392895+0000","last_change":"2024-05-11T08:05:59.560554+0000","last_active":"2024-05-11T08:06:00.392895+0000","last_peered":"2024-05-11T08:06:00.392895+0000","last_clean":"2024-05-11T08:06:00.392895+0000","last_became_active":"2024-05-11T08:05:59.560163+0000","last_became_peered":"2024-05-11T08:05:59.560163+0000","last_unstale":"2024-05-11T08:06:00.392895+0000","last_undegraded":"2024-05-11T08:06:00.392895+0000","last_fullsized":"2024-05-11T08:06:00.392895+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_clean_scrub_stamp":"2024-05-11T08:05:58.374552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:19:03.458410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.645474+0000","last_change":"2024-05-11T08:05:55.371707+0000","last_active":"2024-05-11T08:06:01.645474+0000","last_peered":"2024-05-11T08:06:01.645474+0000","last_clean":"2024-05-11T08:06:01.645474+0000","last_became_active":"2024-05-11T08:05:55.371443+0000","last_became_peered":"2024-05-11T08:05:55.371443+0000","last_unstale":"2024-05-11T08:06:01.645474+0000","last_undegraded":"2024-05-11T08:06:01.645474+0000","last_fullsized":"2024-05-11T08:06:01.645474+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:16:46.423362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.389333+0000","last_change":"2024-05-11T08:05:54.361406+0000","last_active":"2024-05-11T08:06:00.389333+0000","last_peered":"2024-05-11T08:06:00.389333+0000","last_clean":"2024-05-11T08:06:00.389333+0000","last_became_active":"2024-05-11T08:05:54.361065+0000","last_became_peered":"2024-05-11T08:05:54.361065+0000","last_unstale":"2024-05-11T08:06:00.389333+0000","last_undegraded":"2024-05-11T08:06:00.389333+0000","last_fullsized":"2024-05-11T08:06:00.389333+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_clean_scrub_stamp":"2024-05-11T08:05:53.331701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:33:25.907598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":118,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.388646+0000","last_change":"2024-05-11T08:05:45.772368+0000","last_active":"2024-05-11T08:06:00.388646+0000","last_peered":"2024-05-11T08:06:00.388646+0000","last_clean":"2024-05-11T08:06:00.388646+0000","last_became_active":"2024-05-11T08:05:45.771373+0000","last_became_peered":"2024-05-11T08:05:45.771373+0000","last_unstale":"2024-05-11T08:06:00.388646+0000","last_undegraded":"2024-05-11T08:06:00.388646+0000","last_fullsized":"2024-05-11T08:06:00.388646+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:44.616617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:44.616617+0000","last_clean_scrub_stamp":"2024-05-11T08:05:44.616617+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:02:08.078748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.645881+0000","last_change":"2024-05-11T08:05:59.558739+0000","last_active":"2024-05-11T08:06:01.645881+0000","last_peered":"2024-05-11T08:06:01.645881+0000","last_clean":"2024-05-11T08:06:01.645881+0000","last_became_active":"2024-05-11T08:05:59.558307+0000","last_became_peered":"2024-05-11T08:05:59.558307+0000","last_unstale":"2024-05-11T08:06:01.645881+0000","last_undegraded":"2024-05-11T08:06:01.645881+0000","last_fullsized":"2024-05-11T08:06:01.645881+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_clean_scrub_stamp":"2024-05-11T08:05:58.374552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:28:28.773313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.388334+0000","last_change":"2024-05-11T08:05:55.379286+0000","last_active":"2024-05-11T08:06:00.388334+0000","last_peered":"2024-05-11T08:06:00.388334+0000","last_clean":"2024-05-11T08:06:00.388334+0000","last_became_active":"2024-05-11T08:05:55.378804+0000","last_became_peered":"2024-05-11T08:05:55.378804+0000","last_unstale":"2024-05-11T08:06:00.388334+0000","last_undegraded":"2024-05-11T08:06:00.388334+0000","last_fullsized":"2024-05-11T08:06:00.388334+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:56:59.948641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.646359+0000","last_change":"2024-05-11T08:05:54.364902+0000","last_active":"2024-05-11T08:06:01.646359+0000","last_peered":"2024-05-11T08:06:01.646359+0000","last_clean":"2024-05-11T08:06:01.646359+0000","last_became_active":"2024-05-11T08:05:54.364619+0000","last_became_peered":"2024-05-11T08:05:54.364619+0000","last_unstale":"2024-05-11T08:06:01.646359+0000","last_undegraded":"2024-05-11T08:06:01.646359+0000","last_fullsized":"2024-05-11T08:06:01.646359+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_clean_scrub_stamp":"2024-05-11T08:05:53.331701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:10:56.295408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.645635+0000","last_change":"2024-05-11T08:05:59.558409+0000","last_active":"2024-05-11T08:06:01.645635+0000","last_peered":"2024-05-11T08:06:01.645635+0000","last_clean":"2024-05-11T08:06:01.645635+0000","last_became_active":"2024-05-11T08:05:59.557939+0000","last_became_peered":"2024-05-11T08:05:59.557939+0000","last_unstale":"2024-05-11T08:06:01.645635+0000","last_undegraded":"2024-05-11T08:06:01.645635+0000","last_fullsized":"2024-05-11T08:06:01.645635+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_clean_scrub_stamp":"2024-05-11T08:05:58.374552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:57:03.981207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.389620+0000","last_change":"2024-05-11T08:05:55.376614+0000","last_active":"2024-05-11T08:06:00.389620+0000","last_peered":"2024-05-11T08:06:00.389620+0000","last_clean":"2024-05-11T08:06:00.389620+0000","last_became_active":"2024-05-11T08:05:55.376350+0000","last_became_peered":"2024-05-11T08:05:55.376350+0000","last_unstale":"2024-05-11T08:06:00.389620+0000","last_undegraded":"2024-05-11T08:06:00.389620+0000","last_fullsized":"2024-05-11T08:06:00.389620+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:12:38.831913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"22'1","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.645871+0000","last_change":"2024-05-11T08:05:54.369427+0000","last_active":"2024-05-11T08:06:01.645871+0000","last_peered":"2024-05-11T08:06:01.645871+0000","last_clean":"2024-05-11T08:06:01.645871+0000","last_became_active":"2024-05-11T08:05:54.369181+0000","last_became_peered":"2024-05-11T08:05:54.369181+0000","last_unstale":"2024-05-11T08:06:01.645871+0000","last_undegraded":"2024-05-11T08:06:01.645871+0000","last_fullsized":"2024-05-11T08:06:01.645871+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_clean_scrub_stamp":"2024-05-11T08:05:53.331701+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:05:54.484627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.392744+0000","last_change":"2024-05-11T08:05:59.411982+0000","last_active":"2024-05-11T08:06:00.392744+0000","last_peered":"2024-05-11T08:06:00.392744+0000","last_clean":"2024-05-11T08:06:00.392744+0000","last_became_active":"2024-05-11T08:05:59.411561+0000","last_became_peered":"2024-05-11T08:05:59.411561+0000","last_unstale":"2024-05-11T08:06:00.392744+0000","last_undegraded":"2024-05-11T08:06:00.392744+0000","last_fullsized":"2024-05-11T08:06:00.392744+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_clean_scrub_stamp":"2024-05-11T08:05:58.374552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:45:28.584065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.389445+0000","last_change":"2024-05-11T08:05:55.386175+0000","last_active":"2024-05-11T08:06:00.389445+0000","last_peered":"2024-05-11T08:06:00.389445+0000","last_clean":"2024-05-11T08:06:00.389445+0000","last_became_active":"2024-05-11T08:05:55.385883+0000","last_became_peered":"2024-05-11T08:05:55.385883+0000","last_unstale":"2024-05-11T08:06:00.389445+0000","last_undegraded":"2024-05-11T08:06:00.389445+0000","last_fullsized":"2024-05-11T08:06:00.389445+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:43:20.023451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"22'2","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.645669+0000","last_change":"2024-05-11T08:05:54.369810+0000","last_active":"2024-05-11T08:06:01.645669+0000","last_peered":"2024-05-11T08:06:01.645669+0000","last_clean":"2024-05-11T08:06:01.645669+0000","last_became_active":"2024-05-11T08:05:54.369512+0000","last_became_peered":"2024-05-11T08:05:54.369512+0000","last_unstale":"2024-05-11T08:06:01.645669+0000","last_undegraded":"2024-05-11T08:06:01.645669+0000","last_fullsized":"2024-05-11T08:06:01.645669+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_clean_scrub_stamp":"2024-05-11T08:05:53.331701+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:39:04.019231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.388134+0000","last_change":"2024-05-11T08:05:59.562447+0000","last_active":"2024-05-11T08:06:00.388134+0000","last_peered":"2024-05-11T08:06:00.388134+0000","last_clean":"2024-05-11T08:06:00.388134+0000","last_became_active":"2024-05-11T08:05:59.562033+0000","last_became_peered":"2024-05-11T08:05:59.562033+0000","last_unstale":"2024-05-11T08:06:00.388134+0000","last_undegraded":"2024-05-11T08:06:00.388134+0000","last_fullsized":"2024-05-11T08:06:00.388134+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_clean_scrub_stamp":"2024-05-11T08:05:58.374552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:41:33.826242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.388740+0000","last_change":"2024-05-11T08:05:55.386091+0000","last_active":"2024-05-11T08:06:00.388740+0000","last_peered":"2024-05-11T08:06:00.388740+0000","last_clean":"2024-05-11T08:06:00.388740+0000","last_became_active":"2024-05-11T08:05:55.385751+0000","last_became_peered":"2024-05-11T08:05:55.385751+0000","last_unstale":"2024-05-11T08:06:00.388740+0000","last_undegraded":"2024-05-11T08:06:00.388740+0000","last_fullsized":"2024-05-11T08:06:00.388740+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:12:38.831913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"22'1","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.823802+0000","last_change":"2024-05-11T08:05:54.372054+0000","last_active":"2024-05-11T08:06:00.823802+0000","last_peered":"2024-05-11T08:06:00.823802+0000","last_clean":"2024-05-11T08:06:00.823802+0000","last_became_active":"2024-05-11T08:05:54.371721+0000","last_became_peered":"2024-05-11T08:05:54.371721+0000","last_unstale":"2024-05-11T08:06:00.823802+0000","last_undegraded":"2024-05-11T08:06:00.823802+0000","last_fullsized":"2024-05-11T08:06:00.823802+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_clean_scrub_stamp":"2024-05-11T08:05:53.331701+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:40:29.770055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.823074+0000","last_change":"2024-05-11T08:05:59.404048+0000","last_active":"2024-05-11T08:06:00.823074+0000","last_peered":"2024-05-11T08:06:00.823074+0000","last_clean":"2024-05-11T08:06:00.823074+0000","last_became_active":"2024-05-11T08:05:59.403010+0000","last_became_peered":"2024-05-11T08:05:59.403010+0000","last_unstale":"2024-05-11T08:06:00.823074+0000","last_undegraded":"2024-05-11T08:06:00.823074+0000","last_fullsized":"2024-05-11T08:06:00.823074+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_clean_scrub_stamp":"2024-05-11T08:05:58.374552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:44:47.420416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.388312+0000","last_change":"2024-05-11T08:05:55.382224+0000","last_active":"2024-05-11T08:06:00.388312+0000","last_peered":"2024-05-11T08:06:00.388312+0000","last_clean":"2024-05-11T08:06:00.388312+0000","last_became_active":"2024-05-11T08:05:55.381614+0000","last_became_peered":"2024-05-11T08:05:55.381614+0000","last_unstale":"2024-05-11T08:06:00.388312+0000","last_undegraded":"2024-05-11T08:06:00.388312+0000","last_fullsized":"2024-05-11T08:06:00.388312+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:41:38.931447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"22'8","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.645775+0000","last_change":"2024-05-11T08:05:54.365900+0000","last_active":"2024-05-11T08:06:01.645775+0000","last_peered":"2024-05-11T08:06:01.645775+0000","last_clean":"2024-05-11T08:06:01.645775+0000","last_became_active":"2024-05-11T08:05:54.365638+0000","last_became_peered":"2024-05-11T08:05:54.365638+0000","last_unstale":"2024-05-11T08:06:01.645775+0000","last_undegraded":"2024-05-11T08:06:01.645775+0000","last_fullsized":"2024-05-11T08:06:01.645775+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_clean_scrub_stamp":"2024-05-11T08:05:53.331701+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:23:10.014766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.392726+0000","last_change":"2024-05-11T08:05:59.412069+0000","last_active":"2024-05-11T08:06:00.392726+0000","last_peered":"2024-05-11T08:06:00.392726+0000","last_clean":"2024-05-11T08:06:00.392726+0000","last_became_active":"2024-05-11T08:05:59.411674+0000","last_became_peered":"2024-05-11T08:05:59.411674+0000","last_unstale":"2024-05-11T08:06:00.392726+0000","last_undegraded":"2024-05-11T08:06:00.392726+0000","last_fullsized":"2024-05-11T08:06:00.392726+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_clean_scrub_stamp":"2024-05-11T08:05:58.374552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:51:30.882512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.645506+0000","last_change":"2024-05-11T08:05:55.364883+0000","last_active":"2024-05-11T08:06:01.645506+0000","last_peered":"2024-05-11T08:06:01.645506+0000","last_clean":"2024-05-11T08:06:01.645506+0000","last_became_active":"2024-05-11T08:05:55.364622+0000","last_became_peered":"2024-05-11T08:05:55.364622+0000","last_unstale":"2024-05-11T08:06:01.645506+0000","last_undegraded":"2024-05-11T08:06:01.645506+0000","last_fullsized":"2024-05-11T08:06:01.645506+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:35:39.553476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.645631+0000","last_change":"2024-05-11T08:05:54.368776+0000","last_active":"2024-05-11T08:06:01.645631+0000","last_peered":"2024-05-11T08:06:01.645631+0000","last_clean":"2024-05-11T08:06:01.645631+0000","last_became_active":"2024-05-11T08:05:54.368514+0000","last_became_peered":"2024-05-11T08:05:54.368514+0000","last_unstale":"2024-05-11T08:06:01.645631+0000","last_undegraded":"2024-05-11T08:06:01.645631+0000","last_fullsized":"2024-05-11T08:06:01.645631+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_clean_scrub_stamp":"2024-05-11T08:05:53.331701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:01:58.573898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.646534+0000","last_change":"2024-05-11T08:05:59.561579+0000","last_active":"2024-05-11T08:06:01.646534+0000","last_peered":"2024-05-11T08:06:01.646534+0000","last_clean":"2024-05-11T08:06:01.646534+0000","last_became_active":"2024-05-11T08:05:59.561027+0000","last_became_peered":"2024-05-11T08:05:59.561027+0000","last_unstale":"2024-05-11T08:06:01.646534+0000","last_undegraded":"2024-05-11T08:06:01.646534+0000","last_fullsized":"2024-05-11T08:06:01.646534+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_clean_scrub_stamp":"2024-05-11T08:05:58.374552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:06:20.739397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.388603+0000","last_change":"2024-05-11T08:05:55.387188+0000","last_active":"2024-05-11T08:06:00.388603+0000","last_peered":"2024-05-11T08:06:00.388603+0000","last_clean":"2024-05-11T08:06:00.388603+0000","last_became_active":"2024-05-11T08:05:55.386935+0000","last_became_peered":"2024-05-11T08:05:55.386935+0000","last_unstale":"2024-05-11T08:06:00.388603+0000","last_undegraded":"2024-05-11T08:06:00.388603+0000","last_fullsized":"2024-05-11T08:06:00.388603+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:14:09.302889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.389390+0000","last_change":"2024-05-11T08:05:54.370172+0000","last_active":"2024-05-11T08:06:00.389390+0000","last_peered":"2024-05-11T08:06:00.389390+0000","last_clean":"2024-05-11T08:06:00.389390+0000","last_became_active":"2024-05-11T08:05:54.369635+0000","last_became_peered":"2024-05-11T08:05:54.369635+0000","last_unstale":"2024-05-11T08:06:00.389390+0000","last_undegraded":"2024-05-11T08:06:00.389390+0000","last_fullsized":"2024-05-11T08:06:00.389390+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_clean_scrub_stamp":"2024-05-11T08:05:53.331701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:17:22.080661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.647316+0000","last_change":"2024-05-11T08:05:59.559006+0000","last_active":"2024-05-11T08:06:01.647316+0000","last_peered":"2024-05-11T08:06:01.647316+0000","last_clean":"2024-05-11T08:06:01.647316+0000","last_became_active":"2024-05-11T08:05:59.558255+0000","last_became_peered":"2024-05-11T08:05:59.558255+0000","last_unstale":"2024-05-11T08:06:01.647316+0000","last_undegraded":"2024-05-11T08:06:01.647316+0000","last_fullsized":"2024-05-11T08:06:01.647316+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_clean_scrub_stamp":"2024-05-11T08:05:58.374552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:32:53.866284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.392560+0000","last_change":"2024-05-11T08:05:55.375477+0000","last_active":"2024-05-11T08:06:00.392560+0000","last_peered":"2024-05-11T08:06:00.392560+0000","last_clean":"2024-05-11T08:06:00.392560+0000","last_became_active":"2024-05-11T08:05:55.375215+0000","last_became_peered":"2024-05-11T08:05:55.375215+0000","last_unstale":"2024-05-11T08:06:00.392560+0000","last_undegraded":"2024-05-11T08:06:00.392560+0000","last_fullsized":"2024-05-11T08:06:00.392560+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:38:18.066779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"22'2","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.392809+0000","last_change":"2024-05-11T08:05:54.368097+0000","last_active":"2024-05-11T08:06:00.392809+0000","last_peered":"2024-05-11T08:06:00.392809+0000","last_clean":"2024-05-11T08:06:00.392809+0000","last_became_active":"2024-05-11T08:05:54.367812+0000","last_became_peered":"2024-05-11T08:05:54.367812+0000","last_unstale":"2024-05-11T08:06:00.392809+0000","last_undegraded":"2024-05-11T08:06:00.392809+0000","last_fullsized":"2024-05-11T08:06:00.392809+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_clean_scrub_stamp":"2024-05-11T08:05:53.331701+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:00:20.233006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.389793+0000","last_change":"2024-05-11T08:05:59.558753+0000","last_active":"2024-05-11T08:06:00.389793+0000","last_peered":"2024-05-11T08:06:00.389793+0000","last_clean":"2024-05-11T08:06:00.389793+0000","last_became_active":"2024-05-11T08:05:59.558140+0000","last_became_peered":"2024-05-11T08:05:59.558140+0000","last_unstale":"2024-05-11T08:06:00.389793+0000","last_undegraded":"2024-05-11T08:06:00.389793+0000","last_fullsized":"2024-05-11T08:06:00.389793+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_clean_scrub_stamp":"2024-05-11T08:05:58.374552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:54:52.804997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.388229+0000","last_change":"2024-05-11T08:05:55.379691+0000","last_active":"2024-05-11T08:06:00.388229+0000","last_peered":"2024-05-11T08:06:00.388229+0000","last_clean":"2024-05-11T08:06:00.388229+0000","last_became_active":"2024-05-11T08:05:55.378519+0000","last_became_peered":"2024-05-11T08:05:55.378519+0000","last_unstale":"2024-05-11T08:06:00.388229+0000","last_undegraded":"2024-05-11T08:06:00.388229+0000","last_fullsized":"2024-05-11T08:06:00.388229+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:38:02.393559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"27'9","reported_seq":31,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:04.814504+0000","last_change":"2024-05-11T08:05:54.371057+0000","last_active":"2024-05-11T08:06:04.814504+0000","last_peered":"2024-05-11T08:06:04.814504+0000","last_clean":"2024-05-11T08:06:04.814504+0000","last_became_active":"2024-05-11T08:05:54.370746+0000","last_became_peered":"2024-05-11T08:05:54.370746+0000","last_unstale":"2024-05-11T08:06:04.814504+0000","last_undegraded":"2024-05-11T08:06:04.814504+0000","last_fullsized":"2024-05-11T08:06:04.814504+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_clean_scrub_stamp":"2024-05-11T08:05:53.331701+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:17:22.080661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.645768+0000","last_change":"2024-05-11T08:05:59.414118+0000","last_active":"2024-05-11T08:06:01.645768+0000","last_peered":"2024-05-11T08:06:01.645768+0000","last_clean":"2024-05-11T08:06:01.645768+0000","last_became_active":"2024-05-11T08:05:59.413183+0000","last_became_peered":"2024-05-11T08:05:59.413183+0000","last_unstale":"2024-05-11T08:06:01.645768+0000","last_undegraded":"2024-05-11T08:06:01.645768+0000","last_fullsized":"2024-05-11T08:06:01.645768+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_clean_scrub_stamp":"2024-05-11T08:05:58.374552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:28:28.773313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.823505+0000","last_change":"2024-05-11T08:05:55.373641+0000","last_active":"2024-05-11T08:06:00.823505+0000","last_peered":"2024-05-11T08:06:00.823505+0000","last_clean":"2024-05-11T08:06:00.823505+0000","last_became_active":"2024-05-11T08:05:55.373369+0000","last_became_peered":"2024-05-11T08:05:55.373369+0000","last_unstale":"2024-05-11T08:06:00.823505+0000","last_undegraded":"2024-05-11T08:06:00.823505+0000","last_fullsized":"2024-05-11T08:06:00.823505+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:32:44.602233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"22'3","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.645781+0000","last_change":"2024-05-11T08:05:54.366529+0000","last_active":"2024-05-11T08:06:01.645781+0000","last_peered":"2024-05-11T08:06:01.645781+0000","last_clean":"2024-05-11T08:06:01.645781+0000","last_became_active":"2024-05-11T08:05:54.366215+0000","last_became_peered":"2024-05-11T08:05:54.366215+0000","last_unstale":"2024-05-11T08:06:01.645781+0000","last_undegraded":"2024-05-11T08:06:01.645781+0000","last_fullsized":"2024-05-11T08:06:01.645781+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_clean_scrub_stamp":"2024-05-11T08:05:53.331701+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:53:10.612688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.823463+0000","last_change":"2024-05-11T08:05:59.407003+0000","last_active":"2024-05-11T08:06:00.823463+0000","last_peered":"2024-05-11T08:06:00.823463+0000","last_clean":"2024-05-11T08:06:00.823463+0000","last_became_active":"2024-05-11T08:05:59.406437+0000","last_became_peered":"2024-05-11T08:05:59.406437+0000","last_unstale":"2024-05-11T08:06:00.823463+0000","last_undegraded":"2024-05-11T08:06:00.823463+0000","last_fullsized":"2024-05-11T08:06:00.823463+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_clean_scrub_stamp":"2024-05-11T08:05:58.374552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:12:42.864479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.392952+0000","last_change":"2024-05-11T08:05:55.370107+0000","last_active":"2024-05-11T08:06:00.392952+0000","last_peered":"2024-05-11T08:06:00.392952+0000","last_clean":"2024-05-11T08:06:00.392952+0000","last_became_active":"2024-05-11T08:05:55.369823+0000","last_became_peered":"2024-05-11T08:05:55.369823+0000","last_unstale":"2024-05-11T08:06:00.392952+0000","last_undegraded":"2024-05-11T08:06:00.392952+0000","last_fullsized":"2024-05-11T08:06:00.392952+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:22:00.631022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.389731+0000","last_change":"2024-05-11T08:05:54.363934+0000","last_active":"2024-05-11T08:06:00.389731+0000","last_peered":"2024-05-11T08:06:00.389731+0000","last_clean":"2024-05-11T08:06:00.389731+0000","last_became_active":"2024-05-11T08:05:54.363428+0000","last_became_peered":"2024-05-11T08:05:54.363428+0000","last_unstale":"2024-05-11T08:06:00.389731+0000","last_undegraded":"2024-05-11T08:06:00.389731+0000","last_fullsized":"2024-05-11T08:06:00.389731+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_clean_scrub_stamp":"2024-05-11T08:05:53.331701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:30:23.111497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.645683+0000","last_change":"2024-05-11T08:05:59.403691+0000","last_active":"2024-05-11T08:06:01.645683+0000","last_peered":"2024-05-11T08:06:01.645683+0000","last_clean":"2024-05-11T08:06:01.645683+0000","last_became_active":"2024-05-11T08:05:59.403311+0000","last_became_peered":"2024-05-11T08:05:59.403311+0000","last_unstale":"2024-05-11T08:06:01.645683+0000","last_undegraded":"2024-05-11T08:06:01.645683+0000","last_fullsized":"2024-05-11T08:06:01.645683+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_clean_scrub_stamp":"2024-05-11T08:05:58.374552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:54:52.804997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.389689+0000","last_change":"2024-05-11T08:05:55.374886+0000","last_active":"2024-05-11T08:06:00.389689+0000","last_peered":"2024-05-11T08:06:00.389689+0000","last_clean":"2024-05-11T08:06:00.389689+0000","last_became_active":"2024-05-11T08:05:55.374622+0000","last_became_peered":"2024-05-11T08:05:55.374622+0000","last_unstale":"2024-05-11T08:06:00.389689+0000","last_undegraded":"2024-05-11T08:06:00.389689+0000","last_fullsized":"2024-05-11T08:06:00.389689+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:43:37.948904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.646049+0000","last_change":"2024-05-11T08:05:54.364291+0000","last_active":"2024-05-11T08:06:01.646049+0000","last_peered":"2024-05-11T08:06:01.646049+0000","last_clean":"2024-05-11T08:06:01.646049+0000","last_became_active":"2024-05-11T08:05:54.363676+0000","last_became_peered":"2024-05-11T08:05:54.363676+0000","last_unstale":"2024-05-11T08:06:01.646049+0000","last_undegraded":"2024-05-11T08:06:01.646049+0000","last_fullsized":"2024-05-11T08:06:01.646049+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_clean_scrub_stamp":"2024-05-11T08:05:53.331701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:53:10.612688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.647107+0000","last_change":"2024-05-11T08:05:59.558985+0000","last_active":"2024-05-11T08:06:01.647107+0000","last_peered":"2024-05-11T08:06:01.647107+0000","last_clean":"2024-05-11T08:06:01.647107+0000","last_became_active":"2024-05-11T08:05:59.558561+0000","last_became_peered":"2024-05-11T08:05:59.558561+0000","last_unstale":"2024-05-11T08:06:01.647107+0000","last_undegraded":"2024-05-11T08:06:01.647107+0000","last_fullsized":"2024-05-11T08:06:01.647107+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_clean_scrub_stamp":"2024-05-11T08:05:58.374552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:54:53.957903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.392778+0000","last_change":"2024-05-11T08:05:59.561482+0000","last_active":"2024-05-11T08:06:00.392778+0000","last_peered":"2024-05-11T08:06:00.392778+0000","last_clean":"2024-05-11T08:06:00.392778+0000","last_became_active":"2024-05-11T08:05:59.560743+0000","last_became_peered":"2024-05-11T08:05:59.560743+0000","last_unstale":"2024-05-11T08:06:00.392778+0000","last_undegraded":"2024-05-11T08:06:00.392778+0000","last_fullsized":"2024-05-11T08:06:00.392778+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_clean_scrub_stamp":"2024-05-11T08:05:58.374552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:43:24.056017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.645261+0000","last_change":"2024-05-11T08:05:55.367915+0000","last_active":"2024-05-11T08:06:01.645261+0000","last_peered":"2024-05-11T08:06:01.645261+0000","last_clean":"2024-05-11T08:06:01.645261+0000","last_became_active":"2024-05-11T08:05:55.367652+0000","last_became_peered":"2024-05-11T08:05:55.367652+0000","last_unstale":"2024-05-11T08:06:01.645261+0000","last_undegraded":"2024-05-11T08:06:01.645261+0000","last_fullsized":"2024-05-11T08:06:01.645261+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:35:03.601599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.389207+0000","last_change":"2024-05-11T08:05:54.368060+0000","last_active":"2024-05-11T08:06:00.389207+0000","last_peered":"2024-05-11T08:06:00.389207+0000","last_clean":"2024-05-11T08:06:00.389207+0000","last_became_active":"2024-05-11T08:05:54.367770+0000","last_became_peered":"2024-05-11T08:05:54.367770+0000","last_unstale":"2024-05-11T08:06:00.389207+0000","last_undegraded":"2024-05-11T08:06:00.389207+0000","last_fullsized":"2024-05-11T08:06:00.389207+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_clean_scrub_stamp":"2024-05-11T08:05:53.331701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:01:58.573898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.646058+0000","last_change":"2024-05-11T08:05:59.412426+0000","last_active":"2024-05-11T08:06:01.646058+0000","last_peered":"2024-05-11T08:06:01.646058+0000","last_clean":"2024-05-11T08:06:01.646058+0000","last_became_active":"2024-05-11T08:05:59.412011+0000","last_became_peered":"2024-05-11T08:05:59.412011+0000","last_unstale":"2024-05-11T08:06:01.646058+0000","last_undegraded":"2024-05-11T08:06:01.646058+0000","last_fullsized":"2024-05-11T08:06:01.646058+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_clean_scrub_stamp":"2024-05-11T08:05:58.374552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:12:42.864479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.645734+0000","last_change":"2024-05-11T08:05:55.363149+0000","last_active":"2024-05-11T08:06:01.645734+0000","last_peered":"2024-05-11T08:06:01.645734+0000","last_clean":"2024-05-11T08:06:01.645734+0000","last_became_active":"2024-05-11T08:05:55.362892+0000","last_became_peered":"2024-05-11T08:05:55.362892+0000","last_unstale":"2024-05-11T08:06:01.645734+0000","last_undegraded":"2024-05-11T08:06:01.645734+0000","last_fullsized":"2024-05-11T08:06:01.645734+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:38:18.066779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.647209+0000","last_change":"2024-05-11T08:05:54.368847+0000","last_active":"2024-05-11T08:06:01.647209+0000","last_peered":"2024-05-11T08:06:01.647209+0000","last_clean":"2024-05-11T08:06:01.647209+0000","last_became_active":"2024-05-11T08:05:54.368529+0000","last_became_peered":"2024-05-11T08:05:54.368529+0000","last_unstale":"2024-05-11T08:06:01.647209+0000","last_undegraded":"2024-05-11T08:06:01.647209+0000","last_fullsized":"2024-05-11T08:06:01.647209+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_clean_scrub_stamp":"2024-05-11T08:05:53.331701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:22:53.142453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"22'1","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.645710+0000","last_change":"2024-05-11T08:05:54.369435+0000","last_active":"2024-05-11T08:06:01.645710+0000","last_peered":"2024-05-11T08:06:01.645710+0000","last_clean":"2024-05-11T08:06:01.645710+0000","last_became_active":"2024-05-11T08:05:54.369163+0000","last_became_peered":"2024-05-11T08:05:54.369163+0000","last_unstale":"2024-05-11T08:06:01.645710+0000","last_undegraded":"2024-05-11T08:06:01.645710+0000","last_fullsized":"2024-05-11T08:06:01.645710+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_clean_scrub_stamp":"2024-05-11T08:05:53.331701+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:33:25.907598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.392757+0000","last_change":"2024-05-11T08:05:55.378586+0000","last_active":"2024-05-11T08:06:00.392757+0000","last_peered":"2024-05-11T08:06:00.392757+0000","last_clean":"2024-05-11T08:06:00.392757+0000","last_became_active":"2024-05-11T08:05:55.378323+0000","last_became_peered":"2024-05-11T08:05:55.378323+0000","last_unstale":"2024-05-11T08:06:00.392757+0000","last_undegraded":"2024-05-11T08:06:00.392757+0000","last_fullsized":"2024-05-11T08:06:00.392757+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:35:03.601599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.646291+0000","last_change":"2024-05-11T08:05:59.402315+0000","last_active":"2024-05-11T08:06:01.646291+0000","last_peered":"2024-05-11T08:06:01.646291+0000","last_clean":"2024-05-11T08:06:01.646291+0000","last_became_active":"2024-05-11T08:05:59.401886+0000","last_became_peered":"2024-05-11T08:05:59.401886+0000","last_unstale":"2024-05-11T08:06:01.646291+0000","last_undegraded":"2024-05-11T08:06:01.646291+0000","last_fullsized":"2024-05-11T08:06:01.646291+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_clean_scrub_stamp":"2024-05-11T08:05:58.374552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:33:42.953506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"22'1","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.646467+0000","last_change":"2024-05-11T08:05:54.370886+0000","last_active":"2024-05-11T08:06:01.646467+0000","last_peered":"2024-05-11T08:06:01.646467+0000","last_clean":"2024-05-11T08:06:01.646467+0000","last_became_active":"2024-05-11T08:05:54.370611+0000","last_became_peered":"2024-05-11T08:05:54.370611+0000","last_unstale":"2024-05-11T08:06:01.646467+0000","last_undegraded":"2024-05-11T08:06:01.646467+0000","last_fullsized":"2024-05-11T08:06:01.646467+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_clean_scrub_stamp":"2024-05-11T08:05:53.331701+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:36:34.523259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.823139+0000","last_change":"2024-05-11T08:05:55.373391+0000","last_active":"2024-05-11T08:06:00.823139+0000","last_peered":"2024-05-11T08:06:00.823139+0000","last_clean":"2024-05-11T08:06:00.823139+0000","last_became_active":"2024-05-11T08:05:55.373096+0000","last_became_peered":"2024-05-11T08:05:55.373096+0000","last_unstale":"2024-05-11T08:06:00.823139+0000","last_undegraded":"2024-05-11T08:06:00.823139+0000","last_fullsized":"2024-05-11T08:06:00.823139+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:53:11.622973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.647038+0000","last_change":"2024-05-11T08:05:59.403522+0000","last_active":"2024-05-11T08:06:01.647038+0000","last_peered":"2024-05-11T08:06:01.647038+0000","last_clean":"2024-05-11T08:06:01.647038+0000","last_became_active":"2024-05-11T08:05:59.403132+0000","last_became_peered":"2024-05-11T08:05:59.403132+0000","last_unstale":"2024-05-11T08:06:01.647038+0000","last_undegraded":"2024-05-11T08:06:01.647038+0000","last_fullsized":"2024-05-11T08:06:01.647038+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_clean_scrub_stamp":"2024-05-11T08:05:58.374552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:06:03.606038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.645399+0000","last_change":"2024-05-11T08:05:55.364367+0000","last_active":"2024-05-11T08:06:01.645399+0000","last_peered":"2024-05-11T08:06:01.645399+0000","last_clean":"2024-05-11T08:06:01.645399+0000","last_became_active":"2024-05-11T08:05:55.364089+0000","last_became_peered":"2024-05-11T08:05:55.364089+0000","last_unstale":"2024-05-11T08:06:01.645399+0000","last_undegraded":"2024-05-11T08:06:01.645399+0000","last_fullsized":"2024-05-11T08:06:01.645399+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:38:18.066779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.645226+0000","last_change":"2024-05-11T08:05:55.369401+0000","last_active":"2024-05-11T08:06:01.645226+0000","last_peered":"2024-05-11T08:06:01.645226+0000","last_clean":"2024-05-11T08:06:01.645226+0000","last_became_active":"2024-05-11T08:05:55.369154+0000","last_became_peered":"2024-05-11T08:05:55.369154+0000","last_unstale":"2024-05-11T08:06:01.645226+0000","last_undegraded":"2024-05-11T08:06:01.645226+0000","last_fullsized":"2024-05-11T08:06:01.645226+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:46:51.750892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.389323+0000","last_change":"2024-05-11T08:05:55.375787+0000","last_active":"2024-05-11T08:06:00.389323+0000","last_peered":"2024-05-11T08:06:00.389323+0000","last_clean":"2024-05-11T08:06:00.389323+0000","last_became_active":"2024-05-11T08:05:55.374199+0000","last_became_peered":"2024-05-11T08:05:55.374199+0000","last_unstale":"2024-05-11T08:06:00.389323+0000","last_undegraded":"2024-05-11T08:06:00.389323+0000","last_fullsized":"2024-05-11T08:06:00.389323+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:38:02.393559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.392641+0000","last_change":"2024-05-11T08:05:55.369439+0000","last_active":"2024-05-11T08:06:00.392641+0000","last_peered":"2024-05-11T08:06:00.392641+0000","last_clean":"2024-05-11T08:06:00.392641+0000","last_became_active":"2024-05-11T08:05:55.369182+0000","last_became_peered":"2024-05-11T08:05:55.369182+0000","last_unstale":"2024-05-11T08:06:00.392641+0000","last_undegraded":"2024-05-11T08:06:00.392641+0000","last_fullsized":"2024-05-11T08:06:00.392641+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:35:39.553476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.647343+0000","last_change":"2024-05-11T08:05:55.373779+0000","last_active":"2024-05-11T08:06:01.647343+0000","last_peered":"2024-05-11T08:06:01.647343+0000","last_clean":"2024-05-11T08:06:01.647343+0000","last_became_active":"2024-05-11T08:05:55.373523+0000","last_became_peered":"2024-05-11T08:05:55.373523+0000","last_unstale":"2024-05-11T08:06:01.647343+0000","last_undegraded":"2024-05-11T08:06:01.647343+0000","last_fullsized":"2024-05-11T08:06:01.647343+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:35:39.553476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.645671+0000","last_change":"2024-05-11T08:05:55.367765+0000","last_active":"2024-05-11T08:06:01.645671+0000","last_peered":"2024-05-11T08:06:01.645671+0000","last_clean":"2024-05-11T08:06:01.645671+0000","last_became_active":"2024-05-11T08:05:55.367521+0000","last_became_peered":"2024-05-11T08:05:55.367521+0000","last_unstale":"2024-05-11T08:06:01.645671+0000","last_undegraded":"2024-05-11T08:06:01.645671+0000","last_fullsized":"2024-05-11T08:06:01.645671+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:35:39.553476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.646145+0000","last_change":"2024-05-11T08:05:55.373963+0000","last_active":"2024-05-11T08:06:01.646145+0000","last_peered":"2024-05-11T08:06:01.646145+0000","last_clean":"2024-05-11T08:06:01.646145+0000","last_became_active":"2024-05-11T08:05:55.373700+0000","last_became_peered":"2024-05-11T08:05:55.373700+0000","last_unstale":"2024-05-11T08:06:01.646145+0000","last_undegraded":"2024-05-11T08:06:01.646145+0000","last_fullsized":"2024-05-11T08:06:01.646145+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:35:03.601599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.646584+0000","last_change":"2024-05-11T08:05:55.374020+0000","last_active":"2024-05-11T08:06:01.646584+0000","last_peered":"2024-05-11T08:06:01.646584+0000","last_clean":"2024-05-11T08:06:01.646584+0000","last_became_active":"2024-05-11T08:05:55.373697+0000","last_became_peered":"2024-05-11T08:05:55.373697+0000","last_unstale":"2024-05-11T08:06:01.646584+0000","last_undegraded":"2024-05-11T08:06:01.646584+0000","last_fullsized":"2024-05-11T08:06:01.646584+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:02:23.158539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.389077+0000","last_change":"2024-05-11T08:05:55.371797+0000","last_active":"2024-05-11T08:06:00.389077+0000","last_peered":"2024-05-11T08:06:00.389077+0000","last_clean":"2024-05-11T08:06:00.389077+0000","last_became_active":"2024-05-11T08:05:55.371535+0000","last_became_peered":"2024-05-11T08:05:55.371535+0000","last_unstale":"2024-05-11T08:06:00.389077+0000","last_undegraded":"2024-05-11T08:06:00.389077+0000","last_fullsized":"2024-05-11T08:06:00.389077+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:16:46.423362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.647177+0000","last_change":"2024-05-11T08:05:55.373536+0000","last_active":"2024-05-11T08:06:01.647177+0000","last_peered":"2024-05-11T08:06:01.647177+0000","last_clean":"2024-05-11T08:06:01.647177+0000","last_became_active":"2024-05-11T08:05:55.373273+0000","last_became_peered":"2024-05-11T08:05:55.373273+0000","last_unstale":"2024-05-11T08:06:01.647177+0000","last_undegraded":"2024-05-11T08:06:01.647177+0000","last_fullsized":"2024-05-11T08:06:01.647177+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:35:03.601599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.389188+0000","last_change":"2024-05-11T08:05:55.382077+0000","last_active":"2024-05-11T08:06:00.389188+0000","last_peered":"2024-05-11T08:06:00.389188+0000","last_clean":"2024-05-11T08:06:00.389188+0000","last_became_active":"2024-05-11T08:05:55.381767+0000","last_became_peered":"2024-05-11T08:05:55.381767+0000","last_unstale":"2024-05-11T08:06:00.389188+0000","last_undegraded":"2024-05-11T08:06:00.389188+0000","last_fullsized":"2024-05-11T08:06:00.389188+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:56:59.948641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.392492+0000","last_change":"2024-05-11T08:05:55.374245+0000","last_active":"2024-05-11T08:06:00.392492+0000","last_peered":"2024-05-11T08:06:00.392492+0000","last_clean":"2024-05-11T08:06:00.392492+0000","last_became_active":"2024-05-11T08:05:55.373871+0000","last_became_peered":"2024-05-11T08:05:55.373871+0000","last_unstale":"2024-05-11T08:06:00.392492+0000","last_undegraded":"2024-05-11T08:06:00.392492+0000","last_fullsized":"2024-05-11T08:06:00.392492+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:32:48.010809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.645850+0000","last_change":"2024-05-11T08:05:55.376534+0000","last_active":"2024-05-11T08:06:01.645850+0000","last_peered":"2024-05-11T08:06:01.645850+0000","last_clean":"2024-05-11T08:06:01.645850+0000","last_became_active":"2024-05-11T08:05:55.376265+0000","last_became_peered":"2024-05-11T08:05:55.376265+0000","last_unstale":"2024-05-11T08:06:01.645850+0000","last_undegraded":"2024-05-11T08:06:01.645850+0000","last_fullsized":"2024-05-11T08:06:01.645850+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:43:37.948904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.388549+0000","last_change":"2024-05-11T08:05:55.381048+0000","last_active":"2024-05-11T08:06:00.388549+0000","last_peered":"2024-05-11T08:06:00.388549+0000","last_clean":"2024-05-11T08:06:00.388549+0000","last_became_active":"2024-05-11T08:05:55.380781+0000","last_became_peered":"2024-05-11T08:05:55.380781+0000","last_unstale":"2024-05-11T08:06:00.388549+0000","last_undegraded":"2024-05-11T08:06:00.388549+0000","last_fullsized":"2024-05-11T08:06:00.388549+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:28:24.740747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.645586+0000","last_change":"2024-05-11T08:05:55.364750+0000","last_active":"2024-05-11T08:06:01.645586+0000","last_peered":"2024-05-11T08:06:01.645586+0000","last_clean":"2024-05-11T08:06:01.645586+0000","last_became_active":"2024-05-11T08:05:55.364495+0000","last_became_peered":"2024-05-11T08:05:55.364495+0000","last_unstale":"2024-05-11T08:06:01.645586+0000","last_undegraded":"2024-05-11T08:06:01.645586+0000","last_fullsized":"2024-05-11T08:06:01.645586+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:43:46.282095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.389971+0000","last_change":"2024-05-11T08:05:55.375569+0000","last_active":"2024-05-11T08:06:00.389971+0000","last_peered":"2024-05-11T08:06:00.389971+0000","last_clean":"2024-05-11T08:06:00.389971+0000","last_became_active":"2024-05-11T08:05:55.375303+0000","last_became_peered":"2024-05-11T08:05:55.375303+0000","last_unstale":"2024-05-11T08:06:00.389971+0000","last_undegraded":"2024-05-11T08:06:00.389971+0000","last_fullsized":"2024-05-11T08:06:00.389971+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:41:38.931447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.389063+0000","last_change":"2024-05-11T08:05:55.377664+0000","last_active":"2024-05-11T08:06:00.389063+0000","last_peered":"2024-05-11T08:06:00.389063+0000","last_clean":"2024-05-11T08:06:00.389063+0000","last_became_active":"2024-05-11T08:05:55.377373+0000","last_became_peered":"2024-05-11T08:05:55.377373+0000","last_unstale":"2024-05-11T08:06:00.389063+0000","last_undegraded":"2024-05-11T08:06:00.389063+0000","last_fullsized":"2024-05-11T08:06:00.389063+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:35:03.601599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.645312+0000","last_change":"2024-05-11T08:05:55.373802+0000","last_active":"2024-05-11T08:06:01.645312+0000","last_peered":"2024-05-11T08:06:01.645312+0000","last_clean":"2024-05-11T08:06:01.645312+0000","last_became_active":"2024-05-11T08:05:55.373558+0000","last_became_peered":"2024-05-11T08:05:55.373558+0000","last_unstale":"2024-05-11T08:06:01.645312+0000","last_undegraded":"2024-05-11T08:06:01.645312+0000","last_fullsized":"2024-05-11T08:06:01.645312+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:43:37.948904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.392536+0000","last_change":"2024-05-11T08:05:55.366027+0000","last_active":"2024-05-11T08:06:00.392536+0000","last_peered":"2024-05-11T08:06:00.392536+0000","last_clean":"2024-05-11T08:06:00.392536+0000","last_became_active":"2024-05-11T08:05:55.365360+0000","last_became_peered":"2024-05-11T08:05:55.365360+0000","last_unstale":"2024-05-11T08:06:00.392536+0000","last_undegraded":"2024-05-11T08:06:00.392536+0000","last_fullsized":"2024-05-11T08:06:00.392536+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:20:43.832246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.646413+0000","last_change":"2024-05-11T08:05:55.369665+0000","last_active":"2024-05-11T08:06:01.646413+0000","last_peered":"2024-05-11T08:06:01.646413+0000","last_clean":"2024-05-11T08:06:01.646413+0000","last_became_active":"2024-05-11T08:05:55.369420+0000","last_became_peered":"2024-05-11T08:05:55.369420+0000","last_unstale":"2024-05-11T08:06:01.646413+0000","last_undegraded":"2024-05-11T08:06:01.646413+0000","last_fullsized":"2024-05-11T08:06:01.646413+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:43:37.948904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.823465+0000","last_change":"2024-05-11T08:05:55.377949+0000","last_active":"2024-05-11T08:06:00.823465+0000","last_peered":"2024-05-11T08:06:00.823465+0000","last_clean":"2024-05-11T08:06:00.823465+0000","last_became_active":"2024-05-11T08:05:55.377677+0000","last_became_peered":"2024-05-11T08:05:55.377677+0000","last_unstale":"2024-05-11T08:06:00.823465+0000","last_undegraded":"2024-05-11T08:06:00.823465+0000","last_fullsized":"2024-05-11T08:06:00.823465+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:50:18.627516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.645928+0000","last_change":"2024-05-11T08:05:55.368388+0000","last_active":"2024-05-11T08:06:01.645928+0000","last_peered":"2024-05-11T08:06:01.645928+0000","last_clean":"2024-05-11T08:06:01.645928+0000","last_became_active":"2024-05-11T08:05:55.368103+0000","last_became_peered":"2024-05-11T08:05:55.368103+0000","last_unstale":"2024-05-11T08:06:01.645928+0000","last_undegraded":"2024-05-11T08:06:01.645928+0000","last_fullsized":"2024-05-11T08:06:01.645928+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:32:48.010809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.646145+0000","last_change":"2024-05-11T08:05:55.368708+0000","last_active":"2024-05-11T08:06:01.646145+0000","last_peered":"2024-05-11T08:06:01.646145+0000","last_clean":"2024-05-11T08:06:01.646145+0000","last_became_active":"2024-05-11T08:05:55.368177+0000","last_became_peered":"2024-05-11T08:05:55.368177+0000","last_unstale":"2024-05-11T08:06:01.646145+0000","last_undegraded":"2024-05-11T08:06:01.646145+0000","last_fullsized":"2024-05-11T08:06:01.646145+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:02:23.158539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.388971+0000","last_change":"2024-05-11T08:05:55.383485+0000","last_active":"2024-05-11T08:06:00.388971+0000","last_peered":"2024-05-11T08:06:00.388971+0000","last_clean":"2024-05-11T08:06:00.388971+0000","last_became_active":"2024-05-11T08:05:55.383016+0000","last_became_peered":"2024-05-11T08:05:55.383016+0000","last_unstale":"2024-05-11T08:06:00.388971+0000","last_undegraded":"2024-05-11T08:06:00.388971+0000","last_fullsized":"2024-05-11T08:06:00.388971+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:51:26.849946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.645483+0000","last_change":"2024-05-11T08:05:55.370103+0000","last_active":"2024-05-11T08:06:01.645483+0000","last_peered":"2024-05-11T08:06:01.645483+0000","last_clean":"2024-05-11T08:06:01.645483+0000","last_became_active":"2024-05-11T08:05:55.369838+0000","last_became_peered":"2024-05-11T08:05:55.369838+0000","last_unstale":"2024-05-11T08:06:01.645483+0000","last_undegraded":"2024-05-11T08:06:01.645483+0000","last_fullsized":"2024-05-11T08:06:01.645483+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:20:43.832246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.388264+0000","last_change":"2024-05-11T08:05:55.379192+0000","last_active":"2024-05-11T08:06:00.388264+0000","last_peered":"2024-05-11T08:06:00.388264+0000","last_clean":"2024-05-11T08:06:00.388264+0000","last_became_active":"2024-05-11T08:05:55.378919+0000","last_became_peered":"2024-05-11T08:05:55.378919+0000","last_unstale":"2024-05-11T08:06:00.388264+0000","last_undegraded":"2024-05-11T08:06:00.388264+0000","last_fullsized":"2024-05-11T08:06:00.388264+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:22:00.631022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.389205+0000","last_change":"2024-05-11T08:05:55.379230+0000","last_active":"2024-05-11T08:06:00.389205+0000","last_peered":"2024-05-11T08:06:00.389205+0000","last_clean":"2024-05-11T08:06:00.389205+0000","last_became_active":"2024-05-11T08:05:55.378960+0000","last_became_peered":"2024-05-11T08:05:55.378960+0000","last_unstale":"2024-05-11T08:06:00.389205+0000","last_undegraded":"2024-05-11T08:06:00.389205+0000","last_fullsized":"2024-05-11T08:06:00.389205+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:22:00.631022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.388309+0000","last_change":"2024-05-11T08:05:55.378492+0000","last_active":"2024-05-11T08:06:00.388309+0000","last_peered":"2024-05-11T08:06:00.388309+0000","last_clean":"2024-05-11T08:06:00.388309+0000","last_became_active":"2024-05-11T08:05:55.378200+0000","last_became_peered":"2024-05-11T08:05:55.378200+0000","last_unstale":"2024-05-11T08:06:00.388309+0000","last_undegraded":"2024-05-11T08:06:00.388309+0000","last_fullsized":"2024-05-11T08:06:00.388309+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:44:43.387850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.389503+0000","last_change":"2024-05-11T08:05:55.379296+0000","last_active":"2024-05-11T08:06:00.389503+0000","last_peered":"2024-05-11T08:06:00.389503+0000","last_clean":"2024-05-11T08:06:00.389503+0000","last_became_active":"2024-05-11T08:05:55.379047+0000","last_became_peered":"2024-05-11T08:05:55.379047+0000","last_unstale":"2024-05-11T08:06:00.389503+0000","last_undegraded":"2024-05-11T08:06:00.389503+0000","last_fullsized":"2024-05-11T08:06:00.389503+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:44:43.387850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.392612+0000","last_change":"2024-05-11T08:05:55.360980+0000","last_active":"2024-05-11T08:06:00.392612+0000","last_peered":"2024-05-11T08:06:00.392612+0000","last_clean":"2024-05-11T08:06:00.392612+0000","last_became_active":"2024-05-11T08:05:55.360626+0000","last_became_peered":"2024-05-11T08:05:55.360626+0000","last_unstale":"2024-05-11T08:06:00.392612+0000","last_undegraded":"2024-05-11T08:06:00.392612+0000","last_fullsized":"2024-05-11T08:06:00.392612+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:02:23.158539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.646210+0000","last_change":"2024-05-11T08:05:55.363639+0000","last_active":"2024-05-11T08:06:01.646210+0000","last_peered":"2024-05-11T08:06:01.646210+0000","last_clean":"2024-05-11T08:06:01.646210+0000","last_became_active":"2024-05-11T08:05:55.363320+0000","last_became_peered":"2024-05-11T08:05:55.363320+0000","last_unstale":"2024-05-11T08:06:01.646210+0000","last_undegraded":"2024-05-11T08:06:01.646210+0000","last_fullsized":"2024-05-11T08:06:01.646210+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:02:23.158539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.388821+0000","last_change":"2024-05-11T08:05:55.380035+0000","last_active":"2024-05-11T08:06:00.388821+0000","last_peered":"2024-05-11T08:06:00.388821+0000","last_clean":"2024-05-11T08:06:00.388821+0000","last_became_active":"2024-05-11T08:05:55.378781+0000","last_became_peered":"2024-05-11T08:05:55.378781+0000","last_unstale":"2024-05-11T08:06:00.388821+0000","last_undegraded":"2024-05-11T08:06:00.388821+0000","last_fullsized":"2024-05-11T08:06:00.388821+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:45:24.551499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509450,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27884,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343956,"statfs":{"total":96636764160,"available":96608210944,"internally_reserved":0,"allocated":880640,"data_stored":701507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509450,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":294912,"data_stored":114399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509450,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":286720,"data_stored":110647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509450,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":294912,"data_stored":110683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509451,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":294912,"data_stored":114399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509451,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":290816,"data_stored":110717,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509450,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27892,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343948,"statfs":{"total":96636764160,"available":96608202752,"internally_reserved":0,"allocated":888832,"data_stored":701563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574859,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":282624,"data_stored":110557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-11T08:06:19.863 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-11T08:06:20.000 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.000+0000 7f22da9c8640 1 Processor -- start 2024-05-11T08:06:20.001 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.000+0000 7f22da9c8640 1 -- start start 2024-05-11T08:06:20.002 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.001+0000 7f22da9c8640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f22d40b5e90 0x7f22d415f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:20.002 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.001+0000 7f22da9c8640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22d40d2410 con 0x7f22d415fc80 2024-05-11T08:06:20.002 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.001+0000 7f22da9c8640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22d40d26f0 con 0x7f22d415f360 2024-05-11T08:06:20.002 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.001+0000 7f22da9c8640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f22d40d29d0 con 0x7f22d40b5e90 2024-05-11T08:06:20.002 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.001+0000 7f22d8f65640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f22d415fc80 0x7f22d40cd610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:60920/0 (socket says 172.21.15.167:60920) 2024-05-11T08:06:20.002 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.001+0000 7f22d8f65640 1 -- 172.21.15.167:0/1137331264 learned_addr learned my addr 172.21.15.167:0/1137331264 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T08:06:20.002 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.001+0000 7f22cbfff640 1 --2- 172.21.15.167:0/1137331264 >> v2:172.21.15.204:3300/0 conn(0x7f22d40b5e90 0x7f22d415f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:20.002 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.002+0000 7f22d37fe640 1 -- 172.21.15.167:0/1137331264 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 771640329 0 0) 0x7f22d40d2410 con 0x7f22d415fc80 2024-05-11T08:06:20.002 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.002+0000 7f22d37fe640 1 -- 172.21.15.167:0/1137331264 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f22c0003520 con 0x7f22d415fc80 2024-05-11T08:06:20.003 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.002+0000 7f22d37fe640 1 -- 172.21.15.167:0/1137331264 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1523477150 0 0) 0x7f22d40d26f0 con 0x7f22d415f360 2024-05-11T08:06:20.003 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.002+0000 7f22d37fe640 1 -- 172.21.15.167:0/1137331264 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f22d40d2410 con 0x7f22d415f360 2024-05-11T08:06:20.003 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.002+0000 7f22cbfff640 1 -- 172.21.15.167:0/1137331264 >> v1:172.21.15.204:6789/0 conn(0x7f22d415f360 legacy=0x7f22d40cfdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:20.003 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.002+0000 7f22d37fe640 1 -- 172.21.15.167:0/1137331264 <== mon.0 v1:172.21.15.167:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2450087005 0 0) 0x7f22c0003520 con 0x7f22d415fc80 2024-05-11T08:06:20.003 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.002+0000 7f22cbfff640 1 -- 172.21.15.167:0/1137331264 >> v1:172.21.15.167:6789/0 conn(0x7f22d415fc80 legacy=0x7f22d40cd610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:20.003 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.002+0000 7f22cbfff640 1 -- 172.21.15.167:0/1137331264 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22d40b7e20 con 0x7f22d40b5e90 2024-05-11T08:06:20.004 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.003+0000 7f22cbfff640 1 --2- 172.21.15.167:0/1137331264 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f22d40b5e90 0x7f22d415f740 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f22bc0097a0 tx=0x7f22bc02a130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a75304e3d9995fd server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:20.005 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.003+0000 7f22d37fe640 1 -- 172.21.15.167:0/1137331264 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22bc037070 con 0x7f22d40b5e90 2024-05-11T08:06:20.005 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.003+0000 7f22d37fe640 1 -- 172.21.15.167:0/1137331264 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22bc02abc0 con 0x7f22d40b5e90 2024-05-11T08:06:20.005 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.003+0000 7f22d37fe640 1 -- 172.21.15.167:0/1137331264 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22cc001a70 con 0x7f22d40b5e90 2024-05-11T08:06:20.005 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.003+0000 7f22da9c8640 1 -- 172.21.15.167:0/1137331264 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f22d40b5e90 msgr2=0x7f22d415f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:20.005 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.003+0000 7f22da9c8640 1 --2- 172.21.15.167:0/1137331264 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f22d40b5e90 0x7f22d415f740 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f22bc0097a0 tx=0x7f22bc02a130 comp rx=0 tx=0).stop 2024-05-11T08:06:20.005 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.003+0000 7f22da9c8640 1 -- 172.21.15.167:0/1137331264 shutdown_connections 2024-05-11T08:06:20.005 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.003+0000 7f22da9c8640 1 --2- 172.21.15.167:0/1137331264 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f22d40b5e90 0x7f22d415f740 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:20.005 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.003+0000 7f22da9c8640 1 -- 172.21.15.167:0/1137331264 >> 172.21.15.167:0/1137331264 conn(0x7f22d4072730 msgr2=0x7f22d40d2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:20.005 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.003+0000 7f22da9c8640 1 -- 172.21.15.167:0/1137331264 shutdown_connections 2024-05-11T08:06:20.005 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.003+0000 7f22da9c8640 1 -- 172.21.15.167:0/1137331264 wait complete. 2024-05-11T08:06:20.005 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.004+0000 7f22da9c8640 1 Processor -- start 2024-05-11T08:06:20.005 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.005+0000 7f22da9c8640 1 -- start start 2024-05-11T08:06:20.007 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.005+0000 7f22da9c8640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f22d40b5e90 0x7f22d41f5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:20.008 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.005+0000 7f22da9c8640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f22d415f360 0x7f22d4154690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:20.008 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.005+0000 7f22da9c8640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f22d415fc80 0x7f22d41f84e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:20.008 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.005+0000 7f22da9c8640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f22d40cdea0 con 0x7f22d415f360 2024-05-11T08:06:20.008 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.005+0000 7f22da9c8640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f22d40cdd20 con 0x7f22d40b5e90 2024-05-11T08:06:20.008 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.005+0000 7f22da9c8640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f22d40b8340 con 0x7f22d415fc80 2024-05-11T08:06:20.008 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.005+0000 7f22cbfff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f22d415f360 0x7f22d4154690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:20.008 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.005+0000 7f22cbfff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f22d415f360 0x7f22d4154690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:54564/0 (socket says 172.21.15.167:54564) 2024-05-11T08:06:20.008 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.005+0000 7f22cbfff640 1 -- 172.21.15.167:0/1382061339 learned_addr learned my addr 172.21.15.167:0/1382061339 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:20.008 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.005+0000 7f22d8f65640 1 --2- 172.21.15.167:0/1382061339 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f22d415fc80 0x7f22d41f84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:20.008 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.005+0000 7f22d3fff640 1 --2- 172.21.15.167:0/1382061339 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f22d40b5e90 0x7f22d41f5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:20.008 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.006+0000 7f22cbfff640 1 -- 172.21.15.167:0/1382061339 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f22d415fc80 msgr2=0x7f22d41f84e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:20.008 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.006+0000 7f22cbfff640 1 --2- 172.21.15.167:0/1382061339 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f22d415fc80 0x7f22d41f84e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:20.008 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.006+0000 7f22cbfff640 1 -- 172.21.15.167:0/1382061339 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f22d40b5e90 msgr2=0x7f22d41f5900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:20.008 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.006+0000 7f22cbfff640 1 --2- 172.21.15.167:0/1382061339 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f22d40b5e90 0x7f22d41f5900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:20.009 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.006+0000 7f22cbfff640 1 -- 172.21.15.167:0/1382061339 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22d41f5e40 con 0x7f22d415f360 2024-05-11T08:06:20.009 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.006+0000 7f22d3fff640 1 --2- 172.21.15.167:0/1382061339 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f22d40b5e90 0x7f22d41f5900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T08:06:20.009 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.006+0000 7f22cbfff640 1 --2- 172.21.15.167:0/1382061339 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f22d415f360 0x7f22d4154690 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f22bc0098c0 tx=0x7f22bc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:20.009 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.006+0000 7f22d17fa640 1 -- 172.21.15.167:0/1382061339 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22bc037050 con 0x7f22d415f360 2024-05-11T08:06:20.009 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.006+0000 7f22d17fa640 1 -- 172.21.15.167:0/1382061339 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22bc043070 con 0x7f22d415f360 2024-05-11T08:06:20.009 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.006+0000 7f22da9c8640 1 -- 172.21.15.167:0/1382061339 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22d41f8e50 con 0x7f22d415f360 2024-05-11T08:06:20.009 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.006+0000 7f22d17fa640 1 -- 172.21.15.167:0/1382061339 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22bc006e10 con 0x7f22d415f360 2024-05-11T08:06:20.009 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.006+0000 7f22da9c8640 1 -- 172.21.15.167:0/1382061339 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22d41f5100 con 0x7f22d415f360 2024-05-11T08:06:20.009 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.007+0000 7f22da9c8640 1 -- 172.21.15.167:0/1382061339 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22d4069590 con 0x7f22d415f360 2024-05-11T08:06:20.009 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.008+0000 7f22d17fa640 1 -- 172.21.15.167:0/1382061339 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f22bc03ca10 con 0x7f22d415f360 2024-05-11T08:06:20.010 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.008+0000 7f22d17fa640 1 --2- 172.21.15.167:0/1382061339 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f22b40762d0 0x7f22b4078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:20.010 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.008+0000 7f22d3fff640 1 --2- 172.21.15.167:0/1382061339 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f22b40762d0 0x7f22b4078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:20.010 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.008+0000 7f22d17fa640 1 -- 172.21.15.167:0/1382061339 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f22bc0b8280 con 0x7f22d415f360 2024-05-11T08:06:20.010 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.009+0000 7f22d3fff640 1 --2- 172.21.15.167:0/1382061339 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f22b40762d0 0x7f22b4078790 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f22c40017d0 tx=0x7f22c4009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:20.012 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.013+0000 7f22d17fa640 1 -- 172.21.15.167:0/1382061339 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f22d41f5e40 con 0x7f22d415f360 2024-05-11T08:06:20.189 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.189+0000 7f22da9c8640 1 -- 172.21.15.167:0/1382061339 --> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f22d4069120 con 0x7f22b40762d0 2024-05-11T08:06:20.197 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.197+0000 7f22d17fa640 1 -- 172.21.15.167:0/1382061339 <== mgr.4113 v2:172.21.15.167:6832/253569496 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332523 (secure 0 0 0) 0x7f22d4069120 con 0x7f22b40762d0 2024-05-11T08:06:20.200 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T08:06:20.202 INFO:teuthology.orchestra.run.smithi167.stderr:dumped all 2024-05-11T08:06:20.203 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.200+0000 7f22da9c8640 1 -- 172.21.15.167:0/1382061339 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f22b40762d0 msgr2=0x7f22b4078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:20.204 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.200+0000 7f22da9c8640 1 --2- 172.21.15.167:0/1382061339 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f22b40762d0 0x7f22b4078790 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f22c40017d0 tx=0x7f22c4009040 comp rx=0 tx=0).stop 2024-05-11T08:06:20.204 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.200+0000 7f22da9c8640 1 -- 172.21.15.167:0/1382061339 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f22d415f360 msgr2=0x7f22d4154690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:20.204 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.200+0000 7f22da9c8640 1 --2- 172.21.15.167:0/1382061339 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f22d415f360 0x7f22d4154690 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f22bc0098c0 tx=0x7f22bc0046c0 comp rx=0 tx=0).stop 2024-05-11T08:06:20.204 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.200+0000 7f22da9c8640 1 -- 172.21.15.167:0/1382061339 shutdown_connections 2024-05-11T08:06:20.204 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.201+0000 7f22da9c8640 1 --2- 172.21.15.167:0/1382061339 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f22b40762d0 0x7f22b4078790 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:20.204 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.201+0000 7f22da9c8640 1 --2- 172.21.15.167:0/1382061339 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f22d415fc80 0x7f22d41f84e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:20.204 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.201+0000 7f22da9c8640 1 --2- 172.21.15.167:0/1382061339 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f22d415f360 0x7f22d4154690 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:20.204 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.201+0000 7f22da9c8640 1 --2- 172.21.15.167:0/1382061339 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f22d40b5e90 0x7f22d41f5900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:20.204 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.201+0000 7f22da9c8640 1 -- 172.21.15.167:0/1382061339 >> 172.21.15.167:0/1382061339 conn(0x7f22d4072730 msgr2=0x7f22d4160090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:20.204 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.201+0000 7f22da9c8640 1 -- 172.21.15.167:0/1382061339 shutdown_connections 2024-05-11T08:06:20.204 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.201+0000 7f22da9c8640 1 -- 172.21.15.167:0/1382061339 wait complete. 2024-05-11T08:06:20.212 INFO:teuthology.orchestra.run.smithi167.stdout:{"pg_ready":true,"pg_map":{"version":40,"stamp":"2024-05-11T08:06:19.922560+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":149,"num_write_kb":1646,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":114,"ondisk_log_size":114,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219604,"kb_used_data":3432,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755116,"statfs":{"total":773094113280,"available":772869238784,"internally_reserved":0,"allocated":3514368,"data_stored":2074472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003074"},"pg_stats":[{"pgid":"2.1f","version":"22'4","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.392985+0000","last_change":"2024-05-11T08:05:54.368188+0000","last_active":"2024-05-11T08:06:00.392985+0000","last_peered":"2024-05-11T08:06:00.392985+0000","last_clean":"2024-05-11T08:06:00.392985+0000","last_became_active":"2024-05-11T08:05:54.367881+0000","last_became_peered":"2024-05-11T08:05:54.367881+0000","last_unstale":"2024-05-11T08:06:00.392985+0000","last_undegraded":"2024-05-11T08:06:00.392985+0000","last_fullsized":"2024-05-11T08:06:00.392985+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_clean_scrub_stamp":"2024-05-11T08:05:53.331701+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:22:53.142453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.645941+0000","last_change":"2024-05-11T08:05:55.367955+0000","last_active":"2024-05-11T08:06:01.645941+0000","last_peered":"2024-05-11T08:06:01.645941+0000","last_clean":"2024-05-11T08:06:01.645941+0000","last_became_active":"2024-05-11T08:05:55.366260+0000","last_became_peered":"2024-05-11T08:05:55.366260+0000","last_unstale":"2024-05-11T08:06:01.645941+0000","last_undegraded":"2024-05-11T08:06:01.645941+0000","last_fullsized":"2024-05-11T08:06:01.645941+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:44:43.387850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.647113+0000","last_change":"2024-05-11T08:05:59.558909+0000","last_active":"2024-05-11T08:06:01.647113+0000","last_peered":"2024-05-11T08:06:01.647113+0000","last_clean":"2024-05-11T08:06:01.647113+0000","last_became_active":"2024-05-11T08:05:59.558344+0000","last_became_peered":"2024-05-11T08:05:59.558344+0000","last_unstale":"2024-05-11T08:06:01.647113+0000","last_undegraded":"2024-05-11T08:06:01.647113+0000","last_fullsized":"2024-05-11T08:06:01.647113+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_clean_scrub_stamp":"2024-05-11T08:05:58.374552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:50:45.024927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"22'1","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.646170+0000","last_change":"2024-05-11T08:05:54.371497+0000","last_active":"2024-05-11T08:06:01.646170+0000","last_peered":"2024-05-11T08:06:01.646170+0000","last_clean":"2024-05-11T08:06:01.646170+0000","last_became_active":"2024-05-11T08:05:54.371241+0000","last_became_peered":"2024-05-11T08:05:54.371241+0000","last_unstale":"2024-05-11T08:06:01.646170+0000","last_undegraded":"2024-05-11T08:06:01.646170+0000","last_fullsized":"2024-05-11T08:06:01.646170+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_clean_scrub_stamp":"2024-05-11T08:05:53.331701+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:33:25.907598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.392845+0000","last_change":"2024-05-11T08:05:55.369937+0000","last_active":"2024-05-11T08:06:00.392845+0000","last_peered":"2024-05-11T08:06:00.392845+0000","last_clean":"2024-05-11T08:06:00.392845+0000","last_became_active":"2024-05-11T08:05:55.369476+0000","last_became_peered":"2024-05-11T08:05:55.369476+0000","last_unstale":"2024-05-11T08:06:00.392845+0000","last_undegraded":"2024-05-11T08:06:00.392845+0000","last_fullsized":"2024-05-11T08:06:00.392845+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:16:46.423362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.823013+0000","last_change":"2024-05-11T08:05:59.407301+0000","last_active":"2024-05-11T08:06:00.823013+0000","last_peered":"2024-05-11T08:06:00.823013+0000","last_clean":"2024-05-11T08:06:00.823013+0000","last_became_active":"2024-05-11T08:05:59.406627+0000","last_became_peered":"2024-05-11T08:05:59.406627+0000","last_unstale":"2024-05-11T08:06:00.823013+0000","last_undegraded":"2024-05-11T08:06:00.823013+0000","last_fullsized":"2024-05-11T08:06:00.823013+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_clean_scrub_stamp":"2024-05-11T08:05:58.374552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:41:42.964013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"22'2","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.388522+0000","last_change":"2024-05-11T08:05:54.367998+0000","last_active":"2024-05-11T08:06:00.388522+0000","last_peered":"2024-05-11T08:06:00.388522+0000","last_clean":"2024-05-11T08:06:00.388522+0000","last_became_active":"2024-05-11T08:05:54.367681+0000","last_became_peered":"2024-05-11T08:05:54.367681+0000","last_unstale":"2024-05-11T08:06:00.388522+0000","last_undegraded":"2024-05-11T08:06:00.388522+0000","last_fullsized":"2024-05-11T08:06:00.388522+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_clean_scrub_stamp":"2024-05-11T08:05:53.331701+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:50:17.617231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.389537+0000","last_change":"2024-05-11T08:05:55.379931+0000","last_active":"2024-05-11T08:06:00.389537+0000","last_peered":"2024-05-11T08:06:00.389537+0000","last_clean":"2024-05-11T08:06:00.389537+0000","last_became_active":"2024-05-11T08:05:55.379656+0000","last_became_peered":"2024-05-11T08:05:55.379656+0000","last_unstale":"2024-05-11T08:06:00.389537+0000","last_undegraded":"2024-05-11T08:06:00.389537+0000","last_fullsized":"2024-05-11T08:06:00.389537+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:14:09.302889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.823238+0000","last_change":"2024-05-11T08:05:59.404405+0000","last_active":"2024-05-11T08:06:00.823238+0000","last_peered":"2024-05-11T08:06:00.823238+0000","last_clean":"2024-05-11T08:06:00.823238+0000","last_became_active":"2024-05-11T08:05:59.403407+0000","last_became_peered":"2024-05-11T08:05:59.403407+0000","last_unstale":"2024-05-11T08:06:00.823238+0000","last_undegraded":"2024-05-11T08:06:00.823238+0000","last_fullsized":"2024-05-11T08:06:00.823238+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_clean_scrub_stamp":"2024-05-11T08:05:58.374552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:57:03.981207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"22'2","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.388963+0000","last_change":"2024-05-11T08:05:54.369560+0000","last_active":"2024-05-11T08:06:00.388963+0000","last_peered":"2024-05-11T08:06:00.388963+0000","last_clean":"2024-05-11T08:06:00.388963+0000","last_became_active":"2024-05-11T08:05:54.369150+0000","last_became_peered":"2024-05-11T08:05:54.369150+0000","last_unstale":"2024-05-11T08:06:00.388963+0000","last_undegraded":"2024-05-11T08:06:00.388963+0000","last_fullsized":"2024-05-11T08:06:00.388963+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_clean_scrub_stamp":"2024-05-11T08:05:53.331701+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:17:22.080661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.389718+0000","last_change":"2024-05-11T08:05:55.379813+0000","last_active":"2024-05-11T08:06:00.389718+0000","last_peered":"2024-05-11T08:06:00.389718+0000","last_clean":"2024-05-11T08:06:00.389718+0000","last_became_active":"2024-05-11T08:05:55.379575+0000","last_became_peered":"2024-05-11T08:05:55.379575+0000","last_unstale":"2024-05-11T08:06:00.389718+0000","last_undegraded":"2024-05-11T08:06:00.389718+0000","last_fullsized":"2024-05-11T08:06:00.389718+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:28:24.740747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.823228+0000","last_change":"2024-05-11T08:05:59.412477+0000","last_active":"2024-05-11T08:06:00.823228+0000","last_peered":"2024-05-11T08:06:00.823228+0000","last_clean":"2024-05-11T08:06:00.823228+0000","last_became_active":"2024-05-11T08:05:59.411969+0000","last_became_peered":"2024-05-11T08:05:59.411969+0000","last_unstale":"2024-05-11T08:06:00.823228+0000","last_undegraded":"2024-05-11T08:06:00.823228+0000","last_fullsized":"2024-05-11T08:06:00.823228+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_clean_scrub_stamp":"2024-05-11T08:05:58.374552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:38:06.426124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.645499+0000","last_change":"2024-05-11T08:05:55.376503+0000","last_active":"2024-05-11T08:06:01.645499+0000","last_peered":"2024-05-11T08:06:01.645499+0000","last_clean":"2024-05-11T08:06:01.645499+0000","last_became_active":"2024-05-11T08:05:55.376198+0000","last_became_peered":"2024-05-11T08:05:55.376198+0000","last_unstale":"2024-05-11T08:06:01.645499+0000","last_undegraded":"2024-05-11T08:06:01.645499+0000","last_fullsized":"2024-05-11T08:06:01.645499+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:20:43.832246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"22'2","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.388669+0000","last_change":"2024-05-11T08:05:54.370164+0000","last_active":"2024-05-11T08:06:00.388669+0000","last_peered":"2024-05-11T08:06:00.388669+0000","last_clean":"2024-05-11T08:06:00.388669+0000","last_became_active":"2024-05-11T08:05:54.369893+0000","last_became_peered":"2024-05-11T08:05:54.369893+0000","last_unstale":"2024-05-11T08:06:00.388669+0000","last_undegraded":"2024-05-11T08:06:00.388669+0000","last_fullsized":"2024-05-11T08:06:00.388669+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_clean_scrub_stamp":"2024-05-11T08:05:53.331701+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:53:10.612688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.823618+0000","last_change":"2024-05-11T08:05:59.409212+0000","last_active":"2024-05-11T08:06:00.823618+0000","last_peered":"2024-05-11T08:06:00.823618+0000","last_clean":"2024-05-11T08:06:00.823618+0000","last_became_active":"2024-05-11T08:05:59.408832+0000","last_became_peered":"2024-05-11T08:05:59.408832+0000","last_unstale":"2024-05-11T08:06:00.823618+0000","last_undegraded":"2024-05-11T08:06:00.823618+0000","last_fullsized":"2024-05-11T08:06:00.823618+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_clean_scrub_stamp":"2024-05-11T08:05:58.374552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:45:28.584065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.646456+0000","last_change":"2024-05-11T08:05:55.362770+0000","last_active":"2024-05-11T08:06:01.646456+0000","last_peered":"2024-05-11T08:06:01.646456+0000","last_clean":"2024-05-11T08:06:01.646456+0000","last_became_active":"2024-05-11T08:05:55.362481+0000","last_became_peered":"2024-05-11T08:05:55.362481+0000","last_unstale":"2024-05-11T08:06:01.646456+0000","last_undegraded":"2024-05-11T08:06:01.646456+0000","last_fullsized":"2024-05-11T08:06:01.646456+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:22:00.631022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.645921+0000","last_change":"2024-05-11T08:05:54.369496+0000","last_active":"2024-05-11T08:06:01.645921+0000","last_peered":"2024-05-11T08:06:01.645921+0000","last_clean":"2024-05-11T08:06:01.645921+0000","last_became_active":"2024-05-11T08:05:54.369246+0000","last_became_peered":"2024-05-11T08:05:54.369246+0000","last_unstale":"2024-05-11T08:06:01.645921+0000","last_undegraded":"2024-05-11T08:06:01.645921+0000","last_fullsized":"2024-05-11T08:06:01.645921+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_clean_scrub_stamp":"2024-05-11T08:05:53.331701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:30:23.111497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.823390+0000","last_change":"2024-05-11T08:05:59.403693+0000","last_active":"2024-05-11T08:06:00.823390+0000","last_peered":"2024-05-11T08:06:00.823390+0000","last_clean":"2024-05-11T08:06:00.823390+0000","last_became_active":"2024-05-11T08:05:59.403195+0000","last_became_peered":"2024-05-11T08:05:59.403195+0000","last_unstale":"2024-05-11T08:06:00.823390+0000","last_undegraded":"2024-05-11T08:06:00.823390+0000","last_fullsized":"2024-05-11T08:06:00.823390+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_clean_scrub_stamp":"2024-05-11T08:05:58.374552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:28:28.773313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.645822+0000","last_change":"2024-05-11T08:05:55.366582+0000","last_active":"2024-05-11T08:06:01.645822+0000","last_peered":"2024-05-11T08:06:01.645822+0000","last_clean":"2024-05-11T08:06:01.645822+0000","last_became_active":"2024-05-11T08:05:55.366309+0000","last_became_peered":"2024-05-11T08:05:55.366309+0000","last_unstale":"2024-05-11T08:06:01.645822+0000","last_undegraded":"2024-05-11T08:06:01.645822+0000","last_fullsized":"2024-05-11T08:06:01.645822+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:32:48.010809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.645408+0000","last_change":"2024-05-11T08:05:54.368032+0000","last_active":"2024-05-11T08:06:01.645408+0000","last_peered":"2024-05-11T08:06:01.645408+0000","last_clean":"2024-05-11T08:06:01.645408+0000","last_became_active":"2024-05-11T08:05:54.367739+0000","last_became_peered":"2024-05-11T08:05:54.367739+0000","last_unstale":"2024-05-11T08:06:01.645408+0000","last_undegraded":"2024-05-11T08:06:01.645408+0000","last_fullsized":"2024-05-11T08:06:01.645408+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_clean_scrub_stamp":"2024-05-11T08:05:53.331701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:01:58.573898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.646280+0000","last_change":"2024-05-11T08:05:59.560219+0000","last_active":"2024-05-11T08:06:01.646280+0000","last_peered":"2024-05-11T08:06:01.646280+0000","last_clean":"2024-05-11T08:06:01.646280+0000","last_became_active":"2024-05-11T08:05:59.559821+0000","last_became_peered":"2024-05-11T08:05:59.559821+0000","last_unstale":"2024-05-11T08:06:01.646280+0000","last_undegraded":"2024-05-11T08:06:01.646280+0000","last_fullsized":"2024-05-11T08:06:01.646280+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_clean_scrub_stamp":"2024-05-11T08:05:58.374552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:51:30.882512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.823321+0000","last_change":"2024-05-11T08:05:55.370564+0000","last_active":"2024-05-11T08:06:00.823321+0000","last_peered":"2024-05-11T08:06:00.823321+0000","last_clean":"2024-05-11T08:06:00.823321+0000","last_became_active":"2024-05-11T08:05:55.370277+0000","last_became_peered":"2024-05-11T08:05:55.370277+0000","last_unstale":"2024-05-11T08:06:00.823321+0000","last_undegraded":"2024-05-11T08:06:00.823321+0000","last_fullsized":"2024-05-11T08:06:00.823321+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:30:24.121782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.387998+0000","last_change":"2024-05-11T08:05:54.371153+0000","last_active":"2024-05-11T08:06:00.387998+0000","last_peered":"2024-05-11T08:06:00.387998+0000","last_clean":"2024-05-11T08:06:00.387998+0000","last_became_active":"2024-05-11T08:05:54.369944+0000","last_became_peered":"2024-05-11T08:05:54.369944+0000","last_unstale":"2024-05-11T08:06:00.387998+0000","last_undegraded":"2024-05-11T08:06:00.387998+0000","last_fullsized":"2024-05-11T08:06:00.387998+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_clean_scrub_stamp":"2024-05-11T08:05:53.331701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:30:23.111497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.823584+0000","last_change":"2024-05-11T08:05:59.403650+0000","last_active":"2024-05-11T08:06:00.823584+0000","last_peered":"2024-05-11T08:06:00.823584+0000","last_clean":"2024-05-11T08:06:00.823584+0000","last_became_active":"2024-05-11T08:05:59.403181+0000","last_became_peered":"2024-05-11T08:05:59.403181+0000","last_unstale":"2024-05-11T08:06:00.823584+0000","last_undegraded":"2024-05-11T08:06:00.823584+0000","last_fullsized":"2024-05-11T08:06:00.823584+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_clean_scrub_stamp":"2024-05-11T08:05:58.374552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:51:30.882512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.392942+0000","last_change":"2024-05-11T08:05:55.365766+0000","last_active":"2024-05-11T08:06:00.392942+0000","last_peered":"2024-05-11T08:06:00.392942+0000","last_clean":"2024-05-11T08:06:00.392942+0000","last_became_active":"2024-05-11T08:05:55.365513+0000","last_became_peered":"2024-05-11T08:05:55.365513+0000","last_unstale":"2024-05-11T08:06:00.392942+0000","last_undegraded":"2024-05-11T08:06:00.392942+0000","last_fullsized":"2024-05-11T08:06:00.392942+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:43:37.948904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"22'2","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.646133+0000","last_change":"2024-05-11T08:05:54.368015+0000","last_active":"2024-05-11T08:06:01.646133+0000","last_peered":"2024-05-11T08:06:01.646133+0000","last_clean":"2024-05-11T08:06:01.646133+0000","last_became_active":"2024-05-11T08:05:54.367740+0000","last_became_peered":"2024-05-11T08:05:54.367740+0000","last_unstale":"2024-05-11T08:06:01.646133+0000","last_undegraded":"2024-05-11T08:06:01.646133+0000","last_fullsized":"2024-05-11T08:06:01.646133+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_clean_scrub_stamp":"2024-05-11T08:05:53.331701+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:30:23.111497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.823061+0000","last_change":"2024-05-11T08:05:59.559986+0000","last_active":"2024-05-11T08:06:00.823061+0000","last_peered":"2024-05-11T08:06:00.823061+0000","last_clean":"2024-05-11T08:06:00.823061+0000","last_became_active":"2024-05-11T08:05:59.559594+0000","last_became_peered":"2024-05-11T08:05:59.559594+0000","last_unstale":"2024-05-11T08:06:00.823061+0000","last_undegraded":"2024-05-11T08:06:00.823061+0000","last_fullsized":"2024-05-11T08:06:00.823061+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_clean_scrub_stamp":"2024-05-11T08:05:58.374552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:14:13.335454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.646670+0000","last_change":"2024-05-11T08:05:55.374876+0000","last_active":"2024-05-11T08:06:01.646670+0000","last_peered":"2024-05-11T08:06:01.646670+0000","last_clean":"2024-05-11T08:06:01.646670+0000","last_became_active":"2024-05-11T08:05:55.374617+0000","last_became_peered":"2024-05-11T08:05:55.374617+0000","last_unstale":"2024-05-11T08:06:01.646670+0000","last_undegraded":"2024-05-11T08:06:01.646670+0000","last_fullsized":"2024-05-11T08:06:01.646670+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:41:38.931447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"23'1","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.646291+0000","last_change":"2024-05-11T08:05:54.365008+0000","last_active":"2024-05-11T08:06:01.646291+0000","last_peered":"2024-05-11T08:06:01.646291+0000","last_clean":"2024-05-11T08:06:01.646291+0000","last_became_active":"2024-05-11T08:05:54.364698+0000","last_became_peered":"2024-05-11T08:05:54.364698+0000","last_unstale":"2024-05-11T08:06:01.646291+0000","last_undegraded":"2024-05-11T08:06:01.646291+0000","last_fullsized":"2024-05-11T08:06:01.646291+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_clean_scrub_stamp":"2024-05-11T08:05:53.331701+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:17:22.080661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.645075+0000","last_change":"2024-05-11T08:05:59.562139+0000","last_active":"2024-05-11T08:06:01.645075+0000","last_peered":"2024-05-11T08:06:01.645075+0000","last_clean":"2024-05-11T08:06:01.645075+0000","last_became_active":"2024-05-11T08:05:59.561081+0000","last_became_peered":"2024-05-11T08:05:59.561081+0000","last_unstale":"2024-05-11T08:06:01.645075+0000","last_undegraded":"2024-05-11T08:06:01.645075+0000","last_fullsized":"2024-05-11T08:06:01.645075+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_clean_scrub_stamp":"2024-05-11T08:05:58.374552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:41:33.826242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.645886+0000","last_change":"2024-05-11T08:05:55.371779+0000","last_active":"2024-05-11T08:06:01.645886+0000","last_peered":"2024-05-11T08:06:01.645886+0000","last_clean":"2024-05-11T08:06:01.645886+0000","last_became_active":"2024-05-11T08:05:55.368726+0000","last_became_peered":"2024-05-11T08:05:55.368726+0000","last_unstale":"2024-05-11T08:06:01.645886+0000","last_undegraded":"2024-05-11T08:06:01.645886+0000","last_fullsized":"2024-05-11T08:06:01.645886+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:32:48.010809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.388659+0000","last_change":"2024-05-11T08:05:54.370339+0000","last_active":"2024-05-11T08:06:00.388659+0000","last_peered":"2024-05-11T08:06:00.388659+0000","last_clean":"2024-05-11T08:06:00.388659+0000","last_became_active":"2024-05-11T08:05:54.370026+0000","last_became_peered":"2024-05-11T08:05:54.370026+0000","last_unstale":"2024-05-11T08:06:00.388659+0000","last_undegraded":"2024-05-11T08:06:00.388659+0000","last_fullsized":"2024-05-11T08:06:00.388659+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_clean_scrub_stamp":"2024-05-11T08:05:53.331701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:32:43.591947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.392567+0000","last_change":"2024-05-11T08:05:59.558387+0000","last_active":"2024-05-11T08:06:00.392567+0000","last_peered":"2024-05-11T08:06:00.392567+0000","last_clean":"2024-05-11T08:06:00.392567+0000","last_became_active":"2024-05-11T08:05:59.557980+0000","last_became_peered":"2024-05-11T08:05:59.557980+0000","last_unstale":"2024-05-11T08:06:00.392567+0000","last_undegraded":"2024-05-11T08:06:00.392567+0000","last_fullsized":"2024-05-11T08:06:00.392567+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_clean_scrub_stamp":"2024-05-11T08:05:58.374552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:12:42.864479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.389813+0000","last_change":"2024-05-11T08:05:55.373737+0000","last_active":"2024-05-11T08:06:00.389813+0000","last_peered":"2024-05-11T08:06:00.389813+0000","last_clean":"2024-05-11T08:06:00.389813+0000","last_became_active":"2024-05-11T08:05:55.373459+0000","last_became_peered":"2024-05-11T08:05:55.373459+0000","last_unstale":"2024-05-11T08:06:00.389813+0000","last_undegraded":"2024-05-11T08:06:00.389813+0000","last_fullsized":"2024-05-11T08:06:00.389813+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:51:26.849946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.645985+0000","last_change":"2024-05-11T08:05:54.367679+0000","last_active":"2024-05-11T08:06:01.645985+0000","last_peered":"2024-05-11T08:06:01.645985+0000","last_clean":"2024-05-11T08:06:01.645985+0000","last_became_active":"2024-05-11T08:05:54.367393+0000","last_became_peered":"2024-05-11T08:05:54.367393+0000","last_unstale":"2024-05-11T08:06:01.645985+0000","last_undegraded":"2024-05-11T08:06:01.645985+0000","last_fullsized":"2024-05-11T08:06:01.645985+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_clean_scrub_stamp":"2024-05-11T08:05:53.331701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:50:17.617231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.645628+0000","last_change":"2024-05-11T08:05:59.413104+0000","last_active":"2024-05-11T08:06:01.645628+0000","last_peered":"2024-05-11T08:06:01.645628+0000","last_clean":"2024-05-11T08:06:01.645628+0000","last_became_active":"2024-05-11T08:05:59.412442+0000","last_became_peered":"2024-05-11T08:05:59.412442+0000","last_unstale":"2024-05-11T08:06:01.645628+0000","last_undegraded":"2024-05-11T08:06:01.645628+0000","last_fullsized":"2024-05-11T08:06:01.645628+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_clean_scrub_stamp":"2024-05-11T08:05:58.374552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:57:03.981207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.823506+0000","last_change":"2024-05-11T08:05:55.377393+0000","last_active":"2024-05-11T08:06:00.823506+0000","last_peered":"2024-05-11T08:06:00.823506+0000","last_clean":"2024-05-11T08:06:00.823506+0000","last_became_active":"2024-05-11T08:05:55.377129+0000","last_became_peered":"2024-05-11T08:05:55.377129+0000","last_unstale":"2024-05-11T08:06:00.823506+0000","last_undegraded":"2024-05-11T08:06:00.823506+0000","last_fullsized":"2024-05-11T08:06:00.823506+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:05:55.494912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.389461+0000","last_change":"2024-05-11T08:05:54.361751+0000","last_active":"2024-05-11T08:06:00.389461+0000","last_peered":"2024-05-11T08:06:00.389461+0000","last_clean":"2024-05-11T08:06:00.389461+0000","last_became_active":"2024-05-11T08:05:54.361484+0000","last_became_peered":"2024-05-11T08:05:54.361484+0000","last_unstale":"2024-05-11T08:06:00.389461+0000","last_undegraded":"2024-05-11T08:06:00.389461+0000","last_fullsized":"2024-05-11T08:06:00.389461+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_clean_scrub_stamp":"2024-05-11T08:05:53.331701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:53:10.612688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.645837+0000","last_change":"2024-05-11T08:05:59.409532+0000","last_active":"2024-05-11T08:06:01.645837+0000","last_peered":"2024-05-11T08:06:01.645837+0000","last_clean":"2024-05-11T08:06:01.645837+0000","last_became_active":"2024-05-11T08:05:59.409096+0000","last_became_peered":"2024-05-11T08:05:59.409096+0000","last_unstale":"2024-05-11T08:06:01.645837+0000","last_undegraded":"2024-05-11T08:06:01.645837+0000","last_fullsized":"2024-05-11T08:06:01.645837+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_clean_scrub_stamp":"2024-05-11T08:05:58.374552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:12:42.864479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.645313+0000","last_change":"2024-05-11T08:05:55.368600+0000","last_active":"2024-05-11T08:06:01.645313+0000","last_peered":"2024-05-11T08:06:01.645313+0000","last_clean":"2024-05-11T08:06:01.645313+0000","last_became_active":"2024-05-11T08:05:55.368322+0000","last_became_peered":"2024-05-11T08:05:55.368322+0000","last_unstale":"2024-05-11T08:06:01.645313+0000","last_undegraded":"2024-05-11T08:06:01.645313+0000","last_fullsized":"2024-05-11T08:06:01.645313+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:38:18.066779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.645325+0000","last_change":"2024-05-11T08:05:54.369698+0000","last_active":"2024-05-11T08:06:01.645325+0000","last_peered":"2024-05-11T08:06:01.645325+0000","last_clean":"2024-05-11T08:06:01.645325+0000","last_became_active":"2024-05-11T08:05:54.369387+0000","last_became_peered":"2024-05-11T08:05:54.369387+0000","last_unstale":"2024-05-11T08:06:01.645325+0000","last_undegraded":"2024-05-11T08:06:01.645325+0000","last_fullsized":"2024-05-11T08:06:01.645325+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_clean_scrub_stamp":"2024-05-11T08:05:53.331701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:14:18.956461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.646283+0000","last_change":"2024-05-11T08:05:59.413264+0000","last_active":"2024-05-11T08:06:01.646283+0000","last_peered":"2024-05-11T08:06:01.646283+0000","last_clean":"2024-05-11T08:06:01.646283+0000","last_became_active":"2024-05-11T08:05:59.412704+0000","last_became_peered":"2024-05-11T08:05:59.412704+0000","last_unstale":"2024-05-11T08:06:01.646283+0000","last_undegraded":"2024-05-11T08:06:01.646283+0000","last_fullsized":"2024-05-11T08:06:01.646283+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_clean_scrub_stamp":"2024-05-11T08:05:58.374552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:45:28.584065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.392303+0000","last_change":"2024-05-11T08:05:55.373518+0000","last_active":"2024-05-11T08:06:00.392303+0000","last_peered":"2024-05-11T08:06:00.392303+0000","last_clean":"2024-05-11T08:06:00.392303+0000","last_became_active":"2024-05-11T08:05:55.373067+0000","last_became_peered":"2024-05-11T08:05:55.373067+0000","last_unstale":"2024-05-11T08:06:00.392303+0000","last_undegraded":"2024-05-11T08:06:00.392303+0000","last_fullsized":"2024-05-11T08:06:00.392303+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:35:55.478239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.388011+0000","last_change":"2024-05-11T08:05:54.369563+0000","last_active":"2024-05-11T08:06:00.388011+0000","last_peered":"2024-05-11T08:06:00.388011+0000","last_clean":"2024-05-11T08:06:00.388011+0000","last_became_active":"2024-05-11T08:05:54.369257+0000","last_became_peered":"2024-05-11T08:05:54.369257+0000","last_unstale":"2024-05-11T08:06:00.388011+0000","last_undegraded":"2024-05-11T08:06:00.388011+0000","last_fullsized":"2024-05-11T08:06:00.388011+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_clean_scrub_stamp":"2024-05-11T08:05:53.331701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:05:54.484627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.392895+0000","last_change":"2024-05-11T08:05:59.560554+0000","last_active":"2024-05-11T08:06:00.392895+0000","last_peered":"2024-05-11T08:06:00.392895+0000","last_clean":"2024-05-11T08:06:00.392895+0000","last_became_active":"2024-05-11T08:05:59.560163+0000","last_became_peered":"2024-05-11T08:05:59.560163+0000","last_unstale":"2024-05-11T08:06:00.392895+0000","last_undegraded":"2024-05-11T08:06:00.392895+0000","last_fullsized":"2024-05-11T08:06:00.392895+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_clean_scrub_stamp":"2024-05-11T08:05:58.374552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:19:03.458410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.645474+0000","last_change":"2024-05-11T08:05:55.371707+0000","last_active":"2024-05-11T08:06:01.645474+0000","last_peered":"2024-05-11T08:06:01.645474+0000","last_clean":"2024-05-11T08:06:01.645474+0000","last_became_active":"2024-05-11T08:05:55.371443+0000","last_became_peered":"2024-05-11T08:05:55.371443+0000","last_unstale":"2024-05-11T08:06:01.645474+0000","last_undegraded":"2024-05-11T08:06:01.645474+0000","last_fullsized":"2024-05-11T08:06:01.645474+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:16:46.423362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.389333+0000","last_change":"2024-05-11T08:05:54.361406+0000","last_active":"2024-05-11T08:06:00.389333+0000","last_peered":"2024-05-11T08:06:00.389333+0000","last_clean":"2024-05-11T08:06:00.389333+0000","last_became_active":"2024-05-11T08:05:54.361065+0000","last_became_peered":"2024-05-11T08:05:54.361065+0000","last_unstale":"2024-05-11T08:06:00.389333+0000","last_undegraded":"2024-05-11T08:06:00.389333+0000","last_fullsized":"2024-05-11T08:06:00.389333+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_clean_scrub_stamp":"2024-05-11T08:05:53.331701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:33:25.907598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":118,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.388646+0000","last_change":"2024-05-11T08:05:45.772368+0000","last_active":"2024-05-11T08:06:00.388646+0000","last_peered":"2024-05-11T08:06:00.388646+0000","last_clean":"2024-05-11T08:06:00.388646+0000","last_became_active":"2024-05-11T08:05:45.771373+0000","last_became_peered":"2024-05-11T08:05:45.771373+0000","last_unstale":"2024-05-11T08:06:00.388646+0000","last_undegraded":"2024-05-11T08:06:00.388646+0000","last_fullsized":"2024-05-11T08:06:00.388646+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:44.616617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:44.616617+0000","last_clean_scrub_stamp":"2024-05-11T08:05:44.616617+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:02:08.078748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.645881+0000","last_change":"2024-05-11T08:05:59.558739+0000","last_active":"2024-05-11T08:06:01.645881+0000","last_peered":"2024-05-11T08:06:01.645881+0000","last_clean":"2024-05-11T08:06:01.645881+0000","last_became_active":"2024-05-11T08:05:59.558307+0000","last_became_peered":"2024-05-11T08:05:59.558307+0000","last_unstale":"2024-05-11T08:06:01.645881+0000","last_undegraded":"2024-05-11T08:06:01.645881+0000","last_fullsized":"2024-05-11T08:06:01.645881+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_clean_scrub_stamp":"2024-05-11T08:05:58.374552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:28:28.773313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.388334+0000","last_change":"2024-05-11T08:05:55.379286+0000","last_active":"2024-05-11T08:06:00.388334+0000","last_peered":"2024-05-11T08:06:00.388334+0000","last_clean":"2024-05-11T08:06:00.388334+0000","last_became_active":"2024-05-11T08:05:55.378804+0000","last_became_peered":"2024-05-11T08:05:55.378804+0000","last_unstale":"2024-05-11T08:06:00.388334+0000","last_undegraded":"2024-05-11T08:06:00.388334+0000","last_fullsized":"2024-05-11T08:06:00.388334+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:56:59.948641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.646359+0000","last_change":"2024-05-11T08:05:54.364902+0000","last_active":"2024-05-11T08:06:01.646359+0000","last_peered":"2024-05-11T08:06:01.646359+0000","last_clean":"2024-05-11T08:06:01.646359+0000","last_became_active":"2024-05-11T08:05:54.364619+0000","last_became_peered":"2024-05-11T08:05:54.364619+0000","last_unstale":"2024-05-11T08:06:01.646359+0000","last_undegraded":"2024-05-11T08:06:01.646359+0000","last_fullsized":"2024-05-11T08:06:01.646359+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_clean_scrub_stamp":"2024-05-11T08:05:53.331701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:10:56.295408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.645635+0000","last_change":"2024-05-11T08:05:59.558409+0000","last_active":"2024-05-11T08:06:01.645635+0000","last_peered":"2024-05-11T08:06:01.645635+0000","last_clean":"2024-05-11T08:06:01.645635+0000","last_became_active":"2024-05-11T08:05:59.557939+0000","last_became_peered":"2024-05-11T08:05:59.557939+0000","last_unstale":"2024-05-11T08:06:01.645635+0000","last_undegraded":"2024-05-11T08:06:01.645635+0000","last_fullsized":"2024-05-11T08:06:01.645635+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_clean_scrub_stamp":"2024-05-11T08:05:58.374552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:57:03.981207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.389620+0000","last_change":"2024-05-11T08:05:55.376614+0000","last_active":"2024-05-11T08:06:00.389620+0000","last_peered":"2024-05-11T08:06:00.389620+0000","last_clean":"2024-05-11T08:06:00.389620+0000","last_became_active":"2024-05-11T08:05:55.376350+0000","last_became_peered":"2024-05-11T08:05:55.376350+0000","last_unstale":"2024-05-11T08:06:00.389620+0000","last_undegraded":"2024-05-11T08:06:00.389620+0000","last_fullsized":"2024-05-11T08:06:00.389620+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:12:38.831913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"22'1","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.645871+0000","last_change":"2024-05-11T08:05:54.369427+0000","last_active":"2024-05-11T08:06:01.645871+0000","last_peered":"2024-05-11T08:06:01.645871+0000","last_clean":"2024-05-11T08:06:01.645871+0000","last_became_active":"2024-05-11T08:05:54.369181+0000","last_became_peered":"2024-05-11T08:05:54.369181+0000","last_unstale":"2024-05-11T08:06:01.645871+0000","last_undegraded":"2024-05-11T08:06:01.645871+0000","last_fullsized":"2024-05-11T08:06:01.645871+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_clean_scrub_stamp":"2024-05-11T08:05:53.331701+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:05:54.484627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.392744+0000","last_change":"2024-05-11T08:05:59.411982+0000","last_active":"2024-05-11T08:06:00.392744+0000","last_peered":"2024-05-11T08:06:00.392744+0000","last_clean":"2024-05-11T08:06:00.392744+0000","last_became_active":"2024-05-11T08:05:59.411561+0000","last_became_peered":"2024-05-11T08:05:59.411561+0000","last_unstale":"2024-05-11T08:06:00.392744+0000","last_undegraded":"2024-05-11T08:06:00.392744+0000","last_fullsized":"2024-05-11T08:06:00.392744+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_clean_scrub_stamp":"2024-05-11T08:05:58.374552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:45:28.584065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.389445+0000","last_change":"2024-05-11T08:05:55.386175+0000","last_active":"2024-05-11T08:06:00.389445+0000","last_peered":"2024-05-11T08:06:00.389445+0000","last_clean":"2024-05-11T08:06:00.389445+0000","last_became_active":"2024-05-11T08:05:55.385883+0000","last_became_peered":"2024-05-11T08:05:55.385883+0000","last_unstale":"2024-05-11T08:06:00.389445+0000","last_undegraded":"2024-05-11T08:06:00.389445+0000","last_fullsized":"2024-05-11T08:06:00.389445+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:43:20.023451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"22'2","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.645669+0000","last_change":"2024-05-11T08:05:54.369810+0000","last_active":"2024-05-11T08:06:01.645669+0000","last_peered":"2024-05-11T08:06:01.645669+0000","last_clean":"2024-05-11T08:06:01.645669+0000","last_became_active":"2024-05-11T08:05:54.369512+0000","last_became_peered":"2024-05-11T08:05:54.369512+0000","last_unstale":"2024-05-11T08:06:01.645669+0000","last_undegraded":"2024-05-11T08:06:01.645669+0000","last_fullsized":"2024-05-11T08:06:01.645669+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_clean_scrub_stamp":"2024-05-11T08:05:53.331701+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:39:04.019231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.388134+0000","last_change":"2024-05-11T08:05:59.562447+0000","last_active":"2024-05-11T08:06:00.388134+0000","last_peered":"2024-05-11T08:06:00.388134+0000","last_clean":"2024-05-11T08:06:00.388134+0000","last_became_active":"2024-05-11T08:05:59.562033+0000","last_became_peered":"2024-05-11T08:05:59.562033+0000","last_unstale":"2024-05-11T08:06:00.388134+0000","last_undegraded":"2024-05-11T08:06:00.388134+0000","last_fullsized":"2024-05-11T08:06:00.388134+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_clean_scrub_stamp":"2024-05-11T08:05:58.374552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:41:33.826242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.388740+0000","last_change":"2024-05-11T08:05:55.386091+0000","last_active":"2024-05-11T08:06:00.388740+0000","last_peered":"2024-05-11T08:06:00.388740+0000","last_clean":"2024-05-11T08:06:00.388740+0000","last_became_active":"2024-05-11T08:05:55.385751+0000","last_became_peered":"2024-05-11T08:05:55.385751+0000","last_unstale":"2024-05-11T08:06:00.388740+0000","last_undegraded":"2024-05-11T08:06:00.388740+0000","last_fullsized":"2024-05-11T08:06:00.388740+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:12:38.831913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"22'1","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.823802+0000","last_change":"2024-05-11T08:05:54.372054+0000","last_active":"2024-05-11T08:06:00.823802+0000","last_peered":"2024-05-11T08:06:00.823802+0000","last_clean":"2024-05-11T08:06:00.823802+0000","last_became_active":"2024-05-11T08:05:54.371721+0000","last_became_peered":"2024-05-11T08:05:54.371721+0000","last_unstale":"2024-05-11T08:06:00.823802+0000","last_undegraded":"2024-05-11T08:06:00.823802+0000","last_fullsized":"2024-05-11T08:06:00.823802+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_clean_scrub_stamp":"2024-05-11T08:05:53.331701+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:40:29.770055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.823074+0000","last_change":"2024-05-11T08:05:59.404048+0000","last_active":"2024-05-11T08:06:00.823074+0000","last_peered":"2024-05-11T08:06:00.823074+0000","last_clean":"2024-05-11T08:06:00.823074+0000","last_became_active":"2024-05-11T08:05:59.403010+0000","last_became_peered":"2024-05-11T08:05:59.403010+0000","last_unstale":"2024-05-11T08:06:00.823074+0000","last_undegraded":"2024-05-11T08:06:00.823074+0000","last_fullsized":"2024-05-11T08:06:00.823074+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_clean_scrub_stamp":"2024-05-11T08:05:58.374552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:44:47.420416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.388312+0000","last_change":"2024-05-11T08:05:55.382224+0000","last_active":"2024-05-11T08:06:00.388312+0000","last_peered":"2024-05-11T08:06:00.388312+0000","last_clean":"2024-05-11T08:06:00.388312+0000","last_became_active":"2024-05-11T08:05:55.381614+0000","last_became_peered":"2024-05-11T08:05:55.381614+0000","last_unstale":"2024-05-11T08:06:00.388312+0000","last_undegraded":"2024-05-11T08:06:00.388312+0000","last_fullsized":"2024-05-11T08:06:00.388312+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:41:38.931447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"22'8","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.645775+0000","last_change":"2024-05-11T08:05:54.365900+0000","last_active":"2024-05-11T08:06:01.645775+0000","last_peered":"2024-05-11T08:06:01.645775+0000","last_clean":"2024-05-11T08:06:01.645775+0000","last_became_active":"2024-05-11T08:05:54.365638+0000","last_became_peered":"2024-05-11T08:05:54.365638+0000","last_unstale":"2024-05-11T08:06:01.645775+0000","last_undegraded":"2024-05-11T08:06:01.645775+0000","last_fullsized":"2024-05-11T08:06:01.645775+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_clean_scrub_stamp":"2024-05-11T08:05:53.331701+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:23:10.014766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.392726+0000","last_change":"2024-05-11T08:05:59.412069+0000","last_active":"2024-05-11T08:06:00.392726+0000","last_peered":"2024-05-11T08:06:00.392726+0000","last_clean":"2024-05-11T08:06:00.392726+0000","last_became_active":"2024-05-11T08:05:59.411674+0000","last_became_peered":"2024-05-11T08:05:59.411674+0000","last_unstale":"2024-05-11T08:06:00.392726+0000","last_undegraded":"2024-05-11T08:06:00.392726+0000","last_fullsized":"2024-05-11T08:06:00.392726+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_clean_scrub_stamp":"2024-05-11T08:05:58.374552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:51:30.882512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.645506+0000","last_change":"2024-05-11T08:05:55.364883+0000","last_active":"2024-05-11T08:06:01.645506+0000","last_peered":"2024-05-11T08:06:01.645506+0000","last_clean":"2024-05-11T08:06:01.645506+0000","last_became_active":"2024-05-11T08:05:55.364622+0000","last_became_peered":"2024-05-11T08:05:55.364622+0000","last_unstale":"2024-05-11T08:06:01.645506+0000","last_undegraded":"2024-05-11T08:06:01.645506+0000","last_fullsized":"2024-05-11T08:06:01.645506+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:35:39.553476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.645631+0000","last_change":"2024-05-11T08:05:54.368776+0000","last_active":"2024-05-11T08:06:01.645631+0000","last_peered":"2024-05-11T08:06:01.645631+0000","last_clean":"2024-05-11T08:06:01.645631+0000","last_became_active":"2024-05-11T08:05:54.368514+0000","last_became_peered":"2024-05-11T08:05:54.368514+0000","last_unstale":"2024-05-11T08:06:01.645631+0000","last_undegraded":"2024-05-11T08:06:01.645631+0000","last_fullsized":"2024-05-11T08:06:01.645631+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_clean_scrub_stamp":"2024-05-11T08:05:53.331701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:01:58.573898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.646534+0000","last_change":"2024-05-11T08:05:59.561579+0000","last_active":"2024-05-11T08:06:01.646534+0000","last_peered":"2024-05-11T08:06:01.646534+0000","last_clean":"2024-05-11T08:06:01.646534+0000","last_became_active":"2024-05-11T08:05:59.561027+0000","last_became_peered":"2024-05-11T08:05:59.561027+0000","last_unstale":"2024-05-11T08:06:01.646534+0000","last_undegraded":"2024-05-11T08:06:01.646534+0000","last_fullsized":"2024-05-11T08:06:01.646534+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_clean_scrub_stamp":"2024-05-11T08:05:58.374552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:06:20.739397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.388603+0000","last_change":"2024-05-11T08:05:55.387188+0000","last_active":"2024-05-11T08:06:00.388603+0000","last_peered":"2024-05-11T08:06:00.388603+0000","last_clean":"2024-05-11T08:06:00.388603+0000","last_became_active":"2024-05-11T08:05:55.386935+0000","last_became_peered":"2024-05-11T08:05:55.386935+0000","last_unstale":"2024-05-11T08:06:00.388603+0000","last_undegraded":"2024-05-11T08:06:00.388603+0000","last_fullsized":"2024-05-11T08:06:00.388603+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:14:09.302889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.389390+0000","last_change":"2024-05-11T08:05:54.370172+0000","last_active":"2024-05-11T08:06:00.389390+0000","last_peered":"2024-05-11T08:06:00.389390+0000","last_clean":"2024-05-11T08:06:00.389390+0000","last_became_active":"2024-05-11T08:05:54.369635+0000","last_became_peered":"2024-05-11T08:05:54.369635+0000","last_unstale":"2024-05-11T08:06:00.389390+0000","last_undegraded":"2024-05-11T08:06:00.389390+0000","last_fullsized":"2024-05-11T08:06:00.389390+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_clean_scrub_stamp":"2024-05-11T08:05:53.331701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:17:22.080661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.647316+0000","last_change":"2024-05-11T08:05:59.559006+0000","last_active":"2024-05-11T08:06:01.647316+0000","last_peered":"2024-05-11T08:06:01.647316+0000","last_clean":"2024-05-11T08:06:01.647316+0000","last_became_active":"2024-05-11T08:05:59.558255+0000","last_became_peered":"2024-05-11T08:05:59.558255+0000","last_unstale":"2024-05-11T08:06:01.647316+0000","last_undegraded":"2024-05-11T08:06:01.647316+0000","last_fullsized":"2024-05-11T08:06:01.647316+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_clean_scrub_stamp":"2024-05-11T08:05:58.374552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:32:53.866284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.392560+0000","last_change":"2024-05-11T08:05:55.375477+0000","last_active":"2024-05-11T08:06:00.392560+0000","last_peered":"2024-05-11T08:06:00.392560+0000","last_clean":"2024-05-11T08:06:00.392560+0000","last_became_active":"2024-05-11T08:05:55.375215+0000","last_became_peered":"2024-05-11T08:05:55.375215+0000","last_unstale":"2024-05-11T08:06:00.392560+0000","last_undegraded":"2024-05-11T08:06:00.392560+0000","last_fullsized":"2024-05-11T08:06:00.392560+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:38:18.066779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"22'2","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.392809+0000","last_change":"2024-05-11T08:05:54.368097+0000","last_active":"2024-05-11T08:06:00.392809+0000","last_peered":"2024-05-11T08:06:00.392809+0000","last_clean":"2024-05-11T08:06:00.392809+0000","last_became_active":"2024-05-11T08:05:54.367812+0000","last_became_peered":"2024-05-11T08:05:54.367812+0000","last_unstale":"2024-05-11T08:06:00.392809+0000","last_undegraded":"2024-05-11T08:06:00.392809+0000","last_fullsized":"2024-05-11T08:06:00.392809+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_clean_scrub_stamp":"2024-05-11T08:05:53.331701+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:00:20.233006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.389793+0000","last_change":"2024-05-11T08:05:59.558753+0000","last_active":"2024-05-11T08:06:00.389793+0000","last_peered":"2024-05-11T08:06:00.389793+0000","last_clean":"2024-05-11T08:06:00.389793+0000","last_became_active":"2024-05-11T08:05:59.558140+0000","last_became_peered":"2024-05-11T08:05:59.558140+0000","last_unstale":"2024-05-11T08:06:00.389793+0000","last_undegraded":"2024-05-11T08:06:00.389793+0000","last_fullsized":"2024-05-11T08:06:00.389793+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_clean_scrub_stamp":"2024-05-11T08:05:58.374552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:54:52.804997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.388229+0000","last_change":"2024-05-11T08:05:55.379691+0000","last_active":"2024-05-11T08:06:00.388229+0000","last_peered":"2024-05-11T08:06:00.388229+0000","last_clean":"2024-05-11T08:06:00.388229+0000","last_became_active":"2024-05-11T08:05:55.378519+0000","last_became_peered":"2024-05-11T08:05:55.378519+0000","last_unstale":"2024-05-11T08:06:00.388229+0000","last_undegraded":"2024-05-11T08:06:00.388229+0000","last_fullsized":"2024-05-11T08:06:00.388229+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:38:02.393559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"27'9","reported_seq":31,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:04.814504+0000","last_change":"2024-05-11T08:05:54.371057+0000","last_active":"2024-05-11T08:06:04.814504+0000","last_peered":"2024-05-11T08:06:04.814504+0000","last_clean":"2024-05-11T08:06:04.814504+0000","last_became_active":"2024-05-11T08:05:54.370746+0000","last_became_peered":"2024-05-11T08:05:54.370746+0000","last_unstale":"2024-05-11T08:06:04.814504+0000","last_undegraded":"2024-05-11T08:06:04.814504+0000","last_fullsized":"2024-05-11T08:06:04.814504+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_clean_scrub_stamp":"2024-05-11T08:05:53.331701+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:17:22.080661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.645768+0000","last_change":"2024-05-11T08:05:59.414118+0000","last_active":"2024-05-11T08:06:01.645768+0000","last_peered":"2024-05-11T08:06:01.645768+0000","last_clean":"2024-05-11T08:06:01.645768+0000","last_became_active":"2024-05-11T08:05:59.413183+0000","last_became_peered":"2024-05-11T08:05:59.413183+0000","last_unstale":"2024-05-11T08:06:01.645768+0000","last_undegraded":"2024-05-11T08:06:01.645768+0000","last_fullsized":"2024-05-11T08:06:01.645768+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_clean_scrub_stamp":"2024-05-11T08:05:58.374552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:28:28.773313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.823505+0000","last_change":"2024-05-11T08:05:55.373641+0000","last_active":"2024-05-11T08:06:00.823505+0000","last_peered":"2024-05-11T08:06:00.823505+0000","last_clean":"2024-05-11T08:06:00.823505+0000","last_became_active":"2024-05-11T08:05:55.373369+0000","last_became_peered":"2024-05-11T08:05:55.373369+0000","last_unstale":"2024-05-11T08:06:00.823505+0000","last_undegraded":"2024-05-11T08:06:00.823505+0000","last_fullsized":"2024-05-11T08:06:00.823505+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:32:44.602233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"22'3","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.645781+0000","last_change":"2024-05-11T08:05:54.366529+0000","last_active":"2024-05-11T08:06:01.645781+0000","last_peered":"2024-05-11T08:06:01.645781+0000","last_clean":"2024-05-11T08:06:01.645781+0000","last_became_active":"2024-05-11T08:05:54.366215+0000","last_became_peered":"2024-05-11T08:05:54.366215+0000","last_unstale":"2024-05-11T08:06:01.645781+0000","last_undegraded":"2024-05-11T08:06:01.645781+0000","last_fullsized":"2024-05-11T08:06:01.645781+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_clean_scrub_stamp":"2024-05-11T08:05:53.331701+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:53:10.612688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.823463+0000","last_change":"2024-05-11T08:05:59.407003+0000","last_active":"2024-05-11T08:06:00.823463+0000","last_peered":"2024-05-11T08:06:00.823463+0000","last_clean":"2024-05-11T08:06:00.823463+0000","last_became_active":"2024-05-11T08:05:59.406437+0000","last_became_peered":"2024-05-11T08:05:59.406437+0000","last_unstale":"2024-05-11T08:06:00.823463+0000","last_undegraded":"2024-05-11T08:06:00.823463+0000","last_fullsized":"2024-05-11T08:06:00.823463+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_clean_scrub_stamp":"2024-05-11T08:05:58.374552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:12:42.864479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.392952+0000","last_change":"2024-05-11T08:05:55.370107+0000","last_active":"2024-05-11T08:06:00.392952+0000","last_peered":"2024-05-11T08:06:00.392952+0000","last_clean":"2024-05-11T08:06:00.392952+0000","last_became_active":"2024-05-11T08:05:55.369823+0000","last_became_peered":"2024-05-11T08:05:55.369823+0000","last_unstale":"2024-05-11T08:06:00.392952+0000","last_undegraded":"2024-05-11T08:06:00.392952+0000","last_fullsized":"2024-05-11T08:06:00.392952+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:22:00.631022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.389731+0000","last_change":"2024-05-11T08:05:54.363934+0000","last_active":"2024-05-11T08:06:00.389731+0000","last_peered":"2024-05-11T08:06:00.389731+0000","last_clean":"2024-05-11T08:06:00.389731+0000","last_became_active":"2024-05-11T08:05:54.363428+0000","last_became_peered":"2024-05-11T08:05:54.363428+0000","last_unstale":"2024-05-11T08:06:00.389731+0000","last_undegraded":"2024-05-11T08:06:00.389731+0000","last_fullsized":"2024-05-11T08:06:00.389731+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_clean_scrub_stamp":"2024-05-11T08:05:53.331701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:30:23.111497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.645683+0000","last_change":"2024-05-11T08:05:59.403691+0000","last_active":"2024-05-11T08:06:01.645683+0000","last_peered":"2024-05-11T08:06:01.645683+0000","last_clean":"2024-05-11T08:06:01.645683+0000","last_became_active":"2024-05-11T08:05:59.403311+0000","last_became_peered":"2024-05-11T08:05:59.403311+0000","last_unstale":"2024-05-11T08:06:01.645683+0000","last_undegraded":"2024-05-11T08:06:01.645683+0000","last_fullsized":"2024-05-11T08:06:01.645683+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_clean_scrub_stamp":"2024-05-11T08:05:58.374552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:54:52.804997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.389689+0000","last_change":"2024-05-11T08:05:55.374886+0000","last_active":"2024-05-11T08:06:00.389689+0000","last_peered":"2024-05-11T08:06:00.389689+0000","last_clean":"2024-05-11T08:06:00.389689+0000","last_became_active":"2024-05-11T08:05:55.374622+0000","last_became_peered":"2024-05-11T08:05:55.374622+0000","last_unstale":"2024-05-11T08:06:00.389689+0000","last_undegraded":"2024-05-11T08:06:00.389689+0000","last_fullsized":"2024-05-11T08:06:00.389689+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:43:37.948904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.646049+0000","last_change":"2024-05-11T08:05:54.364291+0000","last_active":"2024-05-11T08:06:01.646049+0000","last_peered":"2024-05-11T08:06:01.646049+0000","last_clean":"2024-05-11T08:06:01.646049+0000","last_became_active":"2024-05-11T08:05:54.363676+0000","last_became_peered":"2024-05-11T08:05:54.363676+0000","last_unstale":"2024-05-11T08:06:01.646049+0000","last_undegraded":"2024-05-11T08:06:01.646049+0000","last_fullsized":"2024-05-11T08:06:01.646049+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_clean_scrub_stamp":"2024-05-11T08:05:53.331701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:53:10.612688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.647107+0000","last_change":"2024-05-11T08:05:59.558985+0000","last_active":"2024-05-11T08:06:01.647107+0000","last_peered":"2024-05-11T08:06:01.647107+0000","last_clean":"2024-05-11T08:06:01.647107+0000","last_became_active":"2024-05-11T08:05:59.558561+0000","last_became_peered":"2024-05-11T08:05:59.558561+0000","last_unstale":"2024-05-11T08:06:01.647107+0000","last_undegraded":"2024-05-11T08:06:01.647107+0000","last_fullsized":"2024-05-11T08:06:01.647107+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_clean_scrub_stamp":"2024-05-11T08:05:58.374552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:54:53.957903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.392778+0000","last_change":"2024-05-11T08:05:59.561482+0000","last_active":"2024-05-11T08:06:00.392778+0000","last_peered":"2024-05-11T08:06:00.392778+0000","last_clean":"2024-05-11T08:06:00.392778+0000","last_became_active":"2024-05-11T08:05:59.560743+0000","last_became_peered":"2024-05-11T08:05:59.560743+0000","last_unstale":"2024-05-11T08:06:00.392778+0000","last_undegraded":"2024-05-11T08:06:00.392778+0000","last_fullsized":"2024-05-11T08:06:00.392778+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_clean_scrub_stamp":"2024-05-11T08:05:58.374552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:43:24.056017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.645261+0000","last_change":"2024-05-11T08:05:55.367915+0000","last_active":"2024-05-11T08:06:01.645261+0000","last_peered":"2024-05-11T08:06:01.645261+0000","last_clean":"2024-05-11T08:06:01.645261+0000","last_became_active":"2024-05-11T08:05:55.367652+0000","last_became_peered":"2024-05-11T08:05:55.367652+0000","last_unstale":"2024-05-11T08:06:01.645261+0000","last_undegraded":"2024-05-11T08:06:01.645261+0000","last_fullsized":"2024-05-11T08:06:01.645261+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:35:03.601599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.389207+0000","last_change":"2024-05-11T08:05:54.368060+0000","last_active":"2024-05-11T08:06:00.389207+0000","last_peered":"2024-05-11T08:06:00.389207+0000","last_clean":"2024-05-11T08:06:00.389207+0000","last_became_active":"2024-05-11T08:05:54.367770+0000","last_became_peered":"2024-05-11T08:05:54.367770+0000","last_unstale":"2024-05-11T08:06:00.389207+0000","last_undegraded":"2024-05-11T08:06:00.389207+0000","last_fullsized":"2024-05-11T08:06:00.389207+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_clean_scrub_stamp":"2024-05-11T08:05:53.331701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:01:58.573898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.646058+0000","last_change":"2024-05-11T08:05:59.412426+0000","last_active":"2024-05-11T08:06:01.646058+0000","last_peered":"2024-05-11T08:06:01.646058+0000","last_clean":"2024-05-11T08:06:01.646058+0000","last_became_active":"2024-05-11T08:05:59.412011+0000","last_became_peered":"2024-05-11T08:05:59.412011+0000","last_unstale":"2024-05-11T08:06:01.646058+0000","last_undegraded":"2024-05-11T08:06:01.646058+0000","last_fullsized":"2024-05-11T08:06:01.646058+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_clean_scrub_stamp":"2024-05-11T08:05:58.374552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:12:42.864479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.645734+0000","last_change":"2024-05-11T08:05:55.363149+0000","last_active":"2024-05-11T08:06:01.645734+0000","last_peered":"2024-05-11T08:06:01.645734+0000","last_clean":"2024-05-11T08:06:01.645734+0000","last_became_active":"2024-05-11T08:05:55.362892+0000","last_became_peered":"2024-05-11T08:05:55.362892+0000","last_unstale":"2024-05-11T08:06:01.645734+0000","last_undegraded":"2024-05-11T08:06:01.645734+0000","last_fullsized":"2024-05-11T08:06:01.645734+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:38:18.066779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.647209+0000","last_change":"2024-05-11T08:05:54.368847+0000","last_active":"2024-05-11T08:06:01.647209+0000","last_peered":"2024-05-11T08:06:01.647209+0000","last_clean":"2024-05-11T08:06:01.647209+0000","last_became_active":"2024-05-11T08:05:54.368529+0000","last_became_peered":"2024-05-11T08:05:54.368529+0000","last_unstale":"2024-05-11T08:06:01.647209+0000","last_undegraded":"2024-05-11T08:06:01.647209+0000","last_fullsized":"2024-05-11T08:06:01.647209+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_clean_scrub_stamp":"2024-05-11T08:05:53.331701+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:22:53.142453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"22'1","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.645710+0000","last_change":"2024-05-11T08:05:54.369435+0000","last_active":"2024-05-11T08:06:01.645710+0000","last_peered":"2024-05-11T08:06:01.645710+0000","last_clean":"2024-05-11T08:06:01.645710+0000","last_became_active":"2024-05-11T08:05:54.369163+0000","last_became_peered":"2024-05-11T08:05:54.369163+0000","last_unstale":"2024-05-11T08:06:01.645710+0000","last_undegraded":"2024-05-11T08:06:01.645710+0000","last_fullsized":"2024-05-11T08:06:01.645710+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_clean_scrub_stamp":"2024-05-11T08:05:53.331701+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:33:25.907598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.392757+0000","last_change":"2024-05-11T08:05:55.378586+0000","last_active":"2024-05-11T08:06:00.392757+0000","last_peered":"2024-05-11T08:06:00.392757+0000","last_clean":"2024-05-11T08:06:00.392757+0000","last_became_active":"2024-05-11T08:05:55.378323+0000","last_became_peered":"2024-05-11T08:05:55.378323+0000","last_unstale":"2024-05-11T08:06:00.392757+0000","last_undegraded":"2024-05-11T08:06:00.392757+0000","last_fullsized":"2024-05-11T08:06:00.392757+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:35:03.601599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.646291+0000","last_change":"2024-05-11T08:05:59.402315+0000","last_active":"2024-05-11T08:06:01.646291+0000","last_peered":"2024-05-11T08:06:01.646291+0000","last_clean":"2024-05-11T08:06:01.646291+0000","last_became_active":"2024-05-11T08:05:59.401886+0000","last_became_peered":"2024-05-11T08:05:59.401886+0000","last_unstale":"2024-05-11T08:06:01.646291+0000","last_undegraded":"2024-05-11T08:06:01.646291+0000","last_fullsized":"2024-05-11T08:06:01.646291+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_clean_scrub_stamp":"2024-05-11T08:05:58.374552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:33:42.953506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"22'1","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.646467+0000","last_change":"2024-05-11T08:05:54.370886+0000","last_active":"2024-05-11T08:06:01.646467+0000","last_peered":"2024-05-11T08:06:01.646467+0000","last_clean":"2024-05-11T08:06:01.646467+0000","last_became_active":"2024-05-11T08:05:54.370611+0000","last_became_peered":"2024-05-11T08:05:54.370611+0000","last_unstale":"2024-05-11T08:06:01.646467+0000","last_undegraded":"2024-05-11T08:06:01.646467+0000","last_fullsized":"2024-05-11T08:06:01.646467+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:53.331701+0000","last_clean_scrub_stamp":"2024-05-11T08:05:53.331701+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:36:34.523259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.823139+0000","last_change":"2024-05-11T08:05:55.373391+0000","last_active":"2024-05-11T08:06:00.823139+0000","last_peered":"2024-05-11T08:06:00.823139+0000","last_clean":"2024-05-11T08:06:00.823139+0000","last_became_active":"2024-05-11T08:05:55.373096+0000","last_became_peered":"2024-05-11T08:05:55.373096+0000","last_unstale":"2024-05-11T08:06:00.823139+0000","last_undegraded":"2024-05-11T08:06:00.823139+0000","last_fullsized":"2024-05-11T08:06:00.823139+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:53:11.622973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.647038+0000","last_change":"2024-05-11T08:05:59.403522+0000","last_active":"2024-05-11T08:06:01.647038+0000","last_peered":"2024-05-11T08:06:01.647038+0000","last_clean":"2024-05-11T08:06:01.647038+0000","last_became_active":"2024-05-11T08:05:59.403132+0000","last_became_peered":"2024-05-11T08:05:59.403132+0000","last_unstale":"2024-05-11T08:06:01.647038+0000","last_undegraded":"2024-05-11T08:06:01.647038+0000","last_fullsized":"2024-05-11T08:06:01.647038+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:58.374552+0000","last_clean_scrub_stamp":"2024-05-11T08:05:58.374552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:06:03.606038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.645399+0000","last_change":"2024-05-11T08:05:55.364367+0000","last_active":"2024-05-11T08:06:01.645399+0000","last_peered":"2024-05-11T08:06:01.645399+0000","last_clean":"2024-05-11T08:06:01.645399+0000","last_became_active":"2024-05-11T08:05:55.364089+0000","last_became_peered":"2024-05-11T08:05:55.364089+0000","last_unstale":"2024-05-11T08:06:01.645399+0000","last_undegraded":"2024-05-11T08:06:01.645399+0000","last_fullsized":"2024-05-11T08:06:01.645399+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:38:18.066779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.645226+0000","last_change":"2024-05-11T08:05:55.369401+0000","last_active":"2024-05-11T08:06:01.645226+0000","last_peered":"2024-05-11T08:06:01.645226+0000","last_clean":"2024-05-11T08:06:01.645226+0000","last_became_active":"2024-05-11T08:05:55.369154+0000","last_became_peered":"2024-05-11T08:05:55.369154+0000","last_unstale":"2024-05-11T08:06:01.645226+0000","last_undegraded":"2024-05-11T08:06:01.645226+0000","last_fullsized":"2024-05-11T08:06:01.645226+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:46:51.750892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.389323+0000","last_change":"2024-05-11T08:05:55.375787+0000","last_active":"2024-05-11T08:06:00.389323+0000","last_peered":"2024-05-11T08:06:00.389323+0000","last_clean":"2024-05-11T08:06:00.389323+0000","last_became_active":"2024-05-11T08:05:55.374199+0000","last_became_peered":"2024-05-11T08:05:55.374199+0000","last_unstale":"2024-05-11T08:06:00.389323+0000","last_undegraded":"2024-05-11T08:06:00.389323+0000","last_fullsized":"2024-05-11T08:06:00.389323+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:38:02.393559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.392641+0000","last_change":"2024-05-11T08:05:55.369439+0000","last_active":"2024-05-11T08:06:00.392641+0000","last_peered":"2024-05-11T08:06:00.392641+0000","last_clean":"2024-05-11T08:06:00.392641+0000","last_became_active":"2024-05-11T08:05:55.369182+0000","last_became_peered":"2024-05-11T08:05:55.369182+0000","last_unstale":"2024-05-11T08:06:00.392641+0000","last_undegraded":"2024-05-11T08:06:00.392641+0000","last_fullsized":"2024-05-11T08:06:00.392641+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:35:39.553476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.647343+0000","last_change":"2024-05-11T08:05:55.373779+0000","last_active":"2024-05-11T08:06:01.647343+0000","last_peered":"2024-05-11T08:06:01.647343+0000","last_clean":"2024-05-11T08:06:01.647343+0000","last_became_active":"2024-05-11T08:05:55.373523+0000","last_became_peered":"2024-05-11T08:05:55.373523+0000","last_unstale":"2024-05-11T08:06:01.647343+0000","last_undegraded":"2024-05-11T08:06:01.647343+0000","last_fullsized":"2024-05-11T08:06:01.647343+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:35:39.553476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.645671+0000","last_change":"2024-05-11T08:05:55.367765+0000","last_active":"2024-05-11T08:06:01.645671+0000","last_peered":"2024-05-11T08:06:01.645671+0000","last_clean":"2024-05-11T08:06:01.645671+0000","last_became_active":"2024-05-11T08:05:55.367521+0000","last_became_peered":"2024-05-11T08:05:55.367521+0000","last_unstale":"2024-05-11T08:06:01.645671+0000","last_undegraded":"2024-05-11T08:06:01.645671+0000","last_fullsized":"2024-05-11T08:06:01.645671+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:35:39.553476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.646145+0000","last_change":"2024-05-11T08:05:55.373963+0000","last_active":"2024-05-11T08:06:01.646145+0000","last_peered":"2024-05-11T08:06:01.646145+0000","last_clean":"2024-05-11T08:06:01.646145+0000","last_became_active":"2024-05-11T08:05:55.373700+0000","last_became_peered":"2024-05-11T08:05:55.373700+0000","last_unstale":"2024-05-11T08:06:01.646145+0000","last_undegraded":"2024-05-11T08:06:01.646145+0000","last_fullsized":"2024-05-11T08:06:01.646145+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:35:03.601599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.646584+0000","last_change":"2024-05-11T08:05:55.374020+0000","last_active":"2024-05-11T08:06:01.646584+0000","last_peered":"2024-05-11T08:06:01.646584+0000","last_clean":"2024-05-11T08:06:01.646584+0000","last_became_active":"2024-05-11T08:05:55.373697+0000","last_became_peered":"2024-05-11T08:05:55.373697+0000","last_unstale":"2024-05-11T08:06:01.646584+0000","last_undegraded":"2024-05-11T08:06:01.646584+0000","last_fullsized":"2024-05-11T08:06:01.646584+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:02:23.158539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.389077+0000","last_change":"2024-05-11T08:05:55.371797+0000","last_active":"2024-05-11T08:06:00.389077+0000","last_peered":"2024-05-11T08:06:00.389077+0000","last_clean":"2024-05-11T08:06:00.389077+0000","last_became_active":"2024-05-11T08:05:55.371535+0000","last_became_peered":"2024-05-11T08:05:55.371535+0000","last_unstale":"2024-05-11T08:06:00.389077+0000","last_undegraded":"2024-05-11T08:06:00.389077+0000","last_fullsized":"2024-05-11T08:06:00.389077+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:16:46.423362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.647177+0000","last_change":"2024-05-11T08:05:55.373536+0000","last_active":"2024-05-11T08:06:01.647177+0000","last_peered":"2024-05-11T08:06:01.647177+0000","last_clean":"2024-05-11T08:06:01.647177+0000","last_became_active":"2024-05-11T08:05:55.373273+0000","last_became_peered":"2024-05-11T08:05:55.373273+0000","last_unstale":"2024-05-11T08:06:01.647177+0000","last_undegraded":"2024-05-11T08:06:01.647177+0000","last_fullsized":"2024-05-11T08:06:01.647177+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:35:03.601599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.389188+0000","last_change":"2024-05-11T08:05:55.382077+0000","last_active":"2024-05-11T08:06:00.389188+0000","last_peered":"2024-05-11T08:06:00.389188+0000","last_clean":"2024-05-11T08:06:00.389188+0000","last_became_active":"2024-05-11T08:05:55.381767+0000","last_became_peered":"2024-05-11T08:05:55.381767+0000","last_unstale":"2024-05-11T08:06:00.389188+0000","last_undegraded":"2024-05-11T08:06:00.389188+0000","last_fullsized":"2024-05-11T08:06:00.389188+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:56:59.948641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.392492+0000","last_change":"2024-05-11T08:05:55.374245+0000","last_active":"2024-05-11T08:06:00.392492+0000","last_peered":"2024-05-11T08:06:00.392492+0000","last_clean":"2024-05-11T08:06:00.392492+0000","last_became_active":"2024-05-11T08:05:55.373871+0000","last_became_peered":"2024-05-11T08:05:55.373871+0000","last_unstale":"2024-05-11T08:06:00.392492+0000","last_undegraded":"2024-05-11T08:06:00.392492+0000","last_fullsized":"2024-05-11T08:06:00.392492+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:32:48.010809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.645850+0000","last_change":"2024-05-11T08:05:55.376534+0000","last_active":"2024-05-11T08:06:01.645850+0000","last_peered":"2024-05-11T08:06:01.645850+0000","last_clean":"2024-05-11T08:06:01.645850+0000","last_became_active":"2024-05-11T08:05:55.376265+0000","last_became_peered":"2024-05-11T08:05:55.376265+0000","last_unstale":"2024-05-11T08:06:01.645850+0000","last_undegraded":"2024-05-11T08:06:01.645850+0000","last_fullsized":"2024-05-11T08:06:01.645850+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:43:37.948904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.388549+0000","last_change":"2024-05-11T08:05:55.381048+0000","last_active":"2024-05-11T08:06:00.388549+0000","last_peered":"2024-05-11T08:06:00.388549+0000","last_clean":"2024-05-11T08:06:00.388549+0000","last_became_active":"2024-05-11T08:05:55.380781+0000","last_became_peered":"2024-05-11T08:05:55.380781+0000","last_unstale":"2024-05-11T08:06:00.388549+0000","last_undegraded":"2024-05-11T08:06:00.388549+0000","last_fullsized":"2024-05-11T08:06:00.388549+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:28:24.740747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.645586+0000","last_change":"2024-05-11T08:05:55.364750+0000","last_active":"2024-05-11T08:06:01.645586+0000","last_peered":"2024-05-11T08:06:01.645586+0000","last_clean":"2024-05-11T08:06:01.645586+0000","last_became_active":"2024-05-11T08:05:55.364495+0000","last_became_peered":"2024-05-11T08:05:55.364495+0000","last_unstale":"2024-05-11T08:06:01.645586+0000","last_undegraded":"2024-05-11T08:06:01.645586+0000","last_fullsized":"2024-05-11T08:06:01.645586+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:43:46.282095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.389971+0000","last_change":"2024-05-11T08:05:55.375569+0000","last_active":"2024-05-11T08:06:00.389971+0000","last_peered":"2024-05-11T08:06:00.389971+0000","last_clean":"2024-05-11T08:06:00.389971+0000","last_became_active":"2024-05-11T08:05:55.375303+0000","last_became_peered":"2024-05-11T08:05:55.375303+0000","last_unstale":"2024-05-11T08:06:00.389971+0000","last_undegraded":"2024-05-11T08:06:00.389971+0000","last_fullsized":"2024-05-11T08:06:00.389971+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:41:38.931447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.389063+0000","last_change":"2024-05-11T08:05:55.377664+0000","last_active":"2024-05-11T08:06:00.389063+0000","last_peered":"2024-05-11T08:06:00.389063+0000","last_clean":"2024-05-11T08:06:00.389063+0000","last_became_active":"2024-05-11T08:05:55.377373+0000","last_became_peered":"2024-05-11T08:05:55.377373+0000","last_unstale":"2024-05-11T08:06:00.389063+0000","last_undegraded":"2024-05-11T08:06:00.389063+0000","last_fullsized":"2024-05-11T08:06:00.389063+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:35:03.601599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.645312+0000","last_change":"2024-05-11T08:05:55.373802+0000","last_active":"2024-05-11T08:06:01.645312+0000","last_peered":"2024-05-11T08:06:01.645312+0000","last_clean":"2024-05-11T08:06:01.645312+0000","last_became_active":"2024-05-11T08:05:55.373558+0000","last_became_peered":"2024-05-11T08:05:55.373558+0000","last_unstale":"2024-05-11T08:06:01.645312+0000","last_undegraded":"2024-05-11T08:06:01.645312+0000","last_fullsized":"2024-05-11T08:06:01.645312+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:43:37.948904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.392536+0000","last_change":"2024-05-11T08:05:55.366027+0000","last_active":"2024-05-11T08:06:00.392536+0000","last_peered":"2024-05-11T08:06:00.392536+0000","last_clean":"2024-05-11T08:06:00.392536+0000","last_became_active":"2024-05-11T08:05:55.365360+0000","last_became_peered":"2024-05-11T08:05:55.365360+0000","last_unstale":"2024-05-11T08:06:00.392536+0000","last_undegraded":"2024-05-11T08:06:00.392536+0000","last_fullsized":"2024-05-11T08:06:00.392536+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:20:43.832246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.646413+0000","last_change":"2024-05-11T08:05:55.369665+0000","last_active":"2024-05-11T08:06:01.646413+0000","last_peered":"2024-05-11T08:06:01.646413+0000","last_clean":"2024-05-11T08:06:01.646413+0000","last_became_active":"2024-05-11T08:05:55.369420+0000","last_became_peered":"2024-05-11T08:05:55.369420+0000","last_unstale":"2024-05-11T08:06:01.646413+0000","last_undegraded":"2024-05-11T08:06:01.646413+0000","last_fullsized":"2024-05-11T08:06:01.646413+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:43:37.948904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.823465+0000","last_change":"2024-05-11T08:05:55.377949+0000","last_active":"2024-05-11T08:06:00.823465+0000","last_peered":"2024-05-11T08:06:00.823465+0000","last_clean":"2024-05-11T08:06:00.823465+0000","last_became_active":"2024-05-11T08:05:55.377677+0000","last_became_peered":"2024-05-11T08:05:55.377677+0000","last_unstale":"2024-05-11T08:06:00.823465+0000","last_undegraded":"2024-05-11T08:06:00.823465+0000","last_fullsized":"2024-05-11T08:06:00.823465+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:50:18.627516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.645928+0000","last_change":"2024-05-11T08:05:55.368388+0000","last_active":"2024-05-11T08:06:01.645928+0000","last_peered":"2024-05-11T08:06:01.645928+0000","last_clean":"2024-05-11T08:06:01.645928+0000","last_became_active":"2024-05-11T08:05:55.368103+0000","last_became_peered":"2024-05-11T08:05:55.368103+0000","last_unstale":"2024-05-11T08:06:01.645928+0000","last_undegraded":"2024-05-11T08:06:01.645928+0000","last_fullsized":"2024-05-11T08:06:01.645928+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:32:48.010809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.646145+0000","last_change":"2024-05-11T08:05:55.368708+0000","last_active":"2024-05-11T08:06:01.646145+0000","last_peered":"2024-05-11T08:06:01.646145+0000","last_clean":"2024-05-11T08:06:01.646145+0000","last_became_active":"2024-05-11T08:05:55.368177+0000","last_became_peered":"2024-05-11T08:05:55.368177+0000","last_unstale":"2024-05-11T08:06:01.646145+0000","last_undegraded":"2024-05-11T08:06:01.646145+0000","last_fullsized":"2024-05-11T08:06:01.646145+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:02:23.158539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.388971+0000","last_change":"2024-05-11T08:05:55.383485+0000","last_active":"2024-05-11T08:06:00.388971+0000","last_peered":"2024-05-11T08:06:00.388971+0000","last_clean":"2024-05-11T08:06:00.388971+0000","last_became_active":"2024-05-11T08:05:55.383016+0000","last_became_peered":"2024-05-11T08:05:55.383016+0000","last_unstale":"2024-05-11T08:06:00.388971+0000","last_undegraded":"2024-05-11T08:06:00.388971+0000","last_fullsized":"2024-05-11T08:06:00.388971+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:51:26.849946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.645483+0000","last_change":"2024-05-11T08:05:55.370103+0000","last_active":"2024-05-11T08:06:01.645483+0000","last_peered":"2024-05-11T08:06:01.645483+0000","last_clean":"2024-05-11T08:06:01.645483+0000","last_became_active":"2024-05-11T08:05:55.369838+0000","last_became_peered":"2024-05-11T08:05:55.369838+0000","last_unstale":"2024-05-11T08:06:01.645483+0000","last_undegraded":"2024-05-11T08:06:01.645483+0000","last_fullsized":"2024-05-11T08:06:01.645483+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:20:43.832246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.388264+0000","last_change":"2024-05-11T08:05:55.379192+0000","last_active":"2024-05-11T08:06:00.388264+0000","last_peered":"2024-05-11T08:06:00.388264+0000","last_clean":"2024-05-11T08:06:00.388264+0000","last_became_active":"2024-05-11T08:05:55.378919+0000","last_became_peered":"2024-05-11T08:05:55.378919+0000","last_unstale":"2024-05-11T08:06:00.388264+0000","last_undegraded":"2024-05-11T08:06:00.388264+0000","last_fullsized":"2024-05-11T08:06:00.388264+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:22:00.631022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.389205+0000","last_change":"2024-05-11T08:05:55.379230+0000","last_active":"2024-05-11T08:06:00.389205+0000","last_peered":"2024-05-11T08:06:00.389205+0000","last_clean":"2024-05-11T08:06:00.389205+0000","last_became_active":"2024-05-11T08:05:55.378960+0000","last_became_peered":"2024-05-11T08:05:55.378960+0000","last_unstale":"2024-05-11T08:06:00.389205+0000","last_undegraded":"2024-05-11T08:06:00.389205+0000","last_fullsized":"2024-05-11T08:06:00.389205+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:22:00.631022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.388309+0000","last_change":"2024-05-11T08:05:55.378492+0000","last_active":"2024-05-11T08:06:00.388309+0000","last_peered":"2024-05-11T08:06:00.388309+0000","last_clean":"2024-05-11T08:06:00.388309+0000","last_became_active":"2024-05-11T08:05:55.378200+0000","last_became_peered":"2024-05-11T08:05:55.378200+0000","last_unstale":"2024-05-11T08:06:00.388309+0000","last_undegraded":"2024-05-11T08:06:00.388309+0000","last_fullsized":"2024-05-11T08:06:00.388309+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:44:43.387850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.389503+0000","last_change":"2024-05-11T08:05:55.379296+0000","last_active":"2024-05-11T08:06:00.389503+0000","last_peered":"2024-05-11T08:06:00.389503+0000","last_clean":"2024-05-11T08:06:00.389503+0000","last_became_active":"2024-05-11T08:05:55.379047+0000","last_became_peered":"2024-05-11T08:05:55.379047+0000","last_unstale":"2024-05-11T08:06:00.389503+0000","last_undegraded":"2024-05-11T08:06:00.389503+0000","last_fullsized":"2024-05-11T08:06:00.389503+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:44:43.387850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.392612+0000","last_change":"2024-05-11T08:05:55.360980+0000","last_active":"2024-05-11T08:06:00.392612+0000","last_peered":"2024-05-11T08:06:00.392612+0000","last_clean":"2024-05-11T08:06:00.392612+0000","last_became_active":"2024-05-11T08:05:55.360626+0000","last_became_peered":"2024-05-11T08:05:55.360626+0000","last_unstale":"2024-05-11T08:06:00.392612+0000","last_undegraded":"2024-05-11T08:06:00.392612+0000","last_fullsized":"2024-05-11T08:06:00.392612+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:02:23.158539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:01.646210+0000","last_change":"2024-05-11T08:05:55.363639+0000","last_active":"2024-05-11T08:06:01.646210+0000","last_peered":"2024-05-11T08:06:01.646210+0000","last_clean":"2024-05-11T08:06:01.646210+0000","last_became_active":"2024-05-11T08:05:55.363320+0000","last_became_peered":"2024-05-11T08:05:55.363320+0000","last_unstale":"2024-05-11T08:06:01.646210+0000","last_undegraded":"2024-05-11T08:06:01.646210+0000","last_fullsized":"2024-05-11T08:06:01.646210+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:02:23.158539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-11T08:06:00.388821+0000","last_change":"2024-05-11T08:05:55.380035+0000","last_active":"2024-05-11T08:06:00.388821+0000","last_peered":"2024-05-11T08:06:00.388821+0000","last_clean":"2024-05-11T08:06:00.388821+0000","last_became_active":"2024-05-11T08:05:55.378781+0000","last_became_peered":"2024-05-11T08:05:55.378781+0000","last_unstale":"2024-05-11T08:06:00.388821+0000","last_undegraded":"2024-05-11T08:06:00.388821+0000","last_fullsized":"2024-05-11T08:06:00.388821+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:54.341986+0000","last_clean_scrub_stamp":"2024-05-11T08:05:54.341986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:45:24.551499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509451,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27884,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343956,"statfs":{"total":96636764160,"available":96608210944,"internally_reserved":0,"allocated":880640,"data_stored":701507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509451,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":294912,"data_stored":114399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509451,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":286720,"data_stored":110647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509451,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":294912,"data_stored":110683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509452,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":294912,"data_stored":114399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509452,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":290816,"data_stored":110717,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509451,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27892,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343948,"statfs":{"total":96636764160,"available":96608202752,"internally_reserved":0,"allocated":888832,"data_stored":701563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574859,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":282624,"data_stored":110557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-11T08:06:20.217 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-11T08:06:20.217 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-05-11T08:06:20.217 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-05-11T08:06:20.217 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-05-11T08:06:20.353 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.351+0000 7f5d02a45640 1 Processor -- start 2024-05-11T08:06:20.353 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.351+0000 7f5d02a45640 1 -- start start 2024-05-11T08:06:20.353 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.351+0000 7f5d02a45640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f5cfc070650 0x7f5cfc0cf770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:20.353 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.352+0000 7f5d02a45640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5cfc0cfcb0 0x7f5cfc165a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:20.353 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.352+0000 7f5d02a45640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5cfc145ca0 con 0x7f5cfc0cf390 2024-05-11T08:06:20.353 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.352+0000 7f5d02a45640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f5cfc0d1b80 con 0x7f5cfc070650 2024-05-11T08:06:20.353 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.352+0000 7f5d02a45640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f5cfc0b5960 con 0x7f5cfc0cfcb0 2024-05-11T08:06:20.354 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.352+0000 7f5cfb7fe640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f5cfc070650 0x7f5cfc0cf770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:20.354 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.352+0000 7f5cfb7fe640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f5cfc070650 0x7f5cfc0cf770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:54578/0 (socket says 172.21.15.167:54578) 2024-05-11T08:06:20.354 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.352+0000 7f5cfb7fe640 1 -- 172.21.15.167:0/2438490678 learned_addr learned my addr 172.21.15.167:0/2438490678 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:20.354 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.352+0000 7f5d00fe2640 1 --2- 172.21.15.167:0/2438490678 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5cfc0cfcb0 0x7f5cfc165a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:20.354 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.352+0000 7f5cfb7fe640 1 -- 172.21.15.167:0/2438490678 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5cfc0cfcb0 msgr2=0x7f5cfc165a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:20.354 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.352+0000 7f5cfb7fe640 1 --2- 172.21.15.167:0/2438490678 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5cfc0cfcb0 0x7f5cfc165a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:20.354 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.352+0000 7f5cfb7fe640 1 -- 172.21.15.167:0/2438490678 >> v1:172.21.15.204:6789/0 conn(0x7f5cfc0cf390 legacy=0x7f5cfc0d1470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:20.354 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.352+0000 7f5cfb7fe640 1 -- 172.21.15.167:0/2438490678 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cfc0b5c40 con 0x7f5cfc070650 2024-05-11T08:06:20.354 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.353+0000 7f5cfb7fe640 1 --2- 172.21.15.167:0/2438490678 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5cfc070650 0x7f5cfc0cf770 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5cf0009870 tx=0x7f5cf002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcc91b6cfb2e89f0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:20.354 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.353+0000 7f5cfaffd640 1 -- 172.21.15.167:0/2438490678 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cf0037070 con 0x7f5cfc070650 2024-05-11T08:06:20.354 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.353+0000 7f5cfaffd640 1 -- 172.21.15.167:0/2438490678 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5cf0032440 con 0x7f5cfc070650 2024-05-11T08:06:20.354 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.353+0000 7f5cfaffd640 1 -- 172.21.15.167:0/2438490678 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cf0032810 con 0x7f5cfc070650 2024-05-11T08:06:20.357 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.353+0000 7f5d02a45640 1 -- 172.21.15.167:0/2438490678 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5cfc070650 msgr2=0x7f5cfc0cf770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:20.357 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.353+0000 7f5d02a45640 1 --2- 172.21.15.167:0/2438490678 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5cfc070650 0x7f5cfc0cf770 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5cf0009870 tx=0x7f5cf002a1c0 comp rx=0 tx=0).stop 2024-05-11T08:06:20.358 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.354+0000 7f5d02a45640 1 -- 172.21.15.167:0/2438490678 shutdown_connections 2024-05-11T08:06:20.358 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.354+0000 7f5d02a45640 1 --2- 172.21.15.167:0/2438490678 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5cfc0cfcb0 0x7f5cfc165a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:20.358 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.354+0000 7f5d02a45640 1 --2- 172.21.15.167:0/2438490678 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5cfc070650 0x7f5cfc0cf770 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:20.358 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.354+0000 7f5d02a45640 1 -- 172.21.15.167:0/2438490678 >> 172.21.15.167:0/2438490678 conn(0x7f5cfc0d1e40 msgr2=0x7f5cfc0d2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:20.358 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.354+0000 7f5d02a45640 1 -- 172.21.15.167:0/2438490678 shutdown_connections 2024-05-11T08:06:20.358 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.354+0000 7f5d02a45640 1 -- 172.21.15.167:0/2438490678 wait complete. 2024-05-11T08:06:20.358 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.355+0000 7f5d02a45640 1 Processor -- start 2024-05-11T08:06:20.358 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.355+0000 7f5d02a45640 1 -- start start 2024-05-11T08:06:20.358 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.356+0000 7f5d02a45640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5cfc070650 0x7f5cfc1e3f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:20.358 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.356+0000 7f5d02a45640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5cfc0cf390 0x7f5cfc06d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:20.358 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.356+0000 7f5d02a45640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5cfc0cfcb0 0x7f5cfc15b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:20.358 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.356+0000 7f5d02a45640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5cfc0b64e0 con 0x7f5cfc0cf390 2024-05-11T08:06:20.358 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.356+0000 7f5d02a45640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5cfc0b6200 con 0x7f5cfc0cfcb0 2024-05-11T08:06:20.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.356+0000 7f5d02a45640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f5cfc0d0160 con 0x7f5cfc070650 2024-05-11T08:06:20.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.356+0000 7f5cfb7fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5cfc0cf390 0x7f5cfc06d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:20.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.356+0000 7f5cfb7fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5cfc0cf390 0x7f5cfc06d6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:54584/0 (socket says 172.21.15.167:54584) 2024-05-11T08:06:20.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.356+0000 7f5cfb7fe640 1 -- 172.21.15.167:0/2379662022 learned_addr learned my addr 172.21.15.167:0/2379662022 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:20.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.356+0000 7f5cfbfff640 1 --2- 172.21.15.167:0/2379662022 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5cfc070650 0x7f5cfc1e3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:20.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.356+0000 7f5d00fe2640 1 --2- 172.21.15.167:0/2379662022 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5cfc0cfcb0 0x7f5cfc15b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:20.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.356+0000 7f5cfb7fe640 1 -- 172.21.15.167:0/2379662022 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5cfc070650 msgr2=0x7f5cfc1e3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:20.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.356+0000 7f5cfb7fe640 1 --2- 172.21.15.167:0/2379662022 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5cfc070650 0x7f5cfc1e3f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:20.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.356+0000 7f5cfb7fe640 1 -- 172.21.15.167:0/2379662022 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5cfc0cfcb0 msgr2=0x7f5cfc15b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:20.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.356+0000 7f5cfb7fe640 1 --2- 172.21.15.167:0/2379662022 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5cfc0cfcb0 0x7f5cfc15b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:20.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.356+0000 7f5cfb7fe640 1 -- 172.21.15.167:0/2379662022 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cfc1e4780 con 0x7f5cfc0cf390 2024-05-11T08:06:20.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.356+0000 7f5d00fe2640 1 --2- 172.21.15.167:0/2379662022 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5cfc0cfcb0 0x7f5cfc15b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T08:06:20.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.356+0000 7f5cfb7fe640 1 --2- 172.21.15.167:0/2379662022 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5cfc0cf390 0x7f5cfc06d6e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f5cf0000c00 tx=0x7f5cf00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:20.360 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.356+0000 7f5cf8ff9640 1 -- 172.21.15.167:0/2379662022 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cf0037040 con 0x7f5cfc0cf390 2024-05-11T08:06:20.360 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.356+0000 7f5cf8ff9640 1 -- 172.21.15.167:0/2379662022 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5cf0042070 con 0x7f5cfc0cf390 2024-05-11T08:06:20.360 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.356+0000 7f5d02a45640 1 -- 172.21.15.167:0/2379662022 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cfc1e4a60 con 0x7f5cfc0cf390 2024-05-11T08:06:20.360 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.356+0000 7f5cf8ff9640 1 -- 172.21.15.167:0/2379662022 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cf0030880 con 0x7f5cfc0cf390 2024-05-11T08:06:20.360 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.356+0000 7f5d02a45640 1 -- 172.21.15.167:0/2379662022 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cfc1e5020 con 0x7f5cfc0cf390 2024-05-11T08:06:20.360 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.358+0000 7f5d02a45640 1 -- 172.21.15.167:0/2379662022 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5cfc069bc0 con 0x7f5cfc0cf390 2024-05-11T08:06:20.360 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.358+0000 7f5cf8ff9640 1 -- 172.21.15.167:0/2379662022 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f5cf002ad10 con 0x7f5cfc0cf390 2024-05-11T08:06:20.360 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.358+0000 7f5cf8ff9640 1 --2- 172.21.15.167:0/2379662022 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f5ce80762d0 0x7f5ce8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:20.361 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.359+0000 7f5cf8ff9640 1 -- 172.21.15.167:0/2379662022 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f5cf00b7d50 con 0x7f5cfc0cf390 2024-05-11T08:06:20.361 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.359+0000 7f5cfbfff640 1 --2- 172.21.15.167:0/2379662022 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f5ce80762d0 0x7f5ce8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:20.361 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.359+0000 7f5cfbfff640 1 --2- 172.21.15.167:0/2379662022 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f5ce80762d0 0x7f5ce8078790 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f5cec0017c0 tx=0x7f5cec009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:20.363 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.363+0000 7f5cf8ff9640 1 -- 172.21.15.167:0/2379662022 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f5cf00bb030 con 0x7f5cfc0cf390 2024-05-11T08:06:20.585 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.585+0000 7f5d02a45640 1 -- 172.21.15.167:0/2379662022 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f5cfc0af910 con 0x7f5cfc0cf390 2024-05-11T08:06:20.585 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.585+0000 7f5cf8ff9640 1 -- 172.21.15.167:0/2379662022 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f5cf0081400 con 0x7f5cfc0cf390 2024-05-11T08:06:20.585 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T08:06:20.585 INFO:teuthology.orchestra.run.smithi167.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-05-11T08:06:20.588 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.587+0000 7f5d02a45640 1 -- 172.21.15.167:0/2379662022 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f5ce80762d0 msgr2=0x7f5ce8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:20.588 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.587+0000 7f5d02a45640 1 --2- 172.21.15.167:0/2379662022 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f5ce80762d0 0x7f5ce8078790 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f5cec0017c0 tx=0x7f5cec009040 comp rx=0 tx=0).stop 2024-05-11T08:06:20.588 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.587+0000 7f5d02a45640 1 -- 172.21.15.167:0/2379662022 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5cfc0cf390 msgr2=0x7f5cfc06d6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:20.588 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.587+0000 7f5d02a45640 1 --2- 172.21.15.167:0/2379662022 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5cfc0cf390 0x7f5cfc06d6e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f5cf0000c00 tx=0x7f5cf00046c0 comp rx=0 tx=0).stop 2024-05-11T08:06:20.588 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.587+0000 7f5d02a45640 1 -- 172.21.15.167:0/2379662022 shutdown_connections 2024-05-11T08:06:20.588 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.587+0000 7f5d02a45640 1 --2- 172.21.15.167:0/2379662022 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f5ce80762d0 0x7f5ce8078790 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:20.588 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.587+0000 7f5d02a45640 1 --2- 172.21.15.167:0/2379662022 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5cfc0cfcb0 0x7f5cfc15b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:20.588 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.587+0000 7f5d02a45640 1 --2- 172.21.15.167:0/2379662022 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5cfc0cf390 0x7f5cfc06d6e0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:20.588 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.587+0000 7f5d02a45640 1 --2- 172.21.15.167:0/2379662022 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5cfc070650 0x7f5cfc1e3f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:20.588 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.587+0000 7f5d02a45640 1 -- 172.21.15.167:0/2379662022 >> 172.21.15.167:0/2379662022 conn(0x7f5cfc0d1e40 msgr2=0x7f5cfc0d2220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:20.589 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.588+0000 7f5d02a45640 1 -- 172.21.15.167:0/2379662022 shutdown_connections 2024-05-11T08:06:20.589 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.588+0000 7f5d02a45640 1 -- 172.21.15.167:0/2379662022 wait complete. 2024-05-11T08:06:20.599 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-05-11T08:06:20.599 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-11T08:06:20.700 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.700+0000 7fa430a69640 1 Processor -- start 2024-05-11T08:06:20.700 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.700+0000 7fa430a69640 1 -- start start 2024-05-11T08:06:20.704 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.701+0000 7fa430a69640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fa42c148900 0x7fa42c0710f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:20.704 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.701+0000 7fa430a69640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa42c0b4f50 con 0x7fa42c071630 2024-05-11T08:06:20.704 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.701+0000 7fa430a69640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa42c0b5230 con 0x7fa42c0d0c10 2024-05-11T08:06:20.704 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.701+0000 7fa430a69640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fa42c071a70 con 0x7fa42c148900 2024-05-11T08:06:20.704 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.701+0000 7fa429d9b640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7fa42c071630 0x7fa42c0d0500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:60922/0 (socket says 172.21.15.167:60922) 2024-05-11T08:06:20.704 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.701+0000 7fa429d9b640 1 -- 172.21.15.167:0/3705167454 learned_addr learned my addr 172.21.15.167:0/3705167454 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T08:06:20.704 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.701+0000 7fa42a59c640 1 --2- 172.21.15.167:0/3705167454 >> v2:172.21.15.204:3300/0 conn(0x7fa42c148900 0x7fa42c0710f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:20.704 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.701+0000 7fa42959a640 1 -- 172.21.15.167:0/3705167454 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1023900284 0 0) 0x7fa42c0b4f50 con 0x7fa42c071630 2024-05-11T08:06:20.704 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.701+0000 7fa42959a640 1 -- 172.21.15.167:0/3705167454 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa414003520 con 0x7fa42c071630 2024-05-11T08:06:20.704 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.701+0000 7fa42959a640 1 -- 172.21.15.167:0/3705167454 <== mon.0 v1:172.21.15.167:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1382794482 0 0) 0x7fa414003520 con 0x7fa42c071630 2024-05-11T08:06:20.705 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.701+0000 7fa42959a640 1 -- 172.21.15.167:0/3705167454 >> v2:172.21.15.204:3300/0 conn(0x7fa42c148900 msgr2=0x7fa42c0710f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:20.705 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.701+0000 7fa42959a640 1 --2- 172.21.15.167:0/3705167454 >> v2:172.21.15.204:3300/0 conn(0x7fa42c148900 0x7fa42c0710f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:20.705 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.701+0000 7fa42959a640 1 -- 172.21.15.167:0/3705167454 >> v1:172.21.15.204:6789/0 conn(0x7fa42c0d0c10 legacy=0x7fa42c160f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:20.705 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.701+0000 7fa42959a640 1 -- 172.21.15.167:0/3705167454 --> v1:172.21.15.167:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa42c0b5510 con 0x7fa42c071630 2024-05-11T08:06:20.705 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.701+0000 7fa42959a640 1 -- 172.21.15.167:0/3705167454 <== mon.0 v1:172.21.15.167:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3602277846 0 0) 0x7fa4200016f0 con 0x7fa42c071630 2024-05-11T08:06:20.705 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.701+0000 7fa42959a640 1 -- 172.21.15.167:0/3705167454 >> v1:172.21.15.167:6789/0 conn(0x7fa42c071630 legacy=0x7fa42c0d0500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:20.705 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.702+0000 7fa42959a640 1 --2- 172.21.15.167:0/3705167454 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa414003a00 0x7fa414003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:20.705 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.702+0000 7fa42959a640 1 --2- 172.21.15.167:0/3705167454 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa414004800 0x7fa414004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:20.705 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.702+0000 7fa42959a640 1 --2- 172.21.15.167:0/3705167454 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa4140051c0 0x7fa41400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:20.705 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.702+0000 7fa42959a640 1 -- 172.21.15.167:0/3705167454 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa42c071a70 con 0x7fa414004800 2024-05-11T08:06:20.705 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.702+0000 7fa42959a640 1 -- 172.21.15.167:0/3705167454 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa41400c060 con 0x7fa4140051c0 2024-05-11T08:06:20.705 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.702+0000 7fa42959a640 1 -- 172.21.15.167:0/3705167454 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fa41400c260 con 0x7fa414003a00 2024-05-11T08:06:20.705 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.702+0000 7fa429d9b640 1 --2- 172.21.15.167:0/3705167454 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa414004800 0x7fa414004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:20.705 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.702+0000 7fa42a59c640 1 --2- 172.21.15.167:0/3705167454 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa414003a00 0x7fa414003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:20.706 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.702+0000 7fa42ad9d640 1 --2- 172.21.15.167:0/3705167454 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa4140051c0 0x7fa41400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:20.706 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.702+0000 7fa429d9b640 1 -- 172.21.15.167:0/3705167454 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa414003a00 msgr2=0x7fa414003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:20.706 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.702+0000 7fa429d9b640 1 --2- 172.21.15.167:0/3705167454 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa414003a00 0x7fa414003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:20.706 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.702+0000 7fa429d9b640 1 -- 172.21.15.167:0/3705167454 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa4140051c0 msgr2=0x7fa41400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:20.706 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.702+0000 7fa429d9b640 1 --2- 172.21.15.167:0/3705167454 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa4140051c0 0x7fa41400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:20.706 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.702+0000 7fa42ad9d640 1 -- 172.21.15.167:0/3705167454 reap_dead start 2024-05-11T08:06:20.706 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.702+0000 7fa429d9b640 1 -- 172.21.15.167:0/3705167454 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa414003520 con 0x7fa414004800 2024-05-11T08:06:20.706 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.703+0000 7fa429d9b640 1 --2- 172.21.15.167:0/3705167454 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa414004800 0x7fa414004c80 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa42000c210 tx=0x7fa414005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb28c1b354ea9bbd server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:20.706 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.703+0000 7fa42959a640 1 -- 172.21.15.167:0/3705167454 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa420007d10 con 0x7fa414004800 2024-05-11T08:06:20.706 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.703+0000 7fa42959a640 1 -- 172.21.15.167:0/3705167454 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa420005ad0 con 0x7fa414004800 2024-05-11T08:06:20.706 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.703+0000 7fa42959a640 1 -- 172.21.15.167:0/3705167454 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa420007070 con 0x7fa414004800 2024-05-11T08:06:20.706 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.703+0000 7fa430a69640 1 -- 172.21.15.167:0/3705167454 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa414004800 msgr2=0x7fa414004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:20.706 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.703+0000 7fa430a69640 1 --2- 172.21.15.167:0/3705167454 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa414004800 0x7fa414004c80 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa42000c210 tx=0x7fa414005830 comp rx=0 tx=0).stop 2024-05-11T08:06:20.707 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.703+0000 7fa430a69640 1 -- 172.21.15.167:0/3705167454 shutdown_connections 2024-05-11T08:06:20.707 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.703+0000 7fa430a69640 1 --2- 172.21.15.167:0/3705167454 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa414004800 0x7fa414004c80 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:20.707 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.703+0000 7fa430a69640 1 -- 172.21.15.167:0/3705167454 >> 172.21.15.167:0/3705167454 conn(0x7fa42c0d1e40 msgr2=0x7fa42c0d2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:20.707 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.703+0000 7fa430a69640 1 -- 172.21.15.167:0/3705167454 shutdown_connections 2024-05-11T08:06:20.707 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.703+0000 7fa430a69640 1 -- 172.21.15.167:0/3705167454 wait complete. 2024-05-11T08:06:20.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.705+0000 7fa430a69640 1 Processor -- start 2024-05-11T08:06:20.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.705+0000 7fa430a69640 1 -- start start 2024-05-11T08:06:20.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.705+0000 7fa430a69640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa414004800 0x7fa42c144820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:20.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.705+0000 7fa430a69640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa42c14e420 0x7fa42c147070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:20.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.705+0000 7fa430a69640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa42c1475b0 0x7fa42c13f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:20.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.705+0000 7fa430a69640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa42c148ce0 con 0x7fa42c14e420 2024-05-11T08:06:20.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.705+0000 7fa430a69640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa42c0b5ad0 con 0x7fa42c1475b0 2024-05-11T08:06:20.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.705+0000 7fa430a69640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fa42c15f900 con 0x7fa414004800 2024-05-11T08:06:20.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.706+0000 7fa429d9b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa42c14e420 0x7fa42c147070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:20.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.706+0000 7fa429d9b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa42c14e420 0x7fa42c147070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:54596/0 (socket says 172.21.15.167:54596) 2024-05-11T08:06:20.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.706+0000 7fa429d9b640 1 -- 172.21.15.167:0/1763896642 learned_addr learned my addr 172.21.15.167:0/1763896642 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:20.711 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.706+0000 7fa42a59c640 1 --2- 172.21.15.167:0/1763896642 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa414004800 0x7fa42c144820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:20.711 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.706+0000 7fa42ad9d640 1 --2- 172.21.15.167:0/1763896642 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa42c1475b0 0x7fa42c13f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:20.711 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.706+0000 7fa429d9b640 1 -- 172.21.15.167:0/1763896642 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa414004800 msgr2=0x7fa42c144820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:20.711 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.706+0000 7fa429d9b640 1 --2- 172.21.15.167:0/1763896642 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa414004800 0x7fa42c144820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:20.711 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.706+0000 7fa429d9b640 1 -- 172.21.15.167:0/1763896642 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa42c1475b0 msgr2=0x7fa42c13f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:20.711 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.706+0000 7fa429d9b640 1 --2- 172.21.15.167:0/1763896642 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa42c1475b0 0x7fa42c13f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:20.711 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.706+0000 7fa429d9b640 1 -- 172.21.15.167:0/1763896642 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa42c13e990 con 0x7fa42c14e420 2024-05-11T08:06:20.711 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.706+0000 7fa429d9b640 1 --2- 172.21.15.167:0/1763896642 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa42c14e420 0x7fa42c147070 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fa42000c330 tx=0x7fa42c0b5400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:20.711 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.706+0000 7fa41affd640 1 -- 172.21.15.167:0/1763896642 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa42003f020 con 0x7fa42c14e420 2024-05-11T08:06:20.711 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.706+0000 7fa41affd640 1 -- 172.21.15.167:0/1763896642 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4200014b0 con 0x7fa42c14e420 2024-05-11T08:06:20.711 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.706+0000 7fa41affd640 1 -- 172.21.15.167:0/1763896642 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4200047e0 con 0x7fa42c14e420 2024-05-11T08:06:20.711 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.706+0000 7fa430a69640 1 -- 172.21.15.167:0/1763896642 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa42c147e50 con 0x7fa42c14e420 2024-05-11T08:06:20.711 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.706+0000 7fa430a69640 1 -- 172.21.15.167:0/1763896642 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa42c147a60 con 0x7fa42c14e420 2024-05-11T08:06:20.711 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.707+0000 7fa430a69640 1 -- 172.21.15.167:0/1763896642 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa42c069bc0 con 0x7fa42c14e420 2024-05-11T08:06:20.712 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.708+0000 7fa41affd640 1 -- 172.21.15.167:0/1763896642 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fa420042020 con 0x7fa42c14e420 2024-05-11T08:06:20.712 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.708+0000 7fa41affd640 1 --2- 172.21.15.167:0/1763896642 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fa40c076300 0x7fa40c0787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:20.712 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.709+0000 7fa42a59c640 1 --2- 172.21.15.167:0/1763896642 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fa40c076300 0x7fa40c0787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:20.712 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.709+0000 7fa41affd640 1 -- 172.21.15.167:0/1763896642 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fa4200b72e0 con 0x7fa42c14e420 2024-05-11T08:06:20.712 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.709+0000 7fa42a59c640 1 --2- 172.21.15.167:0/1763896642 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fa40c076300 0x7fa40c0787c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa42c1f8460 tx=0x7fa41c00b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:20.712 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.713+0000 7fa41affd640 1 -- 172.21.15.167:0/1763896642 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fa4200802b0 con 0x7fa42c14e420 2024-05-11T08:06:20.929 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.929+0000 7fa430a69640 1 -- 172.21.15.167:0/1763896642 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa42c071410 con 0x7fa42c14e420 2024-05-11T08:06:20.930 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.930+0000 7fa41affd640 1 -- 172.21.15.167:0/1763896642 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5183 (secure 0 0 0) 0x7fa4200845b0 con 0x7fa42c14e420 2024-05-11T08:06:20.930 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T08:06:20.930 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6834/506789040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":506789040},{"type":"v1","addr":"172.21.15.204:6834","nonce":506789040}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4287,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1448126000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":1448126000},{"type":"v1","addr":"172.21.15.167:6837","nonce":1448126000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4290,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/766834562","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":766834562},{"type":"v1","addr":"172.21.15.167:6836","nonce":766834562}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-11T08:05:54.711448+0000","modified":"2024-05-11T08:06:13.780553+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4305},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4305":{"gid":4305,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.204:6835/1247774569","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6833","nonce":1247774569},{"type":"v1","addr":"172.21.15.204:6835","nonce":1247774569}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4305,"qdb_cluster":[4305]},"id":1}]} 2024-05-11T08:06:20.930 INFO:teuthology.orchestra.run.smithi167.stderr:dumped fsmap epoch 9 2024-05-11T08:06:20.932 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.932+0000 7fa430a69640 1 -- 172.21.15.167:0/1763896642 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fa40c076300 msgr2=0x7fa40c0787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:20.932 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.932+0000 7fa430a69640 1 --2- 172.21.15.167:0/1763896642 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fa40c076300 0x7fa40c0787c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa42c1f8460 tx=0x7fa41c00b040 comp rx=0 tx=0).stop 2024-05-11T08:06:20.932 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.932+0000 7fa430a69640 1 -- 172.21.15.167:0/1763896642 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa42c14e420 msgr2=0x7fa42c147070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:20.933 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.932+0000 7fa430a69640 1 --2- 172.21.15.167:0/1763896642 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa42c14e420 0x7fa42c147070 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fa42000c330 tx=0x7fa42c0b5400 comp rx=0 tx=0).stop 2024-05-11T08:06:20.933 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.932+0000 7fa430a69640 1 -- 172.21.15.167:0/1763896642 shutdown_connections 2024-05-11T08:06:20.933 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.932+0000 7fa430a69640 1 --2- 172.21.15.167:0/1763896642 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fa40c076300 0x7fa40c0787c0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:20.933 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.932+0000 7fa430a69640 1 --2- 172.21.15.167:0/1763896642 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa42c1475b0 0x7fa42c13f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:20.933 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.932+0000 7fa430a69640 1 --2- 172.21.15.167:0/1763896642 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa42c14e420 0x7fa42c147070 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:20.933 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.932+0000 7fa430a69640 1 --2- 172.21.15.167:0/1763896642 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa414004800 0x7fa42c144820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:20.933 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.932+0000 7fa430a69640 1 -- 172.21.15.167:0/1763896642 >> 172.21.15.167:0/1763896642 conn(0x7fa42c0d1e40 msgr2=0x7fa42c15ff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:20.933 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.932+0000 7fa430a69640 1 -- 172.21.15.167:0/1763896642 shutdown_connections 2024-05-11T08:06:20.933 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:20.932+0000 7fa430a69640 1 -- 172.21.15.167:0/1763896642 wait complete. 2024-05-11T08:06:20.944 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-11T08:06:21.044 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.044+0000 7f99e732b640 1 Processor -- start 2024-05-11T08:06:21.045 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.044+0000 7f99e732b640 1 -- start start 2024-05-11T08:06:21.046 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.045+0000 7f99e732b640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f99e015f360 0x7f99e00cfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:21.046 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.045+0000 7f99e732b640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f99e00b5e90 0x7f99e015f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:21.046 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.045+0000 7f99e732b640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99e00d2410 con 0x7f99e015fc80 2024-05-11T08:06:21.046 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.045+0000 7f99e732b640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f99e00d26f0 con 0x7f99e00b5e90 2024-05-11T08:06:21.046 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.045+0000 7f99e732b640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f99e00d29d0 con 0x7f99e015f360 2024-05-11T08:06:21.046 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.045+0000 7f99e48c6640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f99e00b5e90 0x7f99e015f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:21.046 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.045+0000 7f99e58c8640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f99e015fc80 0x7f99e00cd610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:60928/0 (socket says 172.21.15.167:60928) 2024-05-11T08:06:21.046 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.045+0000 7f99e48c6640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f99e00b5e90 0x7f99e015f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:54604/0 (socket says 172.21.15.167:54604) 2024-05-11T08:06:21.046 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.045+0000 7f99e48c6640 1 -- 172.21.15.167:0/160819472 learned_addr learned my addr 172.21.15.167:0/160819472 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:21.046 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.046+0000 7f99e50c7640 1 --2- 172.21.15.167:0/160819472 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f99e015f360 0x7f99e00cfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:21.046 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.046+0000 7f99e48c6640 1 -- 172.21.15.167:0/160819472 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f99e015f360 msgr2=0x7f99e00cfdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:21.047 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.046+0000 7f99e48c6640 1 --2- 172.21.15.167:0/160819472 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f99e015f360 0x7f99e00cfdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:21.047 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.046+0000 7f99e48c6640 1 -- 172.21.15.167:0/160819472 >> v1:172.21.15.167:6789/0 conn(0x7f99e015fc80 legacy=0x7f99e00cd610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:21.047 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.046+0000 7f99dffff640 1 -- 172.21.15.167:0/160819472 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4234065989 0 0) 0x7f99e00d2410 con 0x7f99e015fc80 2024-05-11T08:06:21.047 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.046+0000 7f99e48c6640 1 -- 172.21.15.167:0/160819472 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99e00b7e20 con 0x7f99e00b5e90 2024-05-11T08:06:21.048 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.046+0000 7f99e48c6640 1 --2- 172.21.15.167:0/160819472 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99e00b5e90 0x7f99e015f740 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f99d4009870 tx=0x7f99d402a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e6cf407ac601a2e server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:21.048 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.047+0000 7f99dffff640 1 -- 172.21.15.167:0/160819472 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99d4037070 con 0x7f99e00b5e90 2024-05-11T08:06:21.048 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.047+0000 7f99dffff640 1 -- 172.21.15.167:0/160819472 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99d4032440 con 0x7f99e00b5e90 2024-05-11T08:06:21.048 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.047+0000 7f99dffff640 1 -- 172.21.15.167:0/160819472 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99d4032810 con 0x7f99e00b5e90 2024-05-11T08:06:21.048 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.047+0000 7f99e732b640 1 -- 172.21.15.167:0/160819472 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99e00b5e90 msgr2=0x7f99e015f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:21.048 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.047+0000 7f99e732b640 1 --2- 172.21.15.167:0/160819472 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99e00b5e90 0x7f99e015f740 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f99d4009870 tx=0x7f99d402a1c0 comp rx=0 tx=0).stop 2024-05-11T08:06:21.048 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.047+0000 7f99e732b640 1 -- 172.21.15.167:0/160819472 shutdown_connections 2024-05-11T08:06:21.049 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.047+0000 7f99e732b640 1 --2- 172.21.15.167:0/160819472 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99e00b5e90 0x7f99e015f740 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:21.049 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.047+0000 7f99e732b640 1 --2- 172.21.15.167:0/160819472 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f99e015f360 0x7f99e00cfdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:21.049 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.047+0000 7f99e732b640 1 -- 172.21.15.167:0/160819472 >> 172.21.15.167:0/160819472 conn(0x7f99e0072730 msgr2=0x7f99e00d2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:21.049 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.047+0000 7f99e732b640 1 -- 172.21.15.167:0/160819472 shutdown_connections 2024-05-11T08:06:21.049 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.047+0000 7f99e732b640 1 -- 172.21.15.167:0/160819472 wait complete. 2024-05-11T08:06:21.049 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.048+0000 7f99e732b640 1 Processor -- start 2024-05-11T08:06:21.049 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.048+0000 7f99e732b640 1 -- start start 2024-05-11T08:06:21.051 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.049+0000 7f99e732b640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f99e00b5e90 0x7f99e015a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:21.051 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.049+0000 7f99e732b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f99e015f360 0x7f99e0155fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:21.051 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.049+0000 7f99e732b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99e015fc80 0x7f99e015b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:21.051 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.049+0000 7f99e732b640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f99e00cdd20 con 0x7f99e015fc80 2024-05-11T08:06:21.051 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.049+0000 7f99e732b640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f99e00b7ad0 con 0x7f99e015f360 2024-05-11T08:06:21.051 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.049+0000 7f99e732b640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f99e00cdea0 con 0x7f99e00b5e90 2024-05-11T08:06:21.051 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.049+0000 7f99e58c8640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99e015fc80 0x7f99e015b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:21.051 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.049+0000 7f99e58c8640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99e015fc80 0x7f99e015b080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:54620/0 (socket says 172.21.15.167:54620) 2024-05-11T08:06:21.051 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.049+0000 7f99e58c8640 1 -- 172.21.15.167:0/225127628 learned_addr learned my addr 172.21.15.167:0/225127628 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:21.051 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.049+0000 7f99e50c7640 1 --2- 172.21.15.167:0/225127628 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f99e00b5e90 0x7f99e015a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:21.052 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.049+0000 7f99e48c6640 1 --2- 172.21.15.167:0/225127628 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f99e015f360 0x7f99e0155fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:21.052 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.049+0000 7f99e58c8640 1 -- 172.21.15.167:0/225127628 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f99e00b5e90 msgr2=0x7f99e015a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:21.052 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.050+0000 7f99e58c8640 1 --2- 172.21.15.167:0/225127628 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f99e00b5e90 0x7f99e015a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:21.052 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.050+0000 7f99e58c8640 1 -- 172.21.15.167:0/225127628 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f99e015f360 msgr2=0x7f99e0155fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:21.052 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.050+0000 7f99e58c8640 1 --2- 172.21.15.167:0/225127628 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f99e015f360 0x7f99e0155fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:21.052 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.050+0000 7f99e58c8640 1 -- 172.21.15.167:0/225127628 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99e0156630 con 0x7f99e015fc80 2024-05-11T08:06:21.052 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.050+0000 7f99e58c8640 1 --2- 172.21.15.167:0/225127628 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99e015fc80 0x7f99e015b080 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f99d800c3c0 tx=0x7f99d800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:21.052 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.050+0000 7f99ddffb640 1 -- 172.21.15.167:0/225127628 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99d80074e0 con 0x7f99e015fc80 2024-05-11T08:06:21.052 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.050+0000 7f99ddffb640 1 -- 172.21.15.167:0/225127628 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99d80047e0 con 0x7f99e015fc80 2024-05-11T08:06:21.052 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.050+0000 7f99e732b640 1 -- 172.21.15.167:0/225127628 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99e0156910 con 0x7f99e015fc80 2024-05-11T08:06:21.052 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.050+0000 7f99ddffb640 1 -- 172.21.15.167:0/225127628 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99d8004c50 con 0x7f99e015fc80 2024-05-11T08:06:21.052 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.050+0000 7f99e732b640 1 -- 172.21.15.167:0/225127628 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99e0156ed0 con 0x7f99e015fc80 2024-05-11T08:06:21.052 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.051+0000 7f99e732b640 1 -- 172.21.15.167:0/225127628 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99e0069590 con 0x7f99e015fc80 2024-05-11T08:06:21.053 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.052+0000 7f99ddffb640 1 -- 172.21.15.167:0/225127628 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f99d8011070 con 0x7f99e015fc80 2024-05-11T08:06:21.053 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.052+0000 7f99ddffb640 1 --2- 172.21.15.167:0/225127628 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f99cc0762d0 0x7f99cc078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:21.053 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.052+0000 7f99ddffb640 1 -- 172.21.15.167:0/225127628 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f99d8097430 con 0x7f99e015fc80 2024-05-11T08:06:21.053 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.052+0000 7f99e50c7640 1 --2- 172.21.15.167:0/225127628 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f99cc0762d0 0x7f99cc078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:21.053 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.053+0000 7f99e50c7640 1 --2- 172.21.15.167:0/225127628 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f99cc0762d0 0x7f99cc078790 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f99d0007330 tx=0x7f99d000b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:21.056 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.056+0000 7f99ddffb640 1 -- 172.21.15.167:0/225127628 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f99d8060ae0 con 0x7f99e015fc80 2024-05-11T08:06:21.270 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.270+0000 7f99e732b640 1 -- 172.21.15.167:0/225127628 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f99e00d3420 con 0x7f99e015fc80 2024-05-11T08:06:21.271 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.271+0000 7f99ddffb640 1 -- 172.21.15.167:0/225127628 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5183 (secure 0 0 0) 0x7f99d80603e0 con 0x7f99e015fc80 2024-05-11T08:06:21.271 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T08:06:21.271 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6834/506789040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":506789040},{"type":"v1","addr":"172.21.15.204:6834","nonce":506789040}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4287,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1448126000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":1448126000},{"type":"v1","addr":"172.21.15.167:6837","nonce":1448126000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4290,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/766834562","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":766834562},{"type":"v1","addr":"172.21.15.167:6836","nonce":766834562}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-11T08:05:54.711448+0000","modified":"2024-05-11T08:06:13.780553+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4305},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4305":{"gid":4305,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.204:6835/1247774569","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6833","nonce":1247774569},{"type":"v1","addr":"172.21.15.204:6835","nonce":1247774569}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4305,"qdb_cluster":[4305]},"id":1}]} 2024-05-11T08:06:21.271 INFO:teuthology.orchestra.run.smithi167.stderr:dumped fsmap epoch 9 2024-05-11T08:06:21.273 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.273+0000 7f99e732b640 1 -- 172.21.15.167:0/225127628 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f99cc0762d0 msgr2=0x7f99cc078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:21.273 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.273+0000 7f99e732b640 1 --2- 172.21.15.167:0/225127628 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f99cc0762d0 0x7f99cc078790 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f99d0007330 tx=0x7f99d000b040 comp rx=0 tx=0).stop 2024-05-11T08:06:21.273 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.273+0000 7f99e732b640 1 -- 172.21.15.167:0/225127628 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99e015fc80 msgr2=0x7f99e015b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:21.274 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.273+0000 7f99e732b640 1 --2- 172.21.15.167:0/225127628 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99e015fc80 0x7f99e015b080 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f99d800c3c0 tx=0x7f99d800c880 comp rx=0 tx=0).stop 2024-05-11T08:06:21.274 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.273+0000 7f99e732b640 1 -- 172.21.15.167:0/225127628 shutdown_connections 2024-05-11T08:06:21.274 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.273+0000 7f99e732b640 1 --2- 172.21.15.167:0/225127628 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f99cc0762d0 0x7f99cc078790 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:21.274 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.273+0000 7f99e732b640 1 --2- 172.21.15.167:0/225127628 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99e015fc80 0x7f99e015b080 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:21.274 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.273+0000 7f99e732b640 1 --2- 172.21.15.167:0/225127628 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f99e015f360 0x7f99e0155fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:21.274 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.273+0000 7f99e732b640 1 --2- 172.21.15.167:0/225127628 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f99e00b5e90 0x7f99e015a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:21.274 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.273+0000 7f99e732b640 1 -- 172.21.15.167:0/225127628 >> 172.21.15.167:0/225127628 conn(0x7f99e0072730 msgr2=0x7f99e00ce110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:21.274 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.273+0000 7f99e732b640 1 -- 172.21.15.167:0/225127628 shutdown_connections 2024-05-11T08:06:21.274 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.273+0000 7f99e732b640 1 -- 172.21.15.167:0/225127628 wait complete. 2024-05-11T08:06:21.284 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-11T08:06:21.383 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.383+0000 7fed65246640 1 Processor -- start 2024-05-11T08:06:21.384 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.383+0000 7fed65246640 1 -- start start 2024-05-11T08:06:21.385 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.384+0000 7fed65246640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fed600d2a60 0x7fed600d0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:21.385 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.384+0000 7fed65246640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fed600d1200 0x7fed600b7450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:21.385 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.384+0000 7fed65246640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed60160620 con 0x7fed6015cd50 2024-05-11T08:06:21.385 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.384+0000 7fed65246640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7fed6015d130 con 0x7fed600d1200 2024-05-11T08:06:21.385 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.384+0000 7fed65246640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fed600d1a30 con 0x7fed600d2a60 2024-05-11T08:06:21.385 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.384+0000 7fed5f59e640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fed600d1200 0x7fed600b7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:21.385 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.384+0000 7fed5f59e640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fed600d1200 0x7fed600b7450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:54624/0 (socket says 172.21.15.167:54624) 2024-05-11T08:06:21.385 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.384+0000 7fed5f59e640 1 -- 172.21.15.167:0/95222165 learned_addr learned my addr 172.21.15.167:0/95222165 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:21.385 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.384+0000 7fed5e59c640 1 --2- 172.21.15.167:0/95222165 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fed600d2a60 0x7fed600d0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:21.386 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.385+0000 7fed5f59e640 1 -- 172.21.15.167:0/95222165 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fed600d2a60 msgr2=0x7fed600d0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:21.386 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.385+0000 7fed5f59e640 1 --2- 172.21.15.167:0/95222165 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fed600d2a60 0x7fed600d0cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:21.386 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.385+0000 7fed5f59e640 1 -- 172.21.15.167:0/95222165 >> v1:172.21.15.204:6789/0 conn(0x7fed6015cd50 legacy=0x7fed600d2350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:21.386 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.385+0000 7fed5f59e640 1 -- 172.21.15.167:0/95222165 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed6015f860 con 0x7fed600d1200 2024-05-11T08:06:21.387 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.385+0000 7fed5f59e640 1 --2- 172.21.15.167:0/95222165 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed600d1200 0x7fed600b7450 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fed58009870 tx=0x7fed5802a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e2c93de288f116d server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:21.387 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.385+0000 7fed5dd9b640 1 -- 172.21.15.167:0/95222165 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed58037070 con 0x7fed600d1200 2024-05-11T08:06:21.387 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.385+0000 7fed5dd9b640 1 -- 172.21.15.167:0/95222165 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed58032440 con 0x7fed600d1200 2024-05-11T08:06:21.387 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.385+0000 7fed5dd9b640 1 -- 172.21.15.167:0/95222165 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed58032810 con 0x7fed600d1200 2024-05-11T08:06:21.387 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.386+0000 7fed65246640 1 -- 172.21.15.167:0/95222165 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed600d1200 msgr2=0x7fed600b7450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:21.387 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.386+0000 7fed65246640 1 --2- 172.21.15.167:0/95222165 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed600d1200 0x7fed600b7450 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fed58009870 tx=0x7fed5802a1c0 comp rx=0 tx=0).stop 2024-05-11T08:06:21.387 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.386+0000 7fed65246640 1 -- 172.21.15.167:0/95222165 shutdown_connections 2024-05-11T08:06:21.388 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.386+0000 7fed65246640 1 --2- 172.21.15.167:0/95222165 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed600d1200 0x7fed600b7450 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:21.388 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.386+0000 7fed65246640 1 --2- 172.21.15.167:0/95222165 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fed600d2a60 0x7fed600d0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:21.388 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.386+0000 7fed65246640 1 -- 172.21.15.167:0/95222165 >> 172.21.15.167:0/95222165 conn(0x7fed600b4ae0 msgr2=0x7fed600b4ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:21.388 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.386+0000 7fed65246640 1 -- 172.21.15.167:0/95222165 shutdown_connections 2024-05-11T08:06:21.388 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.386+0000 7fed65246640 1 -- 172.21.15.167:0/95222165 wait complete. 2024-05-11T08:06:21.390 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.388+0000 7fed65246640 1 Processor -- start 2024-05-11T08:06:21.390 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.388+0000 7fed65246640 1 -- start start 2024-05-11T08:06:21.390 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.388+0000 7fed65246640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fed600d1200 0x7fed60156610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:21.391 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.388+0000 7fed65246640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed600d2a60 0x7fed600cf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:21.391 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.388+0000 7fed65246640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fed6015cd50 0x7fed600b0150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:21.391 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.388+0000 7fed65246640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fed60160100 con 0x7fed600d2a60 2024-05-11T08:06:21.391 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.388+0000 7fed65246640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fed6015fe20 con 0x7fed600d1200 2024-05-11T08:06:21.391 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.388+0000 7fed65246640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fed600d1650 con 0x7fed6015cd50 2024-05-11T08:06:21.391 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.388+0000 7fed5e59c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed600d2a60 0x7fed600cf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:21.391 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.388+0000 7fed5e59c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed600d2a60 0x7fed600cf7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:54640/0 (socket says 172.21.15.167:54640) 2024-05-11T08:06:21.391 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.388+0000 7fed5e59c640 1 -- 172.21.15.167:0/2333153177 learned_addr learned my addr 172.21.15.167:0/2333153177 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:21.391 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.388+0000 7fed5f59e640 1 --2- 172.21.15.167:0/2333153177 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fed6015cd50 0x7fed600b0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:21.391 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.389+0000 7fed5ed9d640 1 --2- 172.21.15.167:0/2333153177 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fed600d1200 0x7fed60156610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:21.391 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.389+0000 7fed5e59c640 1 -- 172.21.15.167:0/2333153177 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fed6015cd50 msgr2=0x7fed600b0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:21.391 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.389+0000 7fed5e59c640 1 --2- 172.21.15.167:0/2333153177 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fed6015cd50 0x7fed600b0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:21.391 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.389+0000 7fed5e59c640 1 -- 172.21.15.167:0/2333153177 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fed600d1200 msgr2=0x7fed60156610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:21.391 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.389+0000 7fed5e59c640 1 --2- 172.21.15.167:0/2333153177 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fed600d1200 0x7fed60156610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:21.392 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.389+0000 7fed5e59c640 1 -- 172.21.15.167:0/2333153177 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed601f5990 con 0x7fed600d2a60 2024-05-11T08:06:21.392 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.389+0000 7fed5e59c640 1 --2- 172.21.15.167:0/2333153177 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed600d2a60 0x7fed600cf7b0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fed5400e9b0 tx=0x7fed5400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:21.392 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.389+0000 7fed477fe640 1 -- 172.21.15.167:0/2333153177 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed54017070 con 0x7fed600d2a60 2024-05-11T08:06:21.392 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.389+0000 7fed477fe640 1 -- 172.21.15.167:0/2333153177 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed5400c9a0 con 0x7fed600d2a60 2024-05-11T08:06:21.392 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.389+0000 7fed65246640 1 -- 172.21.15.167:0/2333153177 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed601f5c70 con 0x7fed600d2a60 2024-05-11T08:06:21.392 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.389+0000 7fed477fe640 1 -- 172.21.15.167:0/2333153177 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed5400cd90 con 0x7fed600d2a60 2024-05-11T08:06:21.392 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.389+0000 7fed65246640 1 -- 172.21.15.167:0/2333153177 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed601f6230 con 0x7fed600d2a60 2024-05-11T08:06:21.392 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.390+0000 7fed65246640 1 -- 172.21.15.167:0/2333153177 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed60069bc0 con 0x7fed600d2a60 2024-05-11T08:06:21.392 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.391+0000 7fed477fe640 1 -- 172.21.15.167:0/2333153177 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fed5401a070 con 0x7fed600d2a60 2024-05-11T08:06:21.393 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.391+0000 7fed477fe640 1 --2- 172.21.15.167:0/2333153177 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fed4c0762d0 0x7fed4c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:21.393 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.391+0000 7fed5ed9d640 1 --2- 172.21.15.167:0/2333153177 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fed4c0762d0 0x7fed4c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:21.393 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.391+0000 7fed477fe640 1 -- 172.21.15.167:0/2333153177 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fed54097bc0 con 0x7fed600d2a60 2024-05-11T08:06:21.393 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.392+0000 7fed5ed9d640 1 --2- 172.21.15.167:0/2333153177 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fed4c0762d0 0x7fed4c078790 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fed50001e20 tx=0x7fed50009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:21.395 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.396+0000 7fed477fe640 1 -- 172.21.15.167:0/2333153177 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fed54060c20 con 0x7fed600d2a60 2024-05-11T08:06:21.579 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.579+0000 7fed65246640 1 -- 172.21.15.167:0/2333153177 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fed600d1f00 con 0x7fed600d2a60 2024-05-11T08:06:21.580 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.580+0000 7fed477fe640 1 -- 172.21.15.167:0/2333153177 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18562 (secure 0 0 0) 0x7fed54064f20 con 0x7fed600d2a60 2024-05-11T08:06:21.580 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T08:06:21.580 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":27,"fsid":"2034778d-8ff1-4823-aa44-ded32b518241","created":"2024-05-11T08:05:31.612421+0000","modified":"2024-05-11T08:05:59.756806+0000","last_up_change":"2024-05-11T08:05:42.557382+0000","last_in_change":"2024-05-11T08:05:33.757311+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-11T08:05:43.931263+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-11T08:05:52.573726+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-11T08:05:53.670746+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-11T08:05:57.380272+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"587eaced-5ded-406d-884b-30c1caa8686b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":3863307356},{"type":"v1","addr":"172.21.15.167:6806","nonce":3863307356}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":3863307356},{"type":"v1","addr":"172.21.15.167:6810","nonce":3863307356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":3863307356},{"type":"v1","addr":"172.21.15.167:6815","nonce":3863307356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":3863307356},{"type":"v1","addr":"172.21.15.167:6813","nonce":3863307356}]},"public_addr":"172.21.15.167:6806/3863307356","cluster_addr":"172.21.15.167:6810/3863307356","heartbeat_back_addr":"172.21.15.167:6815/3863307356","heartbeat_front_addr":"172.21.15.167:6813/3863307356","state":["exists","up"]},{"osd":1,"uuid":"37c59104-2dd0-4cf9-85ea-56636f6b02a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":1585024048},{"type":"v1","addr":"172.21.15.167:6817","nonce":1585024048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1585024048},{"type":"v1","addr":"172.21.15.167:6819","nonce":1585024048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1585024048},{"type":"v1","addr":"172.21.15.167:6823","nonce":1585024048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":1585024048},{"type":"v1","addr":"172.21.15.167:6821","nonce":1585024048}]},"public_addr":"172.21.15.167:6817/1585024048","cluster_addr":"172.21.15.167:6819/1585024048","heartbeat_back_addr":"172.21.15.167:6823/1585024048","heartbeat_front_addr":"172.21.15.167:6821/1585024048","state":["exists","up"]},{"osd":2,"uuid":"2bb4f7f5-c698-451b-b4bb-78e716921291","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":3242166900},{"type":"v1","addr":"172.21.15.167:6801","nonce":3242166900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":3242166900},{"type":"v1","addr":"172.21.15.167:6803","nonce":3242166900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6809","nonce":3242166900},{"type":"v1","addr":"172.21.15.167:6811","nonce":3242166900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6805","nonce":3242166900},{"type":"v1","addr":"172.21.15.167:6807","nonce":3242166900}]},"public_addr":"172.21.15.167:6801/3242166900","cluster_addr":"172.21.15.167:6803/3242166900","heartbeat_back_addr":"172.21.15.167:6811/3242166900","heartbeat_front_addr":"172.21.15.167:6807/3242166900","state":["exists","up"]},{"osd":3,"uuid":"9dd26759-a338-49d2-8103-45d068b9c000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":453689293},{"type":"v1","addr":"172.21.15.167:6825","nonce":453689293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":453689293},{"type":"v1","addr":"172.21.15.167:6827","nonce":453689293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":453689293},{"type":"v1","addr":"172.21.15.167:6831","nonce":453689293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":453689293},{"type":"v1","addr":"172.21.15.167:6829","nonce":453689293}]},"public_addr":"172.21.15.167:6825/453689293","cluster_addr":"172.21.15.167:6827/453689293","heartbeat_back_addr":"172.21.15.167:6831/453689293","heartbeat_front_addr":"172.21.15.167:6829/453689293","state":["exists","up"]},{"osd":4,"uuid":"cb27250d-2a80-4c7b-8cef-3a7af05c8a16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1311831500},{"type":"v1","addr":"172.21.15.204:6801","nonce":1311831500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1311831500},{"type":"v1","addr":"172.21.15.204:6803","nonce":1311831500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1311831500},{"type":"v1","addr":"172.21.15.204:6807","nonce":1311831500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1311831500},{"type":"v1","addr":"172.21.15.204:6805","nonce":1311831500}]},"public_addr":"172.21.15.204:6801/1311831500","cluster_addr":"172.21.15.204:6803/1311831500","heartbeat_back_addr":"172.21.15.204:6807/1311831500","heartbeat_front_addr":"172.21.15.204:6805/1311831500","state":["exists","up"]},{"osd":5,"uuid":"92998541-ffec-430b-ab76-8c3cb7c2752d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2921847521},{"type":"v1","addr":"172.21.15.204:6825","nonce":2921847521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2921847521},{"type":"v1","addr":"172.21.15.204:6827","nonce":2921847521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2921847521},{"type":"v1","addr":"172.21.15.204:6831","nonce":2921847521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2921847521},{"type":"v1","addr":"172.21.15.204:6829","nonce":2921847521}]},"public_addr":"172.21.15.204:6825/2921847521","cluster_addr":"172.21.15.204:6827/2921847521","heartbeat_back_addr":"172.21.15.204:6831/2921847521","heartbeat_front_addr":"172.21.15.204:6829/2921847521","state":["exists","up"]},{"osd":6,"uuid":"9543479c-2ec5-42a8-8933-707bfed2e048","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":2889851077},{"type":"v1","addr":"172.21.15.204:6817","nonce":2889851077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":2889851077},{"type":"v1","addr":"172.21.15.204:6819","nonce":2889851077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":2889851077},{"type":"v1","addr":"172.21.15.204:6823","nonce":2889851077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":2889851077},{"type":"v1","addr":"172.21.15.204:6821","nonce":2889851077}]},"public_addr":"172.21.15.204:6817/2889851077","cluster_addr":"172.21.15.204:6819/2889851077","heartbeat_back_addr":"172.21.15.204:6823/2889851077","heartbeat_front_addr":"172.21.15.204:6821/2889851077","state":["exists","up"]},{"osd":7,"uuid":"c45a5647-8358-409e-af91-c05dda65f0b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":1949534184},{"type":"v1","addr":"172.21.15.204:6809","nonce":1949534184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":1949534184},{"type":"v1","addr":"172.21.15.204:6811","nonce":1949534184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1949534184},{"type":"v1","addr":"172.21.15.204:6815","nonce":1949534184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1949534184},{"type":"v1","addr":"172.21.15.204:6813","nonce":1949534184}]},"public_addr":"172.21.15.204:6809/1949534184","cluster_addr":"172.21.15.204:6811/1949534184","heartbeat_back_addr":"172.21.15.204:6815/1949534184","heartbeat_front_addr":"172.21.15.204:6813/1949534184","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.952253+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.940689+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.860682+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.929350+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.901627+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.957451+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.987249+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.951197+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-11T08:06:21.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.582+0000 7fed65246640 1 -- 172.21.15.167:0/2333153177 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fed4c0762d0 msgr2=0x7fed4c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:21.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.582+0000 7fed65246640 1 --2- 172.21.15.167:0/2333153177 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fed4c0762d0 0x7fed4c078790 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fed50001e20 tx=0x7fed50009040 comp rx=0 tx=0).stop 2024-05-11T08:06:21.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.582+0000 7fed65246640 1 -- 172.21.15.167:0/2333153177 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed600d2a60 msgr2=0x7fed600cf7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:21.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.582+0000 7fed65246640 1 --2- 172.21.15.167:0/2333153177 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed600d2a60 0x7fed600cf7b0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fed5400e9b0 tx=0x7fed5400ee70 comp rx=0 tx=0).stop 2024-05-11T08:06:21.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.582+0000 7fed65246640 1 -- 172.21.15.167:0/2333153177 shutdown_connections 2024-05-11T08:06:21.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.582+0000 7fed65246640 1 --2- 172.21.15.167:0/2333153177 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fed4c0762d0 0x7fed4c078790 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:21.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.582+0000 7fed65246640 1 --2- 172.21.15.167:0/2333153177 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fed6015cd50 0x7fed600b0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:21.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.582+0000 7fed65246640 1 --2- 172.21.15.167:0/2333153177 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed600d2a60 0x7fed600cf7b0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:21.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.582+0000 7fed65246640 1 --2- 172.21.15.167:0/2333153177 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fed600d1200 0x7fed60156610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:21.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.582+0000 7fed65246640 1 -- 172.21.15.167:0/2333153177 >> 172.21.15.167:0/2333153177 conn(0x7fed600b4ae0 msgr2=0x7fed60160bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:21.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.582+0000 7fed65246640 1 -- 172.21.15.167:0/2333153177 shutdown_connections 2024-05-11T08:06:21.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.582+0000 7fed65246640 1 -- 172.21.15.167:0/2333153177 wait complete. 2024-05-11T08:06:21.593 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-11T08:06:21.694 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.694+0000 7fe448f9e640 1 Processor -- start 2024-05-11T08:06:21.694 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.694+0000 7fe448f9e640 1 -- start start 2024-05-11T08:06:21.696 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.695+0000 7fe448f9e640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fe4440b8a40 0x7fe444160540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:21.696 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.695+0000 7fe448f9e640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fe4440d0ec0 0x7fe4440d1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:21.696 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.695+0000 7fe448f9e640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe444160a80 con 0x7fe4440b5e90 2024-05-11T08:06:21.696 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.695+0000 7fe448f9e640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7fe4440d1850 con 0x7fe4440b8a40 2024-05-11T08:06:21.696 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.695+0000 7fe448f9e640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fe4440d24a0 con 0x7fe4440d0ec0 2024-05-11T08:06:21.696 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.695+0000 7fe44259c640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fe4440b8a40 0x7fe444160540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:21.697 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.695+0000 7fe44259c640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fe4440b8a40 0x7fe444160540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:54642/0 (socket says 172.21.15.167:54642) 2024-05-11T08:06:21.697 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.695+0000 7fe44259c640 1 -- 172.21.15.167:0/2076653216 learned_addr learned my addr 172.21.15.167:0/2076653216 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:21.697 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.695+0000 7fe44259c640 1 -- 172.21.15.167:0/2076653216 >> v2:172.21.15.204:3300/0 conn(0x7fe4440d0ec0 msgr2=0x7fe4440d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:21.697 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.695+0000 7fe442d9d640 1 --2- 172.21.15.167:0/2076653216 >> v2:172.21.15.204:3300/0 conn(0x7fe4440d0ec0 0x7fe4440d1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:21.697 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.695+0000 7fe44259c640 1 --2- 172.21.15.167:0/2076653216 >> v2:172.21.15.204:3300/0 conn(0x7fe4440d0ec0 0x7fe4440d1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:21.697 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.695+0000 7fe44259c640 1 -- 172.21.15.167:0/2076653216 >> v1:172.21.15.204:6789/0 conn(0x7fe4440b5e90 legacy=0x7fe4440cfba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:21.697 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.695+0000 7fe44259c640 1 -- 172.21.15.167:0/2076653216 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4440cf3e0 con 0x7fe4440b8a40 2024-05-11T08:06:21.697 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.695+0000 7fe442d9d640 1 --2- 172.21.15.167:0/2076653216 >> v2:172.21.15.204:3300/0 conn(0x7fe4440d0ec0 0x7fe4440d1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T08:06:21.697 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.696+0000 7fe44259c640 1 --2- 172.21.15.167:0/2076653216 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe4440b8a40 0x7fe444160540 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fe4440b67e0 tx=0x7fe43402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3436bd5d1aba0cbd server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:21.697 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.696+0000 7fe441d9b640 1 -- 172.21.15.167:0/2076653216 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe434037070 con 0x7fe4440b8a40 2024-05-11T08:06:21.697 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.696+0000 7fe441d9b640 1 -- 172.21.15.167:0/2076653216 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe434032440 con 0x7fe4440b8a40 2024-05-11T08:06:21.697 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.696+0000 7fe441d9b640 1 -- 172.21.15.167:0/2076653216 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe434032810 con 0x7fe4440b8a40 2024-05-11T08:06:21.700 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.697+0000 7fe448f9e640 1 -- 172.21.15.167:0/2076653216 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe4440b8a40 msgr2=0x7fe444160540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:21.700 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.697+0000 7fe448f9e640 1 --2- 172.21.15.167:0/2076653216 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe4440b8a40 0x7fe444160540 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fe4440b67e0 tx=0x7fe43402a140 comp rx=0 tx=0).stop 2024-05-11T08:06:21.700 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.697+0000 7fe448f9e640 1 -- 172.21.15.167:0/2076653216 shutdown_connections 2024-05-11T08:06:21.700 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.697+0000 7fe448f9e640 1 --2- 172.21.15.167:0/2076653216 >> v2:172.21.15.204:3300/0 conn(0x7fe4440d0ec0 0x7fe4440d1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:21.700 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.697+0000 7fe448f9e640 1 --2- 172.21.15.167:0/2076653216 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe4440b8a40 0x7fe444160540 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:21.701 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.697+0000 7fe448f9e640 1 -- 172.21.15.167:0/2076653216 >> 172.21.15.167:0/2076653216 conn(0x7fe444072730 msgr2=0x7fe4440d2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:21.701 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.697+0000 7fe448f9e640 1 -- 172.21.15.167:0/2076653216 shutdown_connections 2024-05-11T08:06:21.701 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.697+0000 7fe448f9e640 1 -- 172.21.15.167:0/2076653216 wait complete. 2024-05-11T08:06:21.701 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.698+0000 7fe448f9e640 1 Processor -- start 2024-05-11T08:06:21.701 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.698+0000 7fe448f9e640 1 -- start start 2024-05-11T08:06:21.701 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.699+0000 7fe448f9e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe4440b5e90 0x7fe4441f7ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:21.701 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.699+0000 7fe448f9e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe4440b8a40 0x7fe444157fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:21.701 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.699+0000 7fe448f9e640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe4440d0ec0 0x7fe4441e8630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:21.701 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.699+0000 7fe448f9e640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe4440cf680 con 0x7fe4440b5e90 2024-05-11T08:06:21.701 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.699+0000 7fe448f9e640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe4440d2a20 con 0x7fe4440b8a40 2024-05-11T08:06:21.701 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.699+0000 7fe448f9e640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fe4440d02b0 con 0x7fe4440d0ec0 2024-05-11T08:06:21.701 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.699+0000 7fe44259c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe4440b5e90 0x7fe4441f7ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:21.701 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.699+0000 7fe44259c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe4440b5e90 0x7fe4441f7ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:54652/0 (socket says 172.21.15.167:54652) 2024-05-11T08:06:21.702 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.699+0000 7fe44259c640 1 -- 172.21.15.167:0/1058012384 learned_addr learned my addr 172.21.15.167:0/1058012384 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:21.702 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.699+0000 7fe442d9d640 1 --2- 172.21.15.167:0/1058012384 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe4440d0ec0 0x7fe4441e8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:21.702 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.699+0000 7fe43bfff640 1 --2- 172.21.15.167:0/1058012384 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe4440b8a40 0x7fe444157fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:21.702 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.699+0000 7fe44259c640 1 -- 172.21.15.167:0/1058012384 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe4440d0ec0 msgr2=0x7fe4441e8630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:21.702 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.699+0000 7fe44259c640 1 --2- 172.21.15.167:0/1058012384 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe4440d0ec0 0x7fe4441e8630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:21.702 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.699+0000 7fe44259c640 1 -- 172.21.15.167:0/1058012384 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe4440b8a40 msgr2=0x7fe444157fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:21.702 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.699+0000 7fe44259c640 1 --2- 172.21.15.167:0/1058012384 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe4440b8a40 0x7fe444157fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:21.702 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.699+0000 7fe44259c640 1 -- 172.21.15.167:0/1058012384 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4441f6e20 con 0x7fe4440b5e90 2024-05-11T08:06:21.702 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.699+0000 7fe43bfff640 1 --2- 172.21.15.167:0/1058012384 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe4440b8a40 0x7fe444157fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T08:06:21.702 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.699+0000 7fe44259c640 1 --2- 172.21.15.167:0/1058012384 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe4440b5e90 0x7fe4441f7ec0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe434000c00 tx=0x7fe4340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:21.702 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.700+0000 7fe43affd640 1 -- 172.21.15.167:0/1058012384 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe434041020 con 0x7fe4440b5e90 2024-05-11T08:06:21.702 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.700+0000 7fe43affd640 1 -- 172.21.15.167:0/1058012384 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe434030970 con 0x7fe4440b5e90 2024-05-11T08:06:21.702 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.700+0000 7fe448f9e640 1 -- 172.21.15.167:0/1058012384 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4441f7100 con 0x7fe4440b5e90 2024-05-11T08:06:21.703 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.700+0000 7fe43affd640 1 -- 172.21.15.167:0/1058012384 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe434037040 con 0x7fe4440b5e90 2024-05-11T08:06:21.703 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.700+0000 7fe448f9e640 1 -- 172.21.15.167:0/1058012384 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4441f76c0 con 0x7fe4440b5e90 2024-05-11T08:06:21.703 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.701+0000 7fe448f9e640 1 -- 172.21.15.167:0/1058012384 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe444069590 con 0x7fe4440b5e90 2024-05-11T08:06:21.703 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.701+0000 7fe43affd640 1 -- 172.21.15.167:0/1058012384 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fe434044020 con 0x7fe4440b5e90 2024-05-11T08:06:21.704 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.702+0000 7fe43affd640 1 --2- 172.21.15.167:0/1058012384 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fe4240762d0 0x7fe424078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:21.704 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.702+0000 7fe43bfff640 1 --2- 172.21.15.167:0/1058012384 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fe4240762d0 0x7fe424078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:21.704 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.702+0000 7fe43affd640 1 -- 172.21.15.167:0/1058012384 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fe4340b8270 con 0x7fe4440b5e90 2024-05-11T08:06:21.704 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.702+0000 7fe43bfff640 1 --2- 172.21.15.167:0/1058012384 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fe4240762d0 0x7fe424078790 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fe4441e9a70 tx=0x7fe42c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:21.706 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.706+0000 7fe43affd640 1 -- 172.21.15.167:0/1058012384 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fe43404b030 con 0x7fe4440b5e90 2024-05-11T08:06:21.928 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.928+0000 7fe448f9e640 1 -- 172.21.15.167:0/1058012384 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe4440d3250 con 0x7fe4440b5e90 2024-05-11T08:06:21.929 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.929+0000 7fe43affd640 1 -- 172.21.15.167:0/1058012384 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5183 (secure 0 0 0) 0x7fe444160a80 con 0x7fe4440b5e90 2024-05-11T08:06:21.929 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T08:06:21.929 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6834/506789040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":506789040},{"type":"v1","addr":"172.21.15.204:6834","nonce":506789040}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4287,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1448126000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":1448126000},{"type":"v1","addr":"172.21.15.167:6837","nonce":1448126000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4290,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/766834562","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":766834562},{"type":"v1","addr":"172.21.15.167:6836","nonce":766834562}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-11T08:05:54.711448+0000","modified":"2024-05-11T08:06:13.780553+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4305},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4305":{"gid":4305,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.204:6835/1247774569","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6833","nonce":1247774569},{"type":"v1","addr":"172.21.15.204:6835","nonce":1247774569}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4305,"qdb_cluster":[4305]},"id":1}]} 2024-05-11T08:06:21.929 INFO:teuthology.orchestra.run.smithi167.stderr:dumped fsmap epoch 9 2024-05-11T08:06:21.931 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.931+0000 7fe448f9e640 1 -- 172.21.15.167:0/1058012384 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fe4240762d0 msgr2=0x7fe424078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:21.931 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.931+0000 7fe448f9e640 1 --2- 172.21.15.167:0/1058012384 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fe4240762d0 0x7fe424078790 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fe4441e9a70 tx=0x7fe42c009040 comp rx=0 tx=0).stop 2024-05-11T08:06:21.932 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.931+0000 7fe448f9e640 1 -- 172.21.15.167:0/1058012384 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe4440b5e90 msgr2=0x7fe4441f7ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:21.932 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.931+0000 7fe448f9e640 1 --2- 172.21.15.167:0/1058012384 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe4440b5e90 0x7fe4441f7ec0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe434000c00 tx=0x7fe4340046c0 comp rx=0 tx=0).stop 2024-05-11T08:06:21.932 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.931+0000 7fe448f9e640 1 -- 172.21.15.167:0/1058012384 shutdown_connections 2024-05-11T08:06:21.932 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.931+0000 7fe448f9e640 1 --2- 172.21.15.167:0/1058012384 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fe4240762d0 0x7fe424078790 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:21.932 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.931+0000 7fe448f9e640 1 --2- 172.21.15.167:0/1058012384 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe4440d0ec0 0x7fe4441e8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:21.932 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.931+0000 7fe448f9e640 1 --2- 172.21.15.167:0/1058012384 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe4440b8a40 0x7fe444157fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:21.932 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.931+0000 7fe448f9e640 1 --2- 172.21.15.167:0/1058012384 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe4440b5e90 0x7fe4441f7ec0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:21.933 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.931+0000 7fe448f9e640 1 -- 172.21.15.167:0/1058012384 >> 172.21.15.167:0/1058012384 conn(0x7fe444072730 msgr2=0x7fe44406ddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:21.933 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.931+0000 7fe448f9e640 1 -- 172.21.15.167:0/1058012384 shutdown_connections 2024-05-11T08:06:21.933 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:21.931+0000 7fe448f9e640 1 -- 172.21.15.167:0/1058012384 wait complete. 2024-05-11T08:06:22.345 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 9, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-11T08:05:54.711448+0000', 'modified': '2024-05-11T08:06:13.780553+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4305}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4305': {'gid': 4305, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.204:6835/1247774569', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.204:6833', 'nonce': 1247774569}, {'type': 'v1', 'addr': '172.21.15.204:6835', 'nonce': 1247774569}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4305, 'qdb_cluster': [4305]} 2024-05-11T08:06:22.345 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-11T08:06:22.345 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-05-11T08:06:22.453 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.453+0000 7f60a8e37640 1 Processor -- start 2024-05-11T08:06:22.453 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.453+0000 7f60a8e37640 1 -- start start 2024-05-11T08:06:22.455 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.454+0000 7f60a8e37640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f60a415f360 0x7f60a40cfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:22.455 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.454+0000 7f60a8e37640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f60a40cd610 0x7f60a40cda60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:22.455 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.454+0000 7f60a8e37640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60a40d2410 con 0x7f60a40b5e90 2024-05-11T08:06:22.455 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.454+0000 7f60a8e37640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f60a40cdfa0 con 0x7f60a415f360 2024-05-11T08:06:22.455 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.454+0000 7f60a8e37640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f60a40d26f0 con 0x7f60a40cd610 2024-05-11T08:06:22.455 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.454+0000 7f60a259c640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f60a415f360 0x7f60a40cfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:22.455 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.454+0000 7f60a1d9b640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f60a40b5e90 0x7f60a415f740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:60932/0 (socket says 172.21.15.167:60932) 2024-05-11T08:06:22.455 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.454+0000 7f60a1d9b640 1 -- 172.21.15.167:0/42587529 learned_addr learned my addr 172.21.15.167:0/42587529 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T08:06:22.455 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.454+0000 7f60a2d9d640 1 --2- 172.21.15.167:0/42587529 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f60a40cd610 0x7f60a40cda60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:22.455 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.454+0000 7f60a259c640 1 -- 172.21.15.167:0/42587529 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f60a40cd610 msgr2=0x7f60a40cda60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:22.456 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.454+0000 7f60a259c640 1 --2- 172.21.15.167:0/42587529 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f60a40cd610 0x7f60a40cda60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:22.456 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.454+0000 7f60a259c640 1 -- 172.21.15.167:0/42587529 >> v1:172.21.15.167:6789/0 conn(0x7f60a40b5e90 legacy=0x7f60a415f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:22.456 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.454+0000 7f60a259c640 1 -- 172.21.15.167:0/42587529 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60a40d29d0 con 0x7f60a415f360 2024-05-11T08:06:22.456 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.454+0000 7f60a159a640 1 -- 172.21.15.167:0/42587529 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1890147676 0 0) 0x7f60a40d2410 con 0x7f60a40b5e90 2024-05-11T08:06:22.456 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.454+0000 7f60a2d9d640 1 --2- 172.21.15.167:0/42587529 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f60a40cd610 0x7f60a40cda60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T08:06:22.456 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.455+0000 7f60a259c640 1 --2- 172.21.15.167:0/42587529 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f60a415f360 0x7f60a40cfdc0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f60940097a0 tx=0x7f609402a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33cb8aa6a0ca000e server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:22.456 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.455+0000 7f60a159a640 1 -- 172.21.15.167:0/42587529 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6094037070 con 0x7f60a415f360 2024-05-11T08:06:22.456 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.455+0000 7f60a159a640 1 -- 172.21.15.167:0/42587529 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f609402ae50 con 0x7f60a415f360 2024-05-11T08:06:22.456 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.455+0000 7f60a159a640 1 -- 172.21.15.167:0/42587529 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6094032660 con 0x7f60a415f360 2024-05-11T08:06:22.457 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.456+0000 7f60a8e37640 1 -- 172.21.15.167:0/42587529 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f60a415f360 msgr2=0x7f60a40cfdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:22.457 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.456+0000 7f60a8e37640 1 --2- 172.21.15.167:0/42587529 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f60a415f360 0x7f60a40cfdc0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f60940097a0 tx=0x7f609402a100 comp rx=0 tx=0).stop 2024-05-11T08:06:22.457 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.456+0000 7f60a8e37640 1 -- 172.21.15.167:0/42587529 shutdown_connections 2024-05-11T08:06:22.457 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.456+0000 7f60a8e37640 1 --2- 172.21.15.167:0/42587529 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f60a40cd610 0x7f60a40cda60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:22.457 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.456+0000 7f60a8e37640 1 --2- 172.21.15.167:0/42587529 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f60a415f360 0x7f60a40cfdc0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:22.457 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.456+0000 7f60a8e37640 1 -- 172.21.15.167:0/42587529 >> 172.21.15.167:0/42587529 conn(0x7f60a4072730 msgr2=0x7f60a40d2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:22.457 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.456+0000 7f60a8e37640 1 -- 172.21.15.167:0/42587529 shutdown_connections 2024-05-11T08:06:22.457 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.456+0000 7f60a8e37640 1 -- 172.21.15.167:0/42587529 wait complete. 2024-05-11T08:06:22.458 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.457+0000 7f60a8e37640 1 Processor -- start 2024-05-11T08:06:22.458 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.457+0000 7f60a8e37640 1 -- start start 2024-05-11T08:06:22.459 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.458+0000 7f60a8e37640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f60a40b5e90 0x7f60a4158560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:22.460 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.458+0000 7f60a8e37640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f60a40cd610 0x7f60a415b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:22.460 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.458+0000 7f60a8e37640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f60a415f360 0x7f60a4147530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:22.460 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.458+0000 7f60a8e37640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f60a40b7ad0 con 0x7f60a415f360 2024-05-11T08:06:22.460 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.458+0000 7f60a8e37640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f60a40d0300 con 0x7f60a40b5e90 2024-05-11T08:06:22.460 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.458+0000 7f60a8e37640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f60a415fe50 con 0x7f60a40cd610 2024-05-11T08:06:22.460 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.458+0000 7f60a2d9d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f60a415f360 0x7f60a4147530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:22.460 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.458+0000 7f60a2d9d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f60a415f360 0x7f60a4147530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:54664/0 (socket says 172.21.15.167:54664) 2024-05-11T08:06:22.460 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.458+0000 7f60a2d9d640 1 -- 172.21.15.167:0/1607748287 learned_addr learned my addr 172.21.15.167:0/1607748287 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:22.460 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.458+0000 7f60a259c640 1 --2- 172.21.15.167:0/1607748287 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f60a40b5e90 0x7f60a4158560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:22.460 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.458+0000 7f60a1d9b640 1 --2- 172.21.15.167:0/1607748287 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f60a40cd610 0x7f60a415b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:22.460 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.458+0000 7f60a2d9d640 1 -- 172.21.15.167:0/1607748287 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f60a40cd610 msgr2=0x7f60a415b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:22.460 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.458+0000 7f60a2d9d640 1 --2- 172.21.15.167:0/1607748287 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f60a40cd610 0x7f60a415b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:22.460 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.458+0000 7f60a2d9d640 1 -- 172.21.15.167:0/1607748287 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f60a40b5e90 msgr2=0x7f60a4158560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:22.460 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.458+0000 7f60a2d9d640 1 --2- 172.21.15.167:0/1607748287 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f60a40b5e90 0x7f60a4158560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:22.461 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.458+0000 7f60a2d9d640 1 -- 172.21.15.167:0/1607748287 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60a4147a70 con 0x7f60a415f360 2024-05-11T08:06:22.461 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.458+0000 7f60a2d9d640 1 --2- 172.21.15.167:0/1607748287 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f60a415f360 0x7f60a4147530 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f609c007ef0 tx=0x7f609c00d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:22.461 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.459+0000 7f609affd640 1 -- 172.21.15.167:0/1607748287 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f609c01a070 con 0x7f60a415f360 2024-05-11T08:06:22.461 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.459+0000 7f609affd640 1 -- 172.21.15.167:0/1607748287 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f609c003b90 con 0x7f60a415f360 2024-05-11T08:06:22.461 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.459+0000 7f609affd640 1 -- 172.21.15.167:0/1607748287 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f609c004030 con 0x7f60a415f360 2024-05-11T08:06:22.461 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.459+0000 7f60a8e37640 1 -- 172.21.15.167:0/1607748287 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60a4158c70 con 0x7f60a415f360 2024-05-11T08:06:22.461 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.459+0000 7f60a8e37640 1 -- 172.21.15.167:0/1607748287 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60a4146d30 con 0x7f60a415f360 2024-05-11T08:06:22.462 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.461+0000 7f609affd640 1 -- 172.21.15.167:0/1607748287 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f609c01d070 con 0x7f60a415f360 2024-05-11T08:06:22.462 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.461+0000 7f609affd640 1 --2- 172.21.15.167:0/1607748287 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f60840762d0 0x7f6084078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:22.462 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.461+0000 7f609affd640 1 -- 172.21.15.167:0/1607748287 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f609c097b50 con 0x7f60a415f360 2024-05-11T08:06:22.462 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.461+0000 7f60a259c640 1 --2- 172.21.15.167:0/1607748287 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f60840762d0 0x7f6084078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:22.462 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.462+0000 7f60a259c640 1 --2- 172.21.15.167:0/1607748287 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f60840762d0 0x7f6084078790 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f6094034000 tx=0x7f609402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:22.462 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.462+0000 7f60abc2d740 1 Processor -- start 2024-05-11T08:06:22.462 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.462+0000 7f60abc2d740 1 -- start start 2024-05-11T08:06:22.465 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.463+0000 7f60abc2d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d84e8fcff0 0x55d84e8ff800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:22.465 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.463+0000 7f60abc2d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d84e8fd3d0 0x55d84e8ffd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:22.465 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.463+0000 7f60abc2d740 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55d84e8cfc50 0x55d84e8d3460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:22.465 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.463+0000 7f60abc2d740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55d84e8398d0 con 0x55d84e8fd3d0 2024-05-11T08:06:22.466 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.463+0000 7f60abc2d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55d84e7f2c10 con 0x55d84e8fcff0 2024-05-11T08:06:22.466 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.463+0000 7f60abc2d740 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x55d84e835890 con 0x55d84e8cfc50 2024-05-11T08:06:22.466 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.463+0000 7f60a1d9b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d84e8fd3d0 0x55d84e8ffd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:22.466 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.463+0000 7f60a1d9b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d84e8fd3d0 0x55d84e8ffd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:54680/0 (socket says 172.21.15.167:54680) 2024-05-11T08:06:22.466 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.463+0000 7f60a1d9b640 1 -- 172.21.15.167:0/1716867725 learned_addr learned my addr 172.21.15.167:0/1716867725 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:22.466 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.463+0000 7f60a259c640 1 --2- 172.21.15.167:0/1716867725 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d84e8fcff0 0x55d84e8ff800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:22.466 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.463+0000 7f60a2d9d640 1 --2- 172.21.15.167:0/1716867725 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55d84e8cfc50 0x55d84e8d3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:22.466 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.463+0000 7f60a1d9b640 1 -- 172.21.15.167:0/1716867725 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55d84e8cfc50 msgr2=0x55d84e8d3460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:22.466 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.463+0000 7f60a1d9b640 1 --2- 172.21.15.167:0/1716867725 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55d84e8cfc50 0x55d84e8d3460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:22.466 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.463+0000 7f60a1d9b640 1 -- 172.21.15.167:0/1716867725 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d84e8fcff0 msgr2=0x55d84e8ff800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:22.466 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.463+0000 7f60a1d9b640 1 --2- 172.21.15.167:0/1716867725 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d84e8fcff0 0x55d84e8ff800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:22.467 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.463+0000 7f60a1d9b640 1 -- 172.21.15.167:0/1716867725 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d84e7fa5d0 con 0x55d84e8fd3d0 2024-05-11T08:06:22.467 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.463+0000 7f60a1d9b640 1 --2- 172.21.15.167:0/1716867725 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d84e8fd3d0 0x55d84e8ffd40 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f608c001220 tx=0x7f608c004bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d160aaeb7d6b51d5 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:22.467 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.463+0000 7f6083fff640 1 -- 172.21.15.167:0/1716867725 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f608c001870 con 0x55d84e8fd3d0 2024-05-11T08:06:22.467 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.463+0000 7f6083fff640 1 -- 172.21.15.167:0/1716867725 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f608c018070 con 0x55d84e8fd3d0 2024-05-11T08:06:22.467 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.463+0000 7f6083fff640 1 -- 172.21.15.167:0/1716867725 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f608c001c30 con 0x55d84e8fd3d0 2024-05-11T08:06:22.467 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.464+0000 7f60abc2d740 1 -- 172.21.15.167:0/1716867725 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d84e8fd3d0 msgr2=0x55d84e8ffd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:22.467 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.464+0000 7f60abc2d740 1 --2- 172.21.15.167:0/1716867725 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d84e8fd3d0 0x55d84e8ffd40 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f608c001220 tx=0x7f608c004bb0 comp rx=0 tx=0).stop 2024-05-11T08:06:22.467 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.464+0000 7f60abc2d740 1 -- 172.21.15.167:0/1716867725 shutdown_connections 2024-05-11T08:06:22.467 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.464+0000 7f60abc2d740 1 --2- 172.21.15.167:0/1716867725 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55d84e8cfc50 0x55d84e8d3460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:22.467 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.464+0000 7f60abc2d740 1 --2- 172.21.15.167:0/1716867725 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d84e8fd3d0 0x55d84e8ffd40 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:22.467 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.464+0000 7f60abc2d740 1 --2- 172.21.15.167:0/1716867725 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d84e8fcff0 0x55d84e8ff800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:22.467 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.464+0000 7f60abc2d740 1 -- 172.21.15.167:0/1716867725 >> 172.21.15.167:0/1716867725 conn(0x55d84e8d1ec0 msgr2=0x55d84e91a800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:22.467 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.464+0000 7f60abc2d740 1 -- 172.21.15.167:0/1716867725 shutdown_connections 2024-05-11T08:06:22.467 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.464+0000 7f60abc2d740 1 -- 172.21.15.167:0/1716867725 wait complete. 2024-05-11T08:06:22.467 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.465+0000 7f60abc2d740 1 -- start start 2024-05-11T08:06:22.468 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.465+0000 7f60abc2d740 1 Processor -- start 2024-05-11T08:06:22.471 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.466+0000 7f60abc2d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d84e8fcff0 0x55d84e8f6f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:22.472 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.466+0000 7f60abc2d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d84e8fd3d0 0x55d84e911e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:22.472 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.466+0000 7f60abc2d740 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55d84e8f60c0 0x55d84e912370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:22.472 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.466+0000 7f60abc2d740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55d84e8d5070 con 0x55d84e8fcff0 2024-05-11T08:06:22.472 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.466+0000 7f60abc2d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55d84e7f6760 con 0x55d84e8fd3d0 2024-05-11T08:06:22.472 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.466+0000 7f60abc2d740 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x55d84e8d5350 con 0x55d84e8f60c0 2024-05-11T08:06:22.472 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.466+0000 7f60abc2d740 10 client.0 _ll_register_callbacks cb 0x55d84e82fcf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T08:06:22.472 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.466+0000 7f60a259c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d84e8fcff0 0x55d84e8f6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:22.472 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.466+0000 7f60a259c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d84e8fcff0 0x55d84e8f6f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:54682/0 (socket says 172.21.15.167:54682) 2024-05-11T08:06:22.472 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.466+0000 7f60a259c640 1 -- 172.21.15.167:0/2988490943 learned_addr learned my addr 172.21.15.167:0/2988490943 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:22.472 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.466+0000 7f60a2d9d640 1 --2- 172.21.15.167:0/2988490943 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55d84e8f60c0 0x55d84e912370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:22.472 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.466+0000 7f60a1d9b640 1 --2- 172.21.15.167:0/2988490943 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d84e8fd3d0 0x55d84e911e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:22.472 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.466+0000 7f60a259c640 1 -- 172.21.15.167:0/2988490943 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55d84e8f60c0 msgr2=0x55d84e912370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:22.472 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.466+0000 7f60a259c640 1 --2- 172.21.15.167:0/2988490943 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55d84e8f60c0 0x55d84e912370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:22.472 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.466+0000 7f60a259c640 1 -- 172.21.15.167:0/2988490943 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d84e8fd3d0 msgr2=0x55d84e911e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:22.472 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.466+0000 7f60a259c640 1 --2- 172.21.15.167:0/2988490943 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d84e8fd3d0 0x55d84e911e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:22.473 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.466+0000 7f60a259c640 1 -- 172.21.15.167:0/2988490943 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d84e54b2d0 con 0x55d84e8fcff0 2024-05-11T08:06:22.473 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.466+0000 7f60a259c640 1 --2- 172.21.15.167:0/2988490943 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d84e8fcff0 0x55d84e8f6f40 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f609402a6f0 tx=0x7f6094004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37c21b697223a57d server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:22.473 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.467+0000 7f60817fa640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-05-11T08:06:22.473 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.467+0000 7f60817fa640 1 -- 172.21.15.167:0/2988490943 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6094050020 con 0x55d84e8fcff0 2024-05-11T08:06:22.473 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.467+0000 7f60817fa640 1 -- 172.21.15.167:0/2988490943 <== mon.0 v2:172.21.15.167:3300/0 2 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f6094003ce0 con 0x55d84e8fcff0 2024-05-11T08:06:22.473 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.467+0000 7f60abc2d740 1 -- 172.21.15.167:0/2988490943 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f608c018070 con 0x55d84e8fcff0 2024-05-11T08:06:22.473 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.467+0000 7f60817fa640 1 -- 172.21.15.167:0/2988490943 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) ==== 24+0+0 (secure 0 0 0) 0x7f6094037040 con 0x55d84e8fcff0 2024-05-11T08:06:22.473 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.467+0000 7f60abc2d740 10 client.4725 fetch_fsmap learned FSMap version 9 2024-05-11T08:06:22.473 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.467+0000 7f60abc2d740 1 -- 172.21.15.167:0/2988490943 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) -- 0x55d84e6156b0 con 0x55d84e8fcff0 2024-05-11T08:06:22.473 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.467+0000 7f60817fa640 1 -- 172.21.15.167:0/2988490943 <== mon.0 v2:172.21.15.167:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f608c018070 con 0x55d84e8fcff0 2024-05-11T08:06:22.473 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.467+0000 7f60817fa640 1 -- 172.21.15.167:0/2988490943 <== mon.0 v2:172.21.15.167:3300/0 5 ==== fsmap(e 9) ==== 3502+0+0 (secure 0 0 0) 0x7f60940553f0 con 0x55d84e8fcff0 2024-05-11T08:06:22.473 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.467+0000 7f60abc2d740 10 client.4725 fetch_fsmap finished waiting for FSMap version 9 2024-05-11T08:06:22.473 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.467+0000 7f60abc2d740 10 client.4725 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-11T08:06:22.473 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.467+0000 7f60abc2d740 20 client.4725 populate_metadata read hostname 'smithi167' 2024-05-11T08:06:22.474 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.467+0000 7f60abc2d740 1 --2- 172.21.15.167:0/2988490943 >> [v2:172.21.15.204:6833/1247774569,v1:172.21.15.204:6835/1247774569] conn(0x55d84e936790 0x55d84e938b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:22.474 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.467+0000 7f60abc2d740 4 client.4725 mds_command: new command op to 4305 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-11T08:06:22.474 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.467+0000 7f60abc2d740 1 -- 172.21.15.167:0/2988490943 --> [v2:172.21.15.204:6833/1247774569,v1:172.21.15.204:6835/1247774569] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d84e7bd760 con 0x55d84e936790 2024-05-11T08:06:22.474 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.467+0000 7f60a1d9b640 1 --2- 172.21.15.167:0/2988490943 >> [v2:172.21.15.204:6833/1247774569,v1:172.21.15.204:6835/1247774569] conn(0x55d84e936790 0x55d84e938b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:22.474 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.468+0000 7f60a1d9b640 1 --2- 172.21.15.167:0/2988490943 >> [v2:172.21.15.204:6833/1247774569,v1:172.21.15.204:6835/1247774569] conn(0x55d84e936790 0x55d84e938b80 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26322942b59d9f33 server_cookie=ceac0f7cc9acd59e in_seq=0 out_seq=0 2024-05-11T08:06:22.474 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.468+0000 7f60817fa640 10 client.4725 ms_handle_connect on v2:172.21.15.204:6833/1247774569 2024-05-11T08:06:22.474 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.468+0000 7f60817fa640 1 -- 172.21.15.167:0/2988490943 <== mds.0 v2:172.21.15.204:6833/1247774569 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d84e7bd760 con 0x55d84e936790 2024-05-11T08:06:22.474 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.468+0000 7f60817fa640 10 client.4725 handle_command_reply: tid=0 2024-05-11T08:06:22.474 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.468+0000 7f60abc2d740 1 client.4725 shutdown 2024-05-11T08:06:22.474 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.469+0000 7f60abc2d740 1 -- 172.21.15.167:0/2988490943 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d84e8fcff0 msgr2=0x55d84e8f6f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:22.474 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.469+0000 7f60abc2d740 1 --2- 172.21.15.167:0/2988490943 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d84e8fcff0 0x55d84e8f6f40 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f609402a6f0 tx=0x7f6094004730 comp rx=0 tx=0).stop 2024-05-11T08:06:22.474 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.469+0000 7f60abc2d740 1 -- 172.21.15.167:0/2988490943 shutdown_connections 2024-05-11T08:06:22.475 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.469+0000 7f60abc2d740 1 --2- 172.21.15.167:0/2988490943 >> [v2:172.21.15.204:6833/1247774569,v1:172.21.15.204:6835/1247774569] conn(0x55d84e936790 0x55d84e938b80 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:22.475 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.469+0000 7f60abc2d740 1 --2- 172.21.15.167:0/2988490943 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55d84e8f60c0 0x55d84e912370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:22.475 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.469+0000 7f60abc2d740 1 --2- 172.21.15.167:0/2988490943 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d84e8fd3d0 0x55d84e911e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:22.475 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.469+0000 7f60abc2d740 1 --2- 172.21.15.167:0/2988490943 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d84e8fcff0 0x55d84e8f6f40 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:22.475 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.469+0000 7f60817fa640 0 client.4725 ms_handle_reset on v2:172.21.15.204:6833/1247774569 2024-05-11T08:06:22.475 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.469+0000 7f60abc2d740 1 -- 172.21.15.167:0/2988490943 >> 172.21.15.167:0/2988490943 conn(0x55d84e8cfc50 msgr2=0x55d84e8fc2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:22.475 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.469+0000 7f60abc2d740 1 -- 172.21.15.167:0/2988490943 shutdown_connections 2024-05-11T08:06:22.475 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.469+0000 7f60abc2d740 1 -- 172.21.15.167:0/2988490943 wait complete. 2024-05-11T08:06:22.475 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.469+0000 7f60abc2d740 20 client.4725 trim_cache size 0 max 16384 2024-05-11T08:06:22.487 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.487+0000 7f60abc2d740 1 Processor -- start 2024-05-11T08:06:22.487 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.487+0000 7f60abc2d740 1 -- start start 2024-05-11T08:06:22.490 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.488+0000 7f60abc2d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d84e8f60c0 0x55d84e8f5220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:22.490 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.488+0000 7f60abc2d740 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55d84e8fcff0 0x55d84e917320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:22.490 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.488+0000 7f60abc2d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d84e8fd3d0 0x55d84e917860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:22.490 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.488+0000 7f60abc2d740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f60a4201b90 con 0x55d84e8fd3d0 2024-05-11T08:06:22.490 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.488+0000 7f60abc2d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f60a4155b80 con 0x55d84e8f60c0 2024-05-11T08:06:22.490 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.488+0000 7f60abc2d740 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f60a4201540 con 0x55d84e8fcff0 2024-05-11T08:06:22.490 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.488+0000 7f60a2d9d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d84e8fd3d0 0x55d84e917860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:22.490 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.488+0000 7f60a2d9d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d84e8fd3d0 0x55d84e917860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:54688/0 (socket says 172.21.15.167:54688) 2024-05-11T08:06:22.491 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.488+0000 7f60a2d9d640 1 -- 172.21.15.167:0/1134674199 learned_addr learned my addr 172.21.15.167:0/1134674199 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:22.491 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.488+0000 7f60a1d9b640 1 --2- 172.21.15.167:0/1134674199 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55d84e8fcff0 0x55d84e917320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:22.491 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.488+0000 7f60a259c640 1 --2- 172.21.15.167:0/1134674199 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d84e8f60c0 0x55d84e8f5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:22.491 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.488+0000 7f60a2d9d640 1 -- 172.21.15.167:0/1134674199 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55d84e8fcff0 msgr2=0x55d84e917320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:22.491 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.488+0000 7f60a2d9d640 1 --2- 172.21.15.167:0/1134674199 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55d84e8fcff0 0x55d84e917320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:22.491 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.488+0000 7f60a2d9d640 1 -- 172.21.15.167:0/1134674199 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d84e8f60c0 msgr2=0x55d84e8f5220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:22.491 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.488+0000 7f60a2d9d640 1 --2- 172.21.15.167:0/1134674199 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d84e8f60c0 0x55d84e8f5220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:22.491 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.488+0000 7f60a2d9d640 1 -- 172.21.15.167:0/1134674199 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d84e9126c0 con 0x55d84e8fd3d0 2024-05-11T08:06:22.491 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.488+0000 7f60a259c640 1 --2- 172.21.15.167:0/1134674199 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d84e8f60c0 0x55d84e8f5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T08:06:22.491 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.488+0000 7f60a1d9b640 1 --2- 172.21.15.167:0/1134674199 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55d84e8fcff0 0x55d84e917320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T08:06:22.491 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.488+0000 7f60a2d9d640 1 --2- 172.21.15.167:0/1134674199 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d84e8fd3d0 0x55d84e917860 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f609c060a20 tx=0x7f609c017040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d8d5a6880215453 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:22.491 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.488+0000 7f60837fe640 1 -- 172.21.15.167:0/1134674199 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f609c060a90 con 0x55d84e8fd3d0 2024-05-11T08:06:22.492 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.488+0000 7f60837fe640 1 -- 172.21.15.167:0/1134674199 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f609c064d90 con 0x55d84e8fd3d0 2024-05-11T08:06:22.492 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.488+0000 7f60837fe640 1 -- 172.21.15.167:0/1134674199 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f609c0695d0 con 0x55d84e8fd3d0 2024-05-11T08:06:22.492 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.489+0000 7f60abc2d740 1 -- 172.21.15.167:0/1134674199 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d84e8fd3d0 msgr2=0x55d84e917860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:22.492 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.489+0000 7f60abc2d740 1 --2- 172.21.15.167:0/1134674199 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d84e8fd3d0 0x55d84e917860 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f609c060a20 tx=0x7f609c017040 comp rx=0 tx=0).stop 2024-05-11T08:06:22.492 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.489+0000 7f60abc2d740 1 -- 172.21.15.167:0/1134674199 shutdown_connections 2024-05-11T08:06:22.492 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.489+0000 7f60abc2d740 1 --2- 172.21.15.167:0/1134674199 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d84e8fd3d0 0x55d84e917860 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:22.492 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.489+0000 7f60abc2d740 1 --2- 172.21.15.167:0/1134674199 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55d84e8fcff0 0x55d84e917320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:22.492 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.489+0000 7f60abc2d740 1 --2- 172.21.15.167:0/1134674199 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d84e8f60c0 0x55d84e8f5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:22.492 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.489+0000 7f60abc2d740 1 -- 172.21.15.167:0/1134674199 >> 172.21.15.167:0/1134674199 conn(0x55d84e8cfc50 msgr2=0x55d84e8d4680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:22.493 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.489+0000 7f60abc2d740 1 -- 172.21.15.167:0/1134674199 shutdown_connections 2024-05-11T08:06:22.493 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.489+0000 7f60abc2d740 1 -- 172.21.15.167:0/1134674199 wait complete. 2024-05-11T08:06:22.493 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.490+0000 7f60abc2d740 1 -- start start 2024-05-11T08:06:22.493 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.490+0000 7f60abc2d740 1 Processor -- start 2024-05-11T08:06:22.497 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.491+0000 7f60abc2d740 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55d84e8f60c0 0x55d84e917720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:22.497 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.491+0000 7f60abc2d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d84e8fcff0 0x55d84e98a230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:22.497 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.491+0000 7f60abc2d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d84e8fd3d0 0x55d84e98a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:22.497 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.491+0000 7f60abc2d740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55d84e912f00 con 0x55d84e8fcff0 2024-05-11T08:06:22.497 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.491+0000 7f60abc2d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55d84e8f7b50 con 0x55d84e8fd3d0 2024-05-11T08:06:22.497 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.491+0000 7f60abc2d740 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x55d84e9348e0 con 0x55d84e8f60c0 2024-05-11T08:06:22.497 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.491+0000 7f60abc2d740 10 client.0 _ll_register_callbacks cb 0x7f6094009720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T08:06:22.497 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.491+0000 7f60a1d9b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d84e8fcff0 0x55d84e98a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:22.497 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.491+0000 7f60a1d9b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d84e8fcff0 0x55d84e98a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:54704/0 (socket says 172.21.15.167:54704) 2024-05-11T08:06:22.497 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.491+0000 7f60a1d9b640 1 -- 172.21.15.167:0/2026708395 learned_addr learned my addr 172.21.15.167:0/2026708395 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:22.498 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.491+0000 7f60a259c640 1 --2- 172.21.15.167:0/2026708395 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55d84e8f60c0 0x55d84e917720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:22.498 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.491+0000 7f60a2d9d640 1 --2- 172.21.15.167:0/2026708395 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d84e8fd3d0 0x55d84e98a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:22.498 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.491+0000 7f60a1d9b640 1 -- 172.21.15.167:0/2026708395 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55d84e8f60c0 msgr2=0x55d84e917720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:22.498 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.491+0000 7f60a1d9b640 1 --2- 172.21.15.167:0/2026708395 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55d84e8f60c0 0x55d84e917720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:22.498 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.491+0000 7f60a1d9b640 1 -- 172.21.15.167:0/2026708395 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d84e8fd3d0 msgr2=0x55d84e98a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:22.498 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.491+0000 7f60a1d9b640 1 --2- 172.21.15.167:0/2026708395 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d84e8fd3d0 0x55d84e98a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:22.498 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.491+0000 7f60a1d9b640 1 -- 172.21.15.167:0/2026708395 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d84e917c60 con 0x55d84e8fcff0 2024-05-11T08:06:22.498 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.491+0000 7f60a1d9b640 1 --2- 172.21.15.167:0/2026708395 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d84e8fcff0 0x55d84e98a230 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f608c0070e0 tx=0x7f608c000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f045bc5ea3909769 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:22.498 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.491+0000 7f60817fa640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-05-11T08:06:22.498 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.491+0000 7f60817fa640 1 -- 172.21.15.167:0/2026708395 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f608c001d00 con 0x55d84e8fcff0 2024-05-11T08:06:22.498 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.491+0000 7f60817fa640 1 -- 172.21.15.167:0/2026708395 <== mon.0 v2:172.21.15.167:3300/0 2 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f608c02ca30 con 0x55d84e8fcff0 2024-05-11T08:06:22.498 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.491+0000 7f60abc2d740 1 -- 172.21.15.167:0/2026708395 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f609c064d90 con 0x55d84e8fcff0 2024-05-11T08:06:22.498 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.491+0000 7f60817fa640 1 -- 172.21.15.167:0/2026708395 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) ==== 24+0+0 (secure 0 0 0) 0x7f608c02cc30 con 0x55d84e8fcff0 2024-05-11T08:06:22.498 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.492+0000 7f60abc2d740 10 client.4731 fetch_fsmap learned FSMap version 9 2024-05-11T08:06:22.498 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.492+0000 7f60abc2d740 1 -- 172.21.15.167:0/2026708395 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) -- 0x55d84e925d00 con 0x55d84e8fcff0 2024-05-11T08:06:22.499 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.492+0000 7f60817fa640 1 -- 172.21.15.167:0/2026708395 <== mon.0 v2:172.21.15.167:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f609c064d90 con 0x55d84e8fcff0 2024-05-11T08:06:22.499 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.492+0000 7f60817fa640 1 -- 172.21.15.167:0/2026708395 <== mon.0 v2:172.21.15.167:3300/0 5 ==== fsmap(e 9) ==== 3502+0+0 (secure 0 0 0) 0x7f608c02a3b0 con 0x55d84e8fcff0 2024-05-11T08:06:22.499 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.492+0000 7f60abc2d740 10 client.4731 fetch_fsmap finished waiting for FSMap version 9 2024-05-11T08:06:22.499 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.492+0000 7f60abc2d740 10 client.4731 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-11T08:06:22.499 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.492+0000 7f60abc2d740 20 client.4731 populate_metadata read hostname 'smithi167' 2024-05-11T08:06:22.499 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.492+0000 7f60abc2d740 1 --2- 172.21.15.167:0/2026708395 >> [v2:172.21.15.204:6833/1247774569,v1:172.21.15.204:6835/1247774569] conn(0x55d84e9280b0 0x55d84e902680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:22.499 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.492+0000 7f60abc2d740 4 client.4731 mds_command: new command op to 4305 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-11T08:06:22.499 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.492+0000 7f60abc2d740 1 -- 172.21.15.167:0/2026708395 --> [v2:172.21.15.204:6833/1247774569,v1:172.21.15.204:6835/1247774569] -- command(tid 0: {"prefix": "status"}) -- 0x55d84e8474e0 con 0x55d84e9280b0 2024-05-11T08:06:22.499 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.492+0000 7f60a259c640 1 --2- 172.21.15.167:0/2026708395 >> [v2:172.21.15.204:6833/1247774569,v1:172.21.15.204:6835/1247774569] conn(0x55d84e9280b0 0x55d84e902680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:22.499 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.492+0000 7f60a259c640 1 --2- 172.21.15.167:0/2026708395 >> [v2:172.21.15.204:6833/1247774569,v1:172.21.15.204:6835/1247774569] conn(0x55d84e9280b0 0x55d84e902680 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=18aca23578e89eed server_cookie=20737c6afa0d1ade in_seq=0 out_seq=0 2024-05-11T08:06:22.499 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.492+0000 7f60817fa640 10 client.4731 ms_handle_connect on v2:172.21.15.204:6833/1247774569 2024-05-11T08:06:22.499 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.493+0000 7f60817fa640 1 -- 172.21.15.167:0/2026708395 <== mds.0 v2:172.21.15.204:6833/1247774569 1 ==== command_reply(tid 0: 0 ) ==== 8+0+330 (crc 0 0 0) 0x55d84e8474e0 con 0x55d84e9280b0 2024-05-11T08:06:22.499 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.493+0000 7f60817fa640 10 client.4731 handle_command_reply: tid=0 2024-05-11T08:06:22.499 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.493+0000 7f60abc2d740 1 client.4731 shutdown 2024-05-11T08:06:22.499 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.493+0000 7f60abc2d740 1 -- 172.21.15.167:0/2026708395 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d84e8fcff0 msgr2=0x55d84e98a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:22.500 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.493+0000 7f60abc2d740 1 --2- 172.21.15.167:0/2026708395 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d84e8fcff0 0x55d84e98a230 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f608c0070e0 tx=0x7f608c000ed0 comp rx=0 tx=0).stop 2024-05-11T08:06:22.500 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.493+0000 7f60abc2d740 1 -- 172.21.15.167:0/2026708395 shutdown_connections 2024-05-11T08:06:22.500 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.493+0000 7f60abc2d740 1 --2- 172.21.15.167:0/2026708395 >> [v2:172.21.15.204:6833/1247774569,v1:172.21.15.204:6835/1247774569] conn(0x55d84e9280b0 0x55d84e902680 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:22.500 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.493+0000 7f60abc2d740 1 --2- 172.21.15.167:0/2026708395 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d84e8fd3d0 0x55d84e98a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:22.500 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.493+0000 7f60abc2d740 1 --2- 172.21.15.167:0/2026708395 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d84e8fcff0 0x55d84e98a230 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:22.500 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.493+0000 7f60817fa640 0 client.4731 ms_handle_reset on v2:172.21.15.204:6833/1247774569 2024-05-11T08:06:22.500 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.493+0000 7f60abc2d740 1 --2- 172.21.15.167:0/2026708395 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55d84e8f60c0 0x55d84e917720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:22.500 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.493+0000 7f60abc2d740 1 -- 172.21.15.167:0/2026708395 >> 172.21.15.167:0/2026708395 conn(0x55d84e8cfc50 msgr2=0x55d84e8cefc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:22.500 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.493+0000 7f60abc2d740 1 -- 172.21.15.167:0/2026708395 shutdown_connections 2024-05-11T08:06:22.500 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.493+0000 7f60abc2d740 1 -- 172.21.15.167:0/2026708395 wait complete. 2024-05-11T08:06:22.500 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.493+0000 7f60abc2d740 20 client.4731 trim_cache size 0 max 16384 2024-05-11T08:06:22.500 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-05-11T08:06:22.501 INFO:teuthology.orchestra.run.smithi167.stdout: "cluster_fsid": "2034778d-8ff1-4823-aa44-ded32b518241", 2024-05-11T08:06:22.501 INFO:teuthology.orchestra.run.smithi167.stdout: "whoami": 0, 2024-05-11T08:06:22.501 INFO:teuthology.orchestra.run.smithi167.stdout: "id": 4305, 2024-05-11T08:06:22.501 INFO:teuthology.orchestra.run.smithi167.stdout: "want_state": "up:active", 2024-05-11T08:06:22.501 INFO:teuthology.orchestra.run.smithi167.stdout: "state": "up:active", 2024-05-11T08:06:22.501 INFO:teuthology.orchestra.run.smithi167.stdout: "fs_name": "cephfs", 2024-05-11T08:06:22.501 INFO:teuthology.orchestra.run.smithi167.stdout: "rank_uptime": 27.142938147999999, 2024-05-11T08:06:22.501 INFO:teuthology.orchestra.run.smithi167.stdout: "mdsmap_epoch": 9, 2024-05-11T08:06:22.501 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_epoch": 27, 2024-05-11T08:06:22.501 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_epoch_barrier": 27, 2024-05-11T08:06:22.501 INFO:teuthology.orchestra.run.smithi167.stdout: "uptime": 33.157775479000001 2024-05-11T08:06:22.501 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-05-11T08:06:22.502 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.493+0000 7f60a8e37640 1 -- 172.21.15.167:0/1607748287 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f60840762d0 msgr2=0x7f6084078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:22.502 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.493+0000 7f60a8e37640 1 --2- 172.21.15.167:0/1607748287 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f60840762d0 0x7f6084078790 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f6094034000 tx=0x7f609402e000 comp rx=0 tx=0).stop 2024-05-11T08:06:22.502 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.493+0000 7f60a8e37640 1 -- 172.21.15.167:0/1607748287 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f60a415f360 msgr2=0x7f60a4147530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:22.502 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.493+0000 7f60a8e37640 1 --2- 172.21.15.167:0/1607748287 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f60a415f360 0x7f60a4147530 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f609c007ef0 tx=0x7f609c00d4a0 comp rx=0 tx=0).stop 2024-05-11T08:06:22.502 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.494+0000 7f60a8e37640 1 -- 172.21.15.167:0/1607748287 shutdown_connections 2024-05-11T08:06:22.503 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.494+0000 7f60a8e37640 1 --2- 172.21.15.167:0/1607748287 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f60840762d0 0x7f6084078790 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:22.503 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.494+0000 7f60a8e37640 1 --2- 172.21.15.167:0/1607748287 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f60a415f360 0x7f60a4147530 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:22.503 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.494+0000 7f60a8e37640 1 --2- 172.21.15.167:0/1607748287 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f60a40cd610 0x7f60a415b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:22.503 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.494+0000 7f60a8e37640 1 --2- 172.21.15.167:0/1607748287 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f60a40b5e90 0x7f60a4158560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:22.503 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.494+0000 7f60a8e37640 1 -- 172.21.15.167:0/1607748287 >> 172.21.15.167:0/1607748287 conn(0x7f60a4072730 msgr2=0x7f60a415e5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:22.503 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.494+0000 7f60a8e37640 1 -- 172.21.15.167:0/1607748287 shutdown_connections 2024-05-11T08:06:22.503 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:22.494+0000 7f60a8e37640 1 -- 172.21.15.167:0/1607748287 wait complete. 2024-05-11T08:06:22.505 INFO:teuthology.run_tasks:Running task sequential... 2024-05-11T08:06:22.514 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-05-11T08:06:22.515 INFO:teuthology.task.sequential:In sequential, running task print... 2024-05-11T08:06:22.515 INFO:teuthology.task.print:Enabling mgr modules 2024-05-11T08:06:22.515 INFO:teuthology.run_tasks:Running task kclient... 2024-05-11T08:06:22.551 INFO:tasks.kclient:Mounting kernel clients... 2024-05-11T08:06:22.551 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-05-11T08:06:22.552 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T08:06:22.552 DEBUG:teuthology.orchestra.run.smithi167:> ip netns list 2024-05-11T08:06:22.567 INFO:teuthology.orchestra.run.smithi167.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-05-11T08:06:22.568 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T08:06:22.568 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-11T08:06:22.633 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T08:06:22.633 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip link delete ceph-brx 2024-05-11T08:06:22.722 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T08:06:22.723 DEBUG:teuthology.orchestra.run.smithi167:> ip netns list 2024-05-11T08:06:22.738 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T08:06:22.738 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip link delete ceph-brx 2024-05-11T08:06:22.806 INFO:teuthology.orchestra.run.smithi167.stderr:Cannot find device "ceph-brx" 2024-05-11T08:06:22.807 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-11T08:06:22.807 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T08:06:22.808 DEBUG:teuthology.orchestra.run.smithi204:> ip netns list 2024-05-11T08:06:22.827 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T08:06:22.827 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip link delete ceph-brx 2024-05-11T08:06:22.901 INFO:teuthology.orchestra.run.smithi204.stderr:Cannot find device "ceph-brx" 2024-05-11T08:06:22.902 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-11T08:06:22.902 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T08:06:22.902 DEBUG:teuthology.orchestra.run.smithi204:> ip netns list 2024-05-11T08:06:22.961 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T08:06:22.961 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip link delete ceph-brx 2024-05-11T08:06:23.031 INFO:teuthology.orchestra.run.smithi204.stderr:Cannot find device "ceph-brx" 2024-05-11T08:06:23.032 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-11T08:06:23.033 INFO:tasks.kclient:client.0 config is {'syntax': 'v1'} 2024-05-11T08:06:23.033 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-11T08:06:23.033 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-11T08:06:23.033 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-11T08:06:23.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:23.136+0000 7fbd592e1640 1 Processor -- start 2024-05-11T08:06:23.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:23.136+0000 7fbd592e1640 1 -- start start 2024-05-11T08:06:23.138 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:23.137+0000 7fbd592e1640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbd540b5a10 0x7fbd540b5e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:23.138 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:23.137+0000 7fbd592e1640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd54163e60 con 0x7fbd541488a0 2024-05-11T08:06:23.138 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:23.137+0000 7fbd592e1640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd540b4f70 con 0x7fbd540710f0 2024-05-11T08:06:23.138 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:23.137+0000 7fbd592e1640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fbd540b63b0 con 0x7fbd540b5a10 2024-05-11T08:06:23.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:23.137+0000 7fbd52ffd640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7fbd541488a0 0x7fbd5415f910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:60946/0 (socket says 172.21.15.167:60946) 2024-05-11T08:06:23.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:23.137+0000 7fbd52ffd640 1 -- 172.21.15.167:0/3845751737 learned_addr learned my addr 172.21.15.167:0/3845751737 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T08:06:23.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:23.137+0000 7fbd527fc640 1 --2- 172.21.15.167:0/3845751737 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbd540b5a10 0x7fbd540b5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:23.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:23.137+0000 7fbd51ffb640 1 -- 172.21.15.167:0/3845751737 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 72500611 0 0) 0x7fbd54163e60 con 0x7fbd541488a0 2024-05-11T08:06:23.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:23.137+0000 7fbd51ffb640 1 -- 172.21.15.167:0/3845751737 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbd3c003520 con 0x7fbd541488a0 2024-05-11T08:06:23.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:23.138+0000 7fbd51ffb640 1 -- 172.21.15.167:0/3845751737 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1571021920 0 0) 0x7fbd540b4f70 con 0x7fbd540710f0 2024-05-11T08:06:23.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:23.138+0000 7fbd51ffb640 1 -- 172.21.15.167:0/3845751737 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbd54163e60 con 0x7fbd540710f0 2024-05-11T08:06:23.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:23.138+0000 7fbd51ffb640 1 -- 172.21.15.167:0/3845751737 <== mon.0 v1:172.21.15.167:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 450478063 0 0) 0x7fbd3c003520 con 0x7fbd541488a0 2024-05-11T08:06:23.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:23.138+0000 7fbd527fc640 1 -- 172.21.15.167:0/3845751737 >> v1:172.21.15.204:6789/0 conn(0x7fbd540710f0 legacy=0x7fbd54162f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:23.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:23.138+0000 7fbd527fc640 1 -- 172.21.15.167:0/3845751737 >> v1:172.21.15.167:6789/0 conn(0x7fbd541488a0 legacy=0x7fbd5415f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:23.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:23.138+0000 7fbd527fc640 1 -- 172.21.15.167:0/3845751737 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd54163b80 con 0x7fbd540b5a10 2024-05-11T08:06:23.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:23.138+0000 7fbd527fc640 1 --2- 172.21.15.167:0/3845751737 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbd540b5a10 0x7fbd540b5e70 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fbd540b5140 tx=0x7fbd4802a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf6b8f1cff569aa2 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:23.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:23.139+0000 7fbd51ffb640 1 -- 172.21.15.167:0/3845751737 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd48037070 con 0x7fbd540b5a10 2024-05-11T08:06:23.140 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:23.139+0000 7fbd51ffb640 1 -- 172.21.15.167:0/3845751737 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd4802ab90 con 0x7fbd540b5a10 2024-05-11T08:06:23.140 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:23.139+0000 7fbd51ffb640 1 -- 172.21.15.167:0/3845751737 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd44001a70 con 0x7fbd540b5a10 2024-05-11T08:06:23.141 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:23.139+0000 7fbd592e1640 1 -- 172.21.15.167:0/3845751737 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbd540b5a10 msgr2=0x7fbd540b5e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:23.141 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:23.139+0000 7fbd592e1640 1 --2- 172.21.15.167:0/3845751737 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbd540b5a10 0x7fbd540b5e70 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fbd540b5140 tx=0x7fbd4802a100 comp rx=0 tx=0).stop 2024-05-11T08:06:23.141 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:23.139+0000 7fbd592e1640 1 -- 172.21.15.167:0/3845751737 shutdown_connections 2024-05-11T08:06:23.141 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:23.139+0000 7fbd592e1640 1 --2- 172.21.15.167:0/3845751737 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbd540b5a10 0x7fbd540b5e70 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:23.141 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:23.139+0000 7fbd592e1640 1 -- 172.21.15.167:0/3845751737 >> 172.21.15.167:0/3845751737 conn(0x7fbd540d2550 msgr2=0x7fbd540ce890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:23.141 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:23.139+0000 7fbd592e1640 1 -- 172.21.15.167:0/3845751737 shutdown_connections 2024-05-11T08:06:23.141 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:23.139+0000 7fbd592e1640 1 -- 172.21.15.167:0/3845751737 wait complete. 2024-05-11T08:06:23.141 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:23.141+0000 7fbd592e1640 1 Processor -- start 2024-05-11T08:06:23.141 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:23.141+0000 7fbd592e1640 1 -- start start 2024-05-11T08:06:23.143 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:23.141+0000 7fbd592e1640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbd540710f0 0x7fbd5413e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:23.143 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:23.141+0000 7fbd592e1640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbd540b5a10 0x7fbd54140590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:23.143 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:23.141+0000 7fbd592e1640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbd541488a0 0x7fbd54162230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:23.144 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:23.141+0000 7fbd592e1640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fbd54160020 con 0x7fbd540b5a10 2024-05-11T08:06:23.144 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:23.141+0000 7fbd592e1640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fbd540b5580 con 0x7fbd540710f0 2024-05-11T08:06:23.144 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:23.141+0000 7fbd592e1640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fbd54148c80 con 0x7fbd541488a0 2024-05-11T08:06:23.144 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:23.141+0000 7fbd527fc640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbd540b5a10 0x7fbd54140590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:23.144 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:23.141+0000 7fbd527fc640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbd540b5a10 0x7fbd54140590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:54710/0 (socket says 172.21.15.167:54710) 2024-05-11T08:06:23.144 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:23.141+0000 7fbd527fc640 1 -- 172.21.15.167:0/2257386970 learned_addr learned my addr 172.21.15.167:0/2257386970 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:23.144 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:23.142+0000 7fbd52ffd640 1 --2- 172.21.15.167:0/2257386970 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbd540710f0 0x7fbd5413e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:23.144 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:23.142+0000 7fbd537fe640 1 --2- 172.21.15.167:0/2257386970 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbd541488a0 0x7fbd54162230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:23.144 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:23.142+0000 7fbd527fc640 1 -- 172.21.15.167:0/2257386970 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbd541488a0 msgr2=0x7fbd54162230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:23.144 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:23.142+0000 7fbd527fc640 1 --2- 172.21.15.167:0/2257386970 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbd541488a0 0x7fbd54162230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:23.144 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:23.142+0000 7fbd527fc640 1 -- 172.21.15.167:0/2257386970 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbd540710f0 msgr2=0x7fbd5413e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:23.144 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:23.142+0000 7fbd527fc640 1 --2- 172.21.15.167:0/2257386970 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbd540710f0 0x7fbd5413e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:23.144 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:23.142+0000 7fbd527fc640 1 -- 172.21.15.167:0/2257386970 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd5413edd0 con 0x7fbd540b5a10 2024-05-11T08:06:23.144 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:23.142+0000 7fbd527fc640 1 --2- 172.21.15.167:0/2257386970 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbd540b5a10 0x7fbd54140590 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fbd480098c0 tx=0x7fbd480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:23.144 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:23.142+0000 7fbd3b7fe640 1 -- 172.21.15.167:0/2257386970 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd48037050 con 0x7fbd540b5a10 2024-05-11T08:06:23.145 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:23.142+0000 7fbd3b7fe640 1 -- 172.21.15.167:0/2257386970 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd48043070 con 0x7fbd540b5a10 2024-05-11T08:06:23.145 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:23.142+0000 7fbd592e1640 1 -- 172.21.15.167:0/2257386970 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd5413f0b0 con 0x7fbd540b5a10 2024-05-11T08:06:23.145 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:23.142+0000 7fbd3b7fe640 1 -- 172.21.15.167:0/2257386970 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd48006e10 con 0x7fbd540b5a10 2024-05-11T08:06:23.145 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:23.142+0000 7fbd592e1640 1 -- 172.21.15.167:0/2257386970 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd54162770 con 0x7fbd540b5a10 2024-05-11T08:06:23.145 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:23.143+0000 7fbd592e1640 1 -- 172.21.15.167:0/2257386970 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd54069590 con 0x7fbd540b5a10 2024-05-11T08:06:23.145 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:23.144+0000 7fbd3b7fe640 1 -- 172.21.15.167:0/2257386970 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fbd4803ca10 con 0x7fbd540b5a10 2024-05-11T08:06:23.146 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:23.144+0000 7fbd3b7fe640 1 --2- 172.21.15.167:0/2257386970 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fbd400762d0 0x7fbd40078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:23.146 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:23.144+0000 7fbd52ffd640 1 --2- 172.21.15.167:0/2257386970 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fbd400762d0 0x7fbd40078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:23.146 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:23.144+0000 7fbd3b7fe640 1 -- 172.21.15.167:0/2257386970 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fbd480b7fb0 con 0x7fbd540b5a10 2024-05-11T08:06:23.146 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:23.145+0000 7fbd52ffd640 1 --2- 172.21.15.167:0/2257386970 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fbd400762d0 0x7fbd40078790 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fbd440012e0 tx=0x7fbd44009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:23.148 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:23.149+0000 7fbd3b7fe640 1 -- 172.21.15.167:0/2257386970 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fbd4804d020 con 0x7fbd540b5a10 2024-05-11T08:06:23.369 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:23.369+0000 7fbd592e1640 1 -- 172.21.15.167:0/2257386970 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fbd540d1220 con 0x7fbd540b5a10 2024-05-11T08:06:23.369 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:23.369+0000 7fbd3b7fe640 1 -- 172.21.15.167:0/2257386970 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7fbd48081010 con 0x7fbd540b5a10 2024-05-11T08:06:23.369 INFO:teuthology.orchestra.run.smithi167.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-05-11T08:06:23.371 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:23.371+0000 7fbd592e1640 1 -- 172.21.15.167:0/2257386970 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fbd400762d0 msgr2=0x7fbd40078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:23.371 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:23.371+0000 7fbd592e1640 1 --2- 172.21.15.167:0/2257386970 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fbd400762d0 0x7fbd40078790 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fbd440012e0 tx=0x7fbd44009040 comp rx=0 tx=0).stop 2024-05-11T08:06:23.372 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:23.371+0000 7fbd592e1640 1 -- 172.21.15.167:0/2257386970 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbd540b5a10 msgr2=0x7fbd54140590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:23.372 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:23.371+0000 7fbd592e1640 1 --2- 172.21.15.167:0/2257386970 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbd540b5a10 0x7fbd54140590 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fbd480098c0 tx=0x7fbd480046c0 comp rx=0 tx=0).stop 2024-05-11T08:06:23.372 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:23.371+0000 7fbd592e1640 1 -- 172.21.15.167:0/2257386970 shutdown_connections 2024-05-11T08:06:23.372 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:23.371+0000 7fbd592e1640 1 --2- 172.21.15.167:0/2257386970 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fbd400762d0 0x7fbd40078790 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:23.372 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:23.371+0000 7fbd592e1640 1 --2- 172.21.15.167:0/2257386970 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbd541488a0 0x7fbd54162230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:23.372 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:23.371+0000 7fbd592e1640 1 --2- 172.21.15.167:0/2257386970 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbd540b5a10 0x7fbd54140590 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:23.372 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:23.371+0000 7fbd592e1640 1 --2- 172.21.15.167:0/2257386970 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbd540710f0 0x7fbd5413e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:23.372 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:23.371+0000 7fbd592e1640 1 -- 172.21.15.167:0/2257386970 >> 172.21.15.167:0/2257386970 conn(0x7fbd540d2550 msgr2=0x7fbd54162d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:23.372 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:23.371+0000 7fbd592e1640 1 -- 172.21.15.167:0/2257386970 shutdown_connections 2024-05-11T08:06:23.372 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:23.371+0000 7fbd592e1640 1 -- 172.21.15.167:0/2257386970 wait complete. 2024-05-11T08:06:23.382 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-11T08:06:23.383 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-11T08:06:23.383 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi167.front.sepia.ceph.com 2024-05-11T08:06:23.383 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-11T08:06:23.383 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-11T08:06:23.383 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-11T08:06:23.383 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-11T08:06:23.383 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-11T08:06:23.383 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-11T08:06:23.383 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T08:06:23.383 DEBUG:teuthology.orchestra.run.smithi167:> ip addr 2024-05-11T08:06:23.399 INFO:teuthology.orchestra.run.smithi167.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-11T08:06:23.399 INFO:teuthology.orchestra.run.smithi167.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-11T08:06:23.399 INFO:teuthology.orchestra.run.smithi167.stdout: inet 127.0.0.1/8 scope host lo 2024-05-11T08:06:23.399 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2024-05-11T08:06:23.400 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 ::1/128 scope host 2024-05-11T08:06:23.400 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2024-05-11T08:06:23.400 INFO:teuthology.orchestra.run.smithi167.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-11T08:06:23.400 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:d9:35:e6 brd ff:ff:ff:ff:ff:ff 2024-05-11T08:06:23.400 INFO:teuthology.orchestra.run.smithi167.stdout: altname enp7s0f0 2024-05-11T08:06:23.400 INFO:teuthology.orchestra.run.smithi167.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-11T08:06:23.400 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:d9:35:e7 brd ff:ff:ff:ff:ff:ff 2024-05-11T08:06:23.400 INFO:teuthology.orchestra.run.smithi167.stdout: altname enp7s0f1 2024-05-11T08:06:23.400 INFO:teuthology.orchestra.run.smithi167.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-11T08:06:23.400 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:88:73:08 brd ff:ff:ff:ff:ff:ff 2024-05-11T08:06:23.400 INFO:teuthology.orchestra.run.smithi167.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-11T08:06:23.400 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:88:73:09 brd ff:ff:ff:ff:ff:ff 2024-05-11T08:06:23.400 INFO:teuthology.orchestra.run.smithi167.stdout: inet 172.21.15.167/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-05-11T08:06:23.400 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft 42700sec preferred_lft 42700sec 2024-05-11T08:06:23.401 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::ec4:7aff:fe88:7309/64 scope link 2024-05-11T08:06:23.401 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2024-05-11T08:06:23.401 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-11T08:06:23.401 DEBUG:teuthology.orchestra.run.smithi167:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-11T08:06:23.401 DEBUG:teuthology.orchestra.run.smithi167:> set -e 2024-05-11T08:06:23.401 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip link add name ceph-brx type bridge 2024-05-11T08:06:23.401 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip addr flush dev ceph-brx 2024-05-11T08:06:23.401 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip link set ceph-brx up 2024-05-11T08:06:23.401 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-11T08:06:23.401 DEBUG:teuthology.orchestra.run.smithi167:> ') 2024-05-11T08:06:23.490 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:23 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-11T08:06:23.533 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T08:06:23.539 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T08:06:23.539 DEBUG:teuthology.orchestra.run.smithi167:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-11T08:06:23.563 INFO:teuthology.orchestra.run.smithi167.stdout:1 2024-05-11T08:06:23.565 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T08:06:23.565 DEBUG:teuthology.orchestra.run.smithi167:> route 2024-05-11T08:06:23.621 INFO:teuthology.orchestra.run.smithi167.stdout:Kernel IP routing table 2024-05-11T08:06:23.621 INFO:teuthology.orchestra.run.smithi167.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-11T08:06:23.621 INFO:teuthology.orchestra.run.smithi167.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-11T08:06:23.621 INFO:teuthology.orchestra.run.smithi167.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-11T08:06:23.621 INFO:teuthology.orchestra.run.smithi167.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-11T08:06:23.621 DEBUG:teuthology.orchestra.run.smithi167:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-11T08:06:23.621 DEBUG:teuthology.orchestra.run.smithi167:> set -e 2024-05-11T08:06:23.621 DEBUG:teuthology.orchestra.run.smithi167:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-11T08:06:23.622 DEBUG:teuthology.orchestra.run.smithi167:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-11T08:06:23.622 DEBUG:teuthology.orchestra.run.smithi167:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-11T08:06:23.622 DEBUG:teuthology.orchestra.run.smithi167:> ') 2024-05-11T08:06:23.709 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:23 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-11T08:06:23.742 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T08:06:23.748 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T08:06:23.748 DEBUG:teuthology.orchestra.run.smithi167:> ip netns list 2024-05-11T08:06:23.764 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T08:06:23.764 DEBUG:teuthology.orchestra.run.smithi167:> ip netns list-id 2024-05-11T08:06:23.819 DEBUG:teuthology.orchestra.run.smithi167:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-11T08:06:23.819 DEBUG:teuthology.orchestra.run.smithi167:> set -e 2024-05-11T08:06:23.820 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-11T08:06:23.820 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-05-11T08:06:23.820 DEBUG:teuthology.orchestra.run.smithi167:> ') 2024-05-11T08:06:23.908 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:23 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-11T08:06:23.931 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T08:06:23.938 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-05-11T08:06:23.938 DEBUG:teuthology.orchestra.run.smithi167:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-11T08:06:23.938 DEBUG:teuthology.orchestra.run.smithi167:> set -e 2024-05-11T08:06:23.938 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-05-11T08:06:23.938 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-11T08:06:23.938 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-05-11T08:06:23.938 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-05-11T08:06:23.938 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-05-11T08:06:23.938 DEBUG:teuthology.orchestra.run.smithi167:> ') 2024-05-11T08:06:23.987 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:23 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-11T08:06:24.046 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T08:06:24.053 DEBUG:teuthology.orchestra.run.smithi167:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-11T08:06:24.053 DEBUG:teuthology.orchestra.run.smithi167:> set -e 2024-05-11T08:06:24.053 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip link set brx.0 up 2024-05-11T08:06:24.053 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip link set dev brx.0 master ceph-brx 2024-05-11T08:06:24.053 DEBUG:teuthology.orchestra.run.smithi167:> ') 2024-05-11T08:06:24.102 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:24 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-11T08:06:24.124 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T08:06:24.130 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-11T08:06:24.130 DEBUG:teuthology.orchestra.run.smithi167:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-11T08:06:24.145 INFO:teuthology.orchestra.run.smithi167.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-11T08:06:24.145 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-11T08:06:24.145 DEBUG:teuthology.orchestra.run.smithi167:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-11T08:06:24.200 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-05-11T08:06:24.200 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T08:06:24.201 DEBUG:teuthology.orchestra.run.smithi167:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-11T08:06:24.284 INFO:teuthology.orchestra.run.smithi167.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-11T08:06:24.284 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-11T08:06:24.284 INFO:teuthology.orchestra.run.smithi167.stdout:invalid new device string format 2024-05-11T08:06:24.392 INFO:teuthology.orchestra.run.smithi167.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-11T08:06:24.610 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-11T08:06:24.610 INFO:teuthology.orchestra.run.smithi167.stdout:invalid new device string format 2024-05-11T08:06:24.610 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: resolved to: "172.21.15.167:3300,172.21.15.204:3300,172.21.15.204:3301" 2024-05-11T08:06:24.610 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-11T08:06:24.612 INFO:teuthology.orchestra.run.smithi167.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-11T08:06:24.612 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-11T08:06:24.612 INFO:teuthology.orchestra.run.smithi167.stdout:invalid new device string format 2024-05-11T08:06:24.612 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: resolved to: "172.21.15.167:3300,172.21.15.204:3300,172.21.15.204:3301" 2024-05-11T08:06:24.612 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-11T08:06:24.612 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: resolved to: "172.21.15.167:6789,172.21.15.204:6789,172.21.15.204:6790" 2024-05-11T08:06:24.612 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.167:6789,172.21.15.204:6789,172.21.15.204:6790:/ 2024-05-11T08:06:24.612 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=2034778d-8ff1-4823-aa44-ded32b518241" will pass to kernel 2024-05-11T08:06:24.612 INFO:teuthology.orchestra.run.smithi167.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-05-11T08:06:24.612 INFO:teuthology.orchestra.run.smithi167.stdout: You just mounted a file system that supports labels which does not 2024-05-11T08:06:24.612 INFO:teuthology.orchestra.run.smithi167.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-11T08:06:24.613 INFO:teuthology.orchestra.run.smithi167.stdout: applications will generate AVC messages and not be allowed access to 2024-05-11T08:06:24.613 INFO:teuthology.orchestra.run.smithi167.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-11T08:06:24.613 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-11T08:06:24.613 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T08:06:24.613 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-11T08:06:24.677 DEBUG:teuthology.orchestra.run.smithi167:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-11T08:06:24.767 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:24 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-11T08:06:24.769 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T08:06:24.775 INFO:teuthology.orchestra.run.smithi167.stdout:client4585 2024-05-11T08:06:24.775 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-05-11T08:06:24.775 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T08:06:24.791 DEBUG:teuthology.orchestra.run.smithi167:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-11T08:06:24.879 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:24 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-11T08:06:24.881 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T08:06:24.887 INFO:teuthology.orchestra.run.smithi167.stdout:client4585 2024-05-11T08:06:24.887 DEBUG:teuthology.orchestra.run.smithi167:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-11T08:06:24.935 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:24 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-11T08:06:24.937 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T08:06:24.943 INFO:teuthology.orchestra.run.smithi167.stdout:client4585 2024-05-11T08:06:24.943 DEBUG:teuthology.orchestra.run.smithi167:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/2034778d-8ff1-4823-aa44-ded32b518241.client4585/status') 2024-05-11T08:06:24.991 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:24 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-11T08:06:25.002 INFO:teuthology.orchestra.run.smithi167.stdout:instance: client.4585 (3)172.21.15.167:0/2106051085 2024-05-11T08:06:25.002 INFO:teuthology.orchestra.run.smithi167.stdout:blocklisted: false 2024-05-11T08:06:25.002 INFO:teuthology.orchestra.run.smithi167.stderr:0+1 records in 2024-05-11T08:06:25.002 INFO:teuthology.orchestra.run.smithi167.stderr:0+1 records out 2024-05-11T08:06:25.002 INFO:teuthology.orchestra.run.smithi167.stderr:71 bytes copied, 2.4132e-05 s, 2.9 MB/s 2024-05-11T08:06:25.003 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T08:06:25.009 INFO:tasks.kclient:client.2 config is {'syntax': 'v1'} 2024-05-11T08:06:25.009 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-11T08:06:25.009 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-05-11T08:06:25.009 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-11T08:06:25.112 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:25.112+0000 7fe197af8640 1 Processor -- start 2024-05-11T08:06:25.112 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:25.112+0000 7fe197af8640 1 -- start start 2024-05-11T08:06:25.113 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:25.113+0000 7fe197af8640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe190073210 0x7fe1900b47d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:25.113 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:25.113+0000 7fe197af8640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe19015bc40 con 0x7fe19015b860 2024-05-11T08:06:25.113 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:25.113+0000 7fe197af8640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe19006e410 con 0x7fe1900736a0 2024-05-11T08:06:25.113 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:25.113+0000 7fe197af8640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fe19006e6f0 con 0x7fe190073210 2024-05-11T08:06:25.116 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:25.113+0000 7fe195894640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7fe19015b860 0x7fe19006f4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:47304/0 (socket says 172.21.15.167:47304) 2024-05-11T08:06:25.116 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:25.113+0000 7fe195894640 1 -- 172.21.15.167:0/4164231084 learned_addr learned my addr 172.21.15.167:0/4164231084 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T08:06:25.116 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:25.114+0000 7fe196095640 1 --2- 172.21.15.167:0/4164231084 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe190073210 0x7fe1900b47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:25.117 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:25.114+0000 7fe194892640 1 -- 172.21.15.167:0/4164231084 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1347904585 0 0) 0x7fe184001110 con 0x7fe19015b860 2024-05-11T08:06:25.117 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:25.114+0000 7fe194892640 1 -- 172.21.15.167:0/4164231084 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe17c003440 con 0x7fe19015b860 2024-05-11T08:06:25.117 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:25.114+0000 7fe194892640 1 -- 172.21.15.167:0/4164231084 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1786677660 0 0) 0x7fe19006e410 con 0x7fe1900736a0 2024-05-11T08:06:25.117 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:25.114+0000 7fe194892640 1 -- 172.21.15.167:0/4164231084 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe184001110 con 0x7fe1900736a0 2024-05-11T08:06:25.117 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:25.114+0000 7fe194892640 1 -- 172.21.15.167:0/4164231084 <== mon.0 v1:172.21.15.167:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2541701660 0 0) 0x7fe17c003440 con 0x7fe19015b860 2024-05-11T08:06:25.117 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:25.114+0000 7fe194892640 1 -- 172.21.15.167:0/4164231084 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe190073210 msgr2=0x7fe1900b47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:25.117 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:25.114+0000 7fe194892640 1 --2- 172.21.15.167:0/4164231084 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe190073210 0x7fe1900b47d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:25.117 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:25.114+0000 7fe194892640 1 -- 172.21.15.167:0/4164231084 >> v1:172.21.15.204:6789/0 conn(0x7fe1900736a0 legacy=0x7fe190072b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:25.117 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:25.114+0000 7fe194892640 1 -- 172.21.15.167:0/4164231084 --> v1:172.21.15.167:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe19006e9d0 con 0x7fe19015b860 2024-05-11T08:06:25.117 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:25.114+0000 7fe194892640 1 -- 172.21.15.167:0/4164231084 <== mon.0 v1:172.21.15.167:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3602277846 0 0) 0x7fe1840018e0 con 0x7fe19015b860 2024-05-11T08:06:25.117 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:25.114+0000 7fe194892640 1 -- 172.21.15.167:0/4164231084 >> v1:172.21.15.167:6789/0 conn(0x7fe19015b860 legacy=0x7fe19006f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:25.117 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:25.114+0000 7fe194892640 1 --2- 172.21.15.167:0/4164231084 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe17c003d00 0x7fe17c008850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:25.117 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:25.114+0000 7fe194892640 1 --2- 172.21.15.167:0/4164231084 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe17c008d90 0x7fe17c00b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:25.117 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:25.114+0000 7fe194892640 1 --2- 172.21.15.167:0/4164231084 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe17c00b6c0 0x7fe17c00dab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:25.117 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:25.114+0000 7fe194892640 1 -- 172.21.15.167:0/4164231084 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe19006e6f0 con 0x7fe17c003d00 2024-05-11T08:06:25.118 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:25.114+0000 7fe194892640 1 -- 172.21.15.167:0/4164231084 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe17c00e2a0 con 0x7fe17c008d90 2024-05-11T08:06:25.118 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:25.114+0000 7fe194892640 1 -- 172.21.15.167:0/4164231084 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fe17c00e4a0 con 0x7fe17c00b6c0 2024-05-11T08:06:25.118 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:25.114+0000 7fe196095640 1 --2- 172.21.15.167:0/4164231084 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe190073210 0x7fe1900b47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T08:06:25.118 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:25.114+0000 7fe195894640 1 --2- 172.21.15.167:0/4164231084 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe17c003d00 0x7fe17c008850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:25.118 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:25.114+0000 7fe195093640 1 --2- 172.21.15.167:0/4164231084 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe17c008d90 0x7fe17c00b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:25.118 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:25.114+0000 7fe196095640 1 --2- 172.21.15.167:0/4164231084 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe17c00b6c0 0x7fe17c00dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:25.118 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:25.114+0000 7fe195894640 1 -- 172.21.15.167:0/4164231084 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe17c00b6c0 msgr2=0x7fe17c00dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:25.118 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:25.114+0000 7fe195894640 1 --2- 172.21.15.167:0/4164231084 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe17c00b6c0 0x7fe17c00dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:25.118 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:25.114+0000 7fe195894640 1 -- 172.21.15.167:0/4164231084 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe17c008d90 msgr2=0x7fe17c00b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:25.118 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:25.114+0000 7fe195894640 1 --2- 172.21.15.167:0/4164231084 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe17c008d90 0x7fe17c00b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:25.118 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:25.114+0000 7fe195894640 1 -- 172.21.15.167:0/4164231084 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe17c003440 con 0x7fe17c003d00 2024-05-11T08:06:25.118 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:25.115+0000 7fe195894640 1 --2- 172.21.15.167:0/4164231084 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe17c003d00 0x7fe17c008850 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe18400c490 tx=0x7fe17c003730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f6b89931c7da79d server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:25.119 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:25.115+0000 7fe194892640 1 -- 172.21.15.167:0/4164231084 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe184036070 con 0x7fe17c003d00 2024-05-11T08:06:25.119 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:25.115+0000 7fe194892640 1 -- 172.21.15.167:0/4164231084 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe184005af0 con 0x7fe17c003d00 2024-05-11T08:06:25.119 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:25.115+0000 7fe194892640 1 -- 172.21.15.167:0/4164231084 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe18400c0e0 con 0x7fe17c003d00 2024-05-11T08:06:25.119 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:25.116+0000 7fe197af8640 1 -- 172.21.15.167:0/4164231084 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe17c003d00 msgr2=0x7fe17c008850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:25.119 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:25.116+0000 7fe197af8640 1 --2- 172.21.15.167:0/4164231084 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe17c003d00 0x7fe17c008850 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe18400c490 tx=0x7fe17c003730 comp rx=0 tx=0).stop 2024-05-11T08:06:25.119 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:25.116+0000 7fe196095640 1 -- 172.21.15.167:0/4164231084 reap_dead start 2024-05-11T08:06:25.119 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:25.116+0000 7fe197af8640 1 -- 172.21.15.167:0/4164231084 shutdown_connections 2024-05-11T08:06:25.119 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:25.116+0000 7fe197af8640 1 -- 172.21.15.167:0/4164231084 >> 172.21.15.167:0/4164231084 conn(0x7fe1900cd8f0 msgr2=0x7fe1900cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:25.119 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:25.116+0000 7fe197af8640 1 -- 172.21.15.167:0/4164231084 shutdown_connections 2024-05-11T08:06:25.119 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:25.116+0000 7fe197af8640 1 -- 172.21.15.167:0/4164231084 wait complete. 2024-05-11T08:06:25.122 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:25.117+0000 7fe197af8640 1 Processor -- start 2024-05-11T08:06:25.122 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:25.117+0000 7fe197af8640 1 -- start start 2024-05-11T08:06:25.122 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:25.118+0000 7fe197af8640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe17c003d00 0x7fe190151fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:25.122 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:25.118+0000 7fe197af8640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe1901465e0 0x7fe1901504b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:25.122 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:25.118+0000 7fe197af8640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe190146a40 0x7fe1900aad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:25.122 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:25.118+0000 7fe197af8640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe19006fbc0 con 0x7fe1901465e0 2024-05-11T08:06:25.122 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:25.118+0000 7fe197af8640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe19006ef90 con 0x7fe190146a40 2024-05-11T08:06:25.122 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:25.118+0000 7fe197af8640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fe190072640 con 0x7fe17c003d00 2024-05-11T08:06:25.122 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:25.118+0000 7fe195093640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe1901465e0 0x7fe1901504b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:25.122 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:25.118+0000 7fe195093640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe1901465e0 0x7fe1901504b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:58542/0 (socket says 172.21.15.167:58542) 2024-05-11T08:06:25.122 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:25.118+0000 7fe195093640 1 -- 172.21.15.167:0/3551028518 learned_addr learned my addr 172.21.15.167:0/3551028518 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:25.123 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:25.118+0000 7fe196095640 1 --2- 172.21.15.167:0/3551028518 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe190146a40 0x7fe1900aad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:25.123 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:25.118+0000 7fe195894640 1 --2- 172.21.15.167:0/3551028518 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe17c003d00 0x7fe190151fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:25.123 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:25.118+0000 7fe195093640 1 -- 172.21.15.167:0/3551028518 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe17c003d00 msgr2=0x7fe190151fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:25.123 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:25.118+0000 7fe195093640 1 --2- 172.21.15.167:0/3551028518 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe17c003d00 0x7fe190151fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:25.123 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:25.118+0000 7fe195093640 1 -- 172.21.15.167:0/3551028518 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe190146a40 msgr2=0x7fe1900aad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:25.123 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:25.118+0000 7fe195093640 1 --2- 172.21.15.167:0/3551028518 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe190146a40 0x7fe1900aad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:25.123 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:25.118+0000 7fe195093640 1 -- 172.21.15.167:0/3551028518 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1901524e0 con 0x7fe1901465e0 2024-05-11T08:06:25.123 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:25.118+0000 7fe195894640 1 --2- 172.21.15.167:0/3551028518 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe17c003d00 0x7fe190151fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T08:06:25.123 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:25.119+0000 7fe195093640 1 --2- 172.21.15.167:0/3551028518 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe1901465e0 0x7fe1901504b0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe18c00f820 tx=0x7fe18c00fce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:25.123 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:25.119+0000 7fe1827fc640 1 -- 172.21.15.167:0/3551028518 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe18c0050d0 con 0x7fe1901465e0 2024-05-11T08:06:25.123 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:25.119+0000 7fe1827fc640 1 -- 172.21.15.167:0/3551028518 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe18c018070 con 0x7fe1901465e0 2024-05-11T08:06:25.123 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:25.119+0000 7fe1827fc640 1 -- 172.21.15.167:0/3551028518 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe18c00d730 con 0x7fe1901465e0 2024-05-11T08:06:25.123 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:25.119+0000 7fe197af8640 1 -- 172.21.15.167:0/3551028518 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe190150f00 con 0x7fe1901465e0 2024-05-11T08:06:25.123 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:25.119+0000 7fe197af8640 1 -- 172.21.15.167:0/3551028518 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1901514c0 con 0x7fe1901465e0 2024-05-11T08:06:25.124 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:25.120+0000 7fe197af8640 1 -- 172.21.15.167:0/3551028518 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe190069590 con 0x7fe1901465e0 2024-05-11T08:06:25.124 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:25.120+0000 7fe1827fc640 1 -- 172.21.15.167:0/3551028518 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fe18c00d8d0 con 0x7fe1901465e0 2024-05-11T08:06:25.124 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:25.121+0000 7fe1827fc640 1 --2- 172.21.15.167:0/3551028518 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fe1740762d0 0x7fe174078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:25.124 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:25.121+0000 7fe1827fc640 1 -- 172.21.15.167:0/3551028518 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fe18c0977f0 con 0x7fe1901465e0 2024-05-11T08:06:25.124 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:25.121+0000 7fe195894640 1 --2- 172.21.15.167:0/3551028518 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fe1740762d0 0x7fe174078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:25.124 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:25.121+0000 7fe195894640 1 --2- 172.21.15.167:0/3551028518 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fe1740762d0 0x7fe174078790 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fe184033000 tx=0x7fe18402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:25.125 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:25.125+0000 7fe1827fc640 1 -- 172.21.15.167:0/3551028518 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fe18c060e10 con 0x7fe1901465e0 2024-05-11T08:06:25.356 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:25.355+0000 7fe197af8640 1 -- 172.21.15.167:0/3551028518 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fe19006dbb0 con 0x7fe1901465e0 2024-05-11T08:06:25.356 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:25.356+0000 7fe1827fc640 1 -- 172.21.15.167:0/3551028518 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7fe18c060710 con 0x7fe1901465e0 2024-05-11T08:06:25.356 INFO:teuthology.orchestra.run.smithi167.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-05-11T08:06:25.358 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:25.358+0000 7fe197af8640 1 -- 172.21.15.167:0/3551028518 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fe1740762d0 msgr2=0x7fe174078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:25.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:25.358+0000 7fe197af8640 1 --2- 172.21.15.167:0/3551028518 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fe1740762d0 0x7fe174078790 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fe184033000 tx=0x7fe18402e000 comp rx=0 tx=0).stop 2024-05-11T08:06:25.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:25.358+0000 7fe197af8640 1 -- 172.21.15.167:0/3551028518 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe1901465e0 msgr2=0x7fe1901504b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:25.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:25.358+0000 7fe197af8640 1 --2- 172.21.15.167:0/3551028518 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe1901465e0 0x7fe1901504b0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe18c00f820 tx=0x7fe18c00fce0 comp rx=0 tx=0).stop 2024-05-11T08:06:25.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:25.358+0000 7fe197af8640 1 -- 172.21.15.167:0/3551028518 shutdown_connections 2024-05-11T08:06:25.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:25.358+0000 7fe197af8640 1 --2- 172.21.15.167:0/3551028518 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fe1740762d0 0x7fe174078790 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:25.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:25.358+0000 7fe197af8640 1 --2- 172.21.15.167:0/3551028518 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe190146a40 0x7fe1900aad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:25.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:25.358+0000 7fe197af8640 1 --2- 172.21.15.167:0/3551028518 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe1901465e0 0x7fe1901504b0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:25.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:25.358+0000 7fe197af8640 1 --2- 172.21.15.167:0/3551028518 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe17c003d00 0x7fe190151fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:25.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:25.358+0000 7fe197af8640 1 -- 172.21.15.167:0/3551028518 >> 172.21.15.167:0/3551028518 conn(0x7fe1900cd8f0 msgr2=0x7fe190157be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:25.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:25.358+0000 7fe197af8640 1 -- 172.21.15.167:0/3551028518 shutdown_connections 2024-05-11T08:06:25.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:25.358+0000 7fe197af8640 1 -- 172.21.15.167:0/3551028518 wait complete. 2024-05-11T08:06:25.370 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-11T08:06:25.370 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-11T08:06:25.370 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi167.front.sepia.ceph.com 2024-05-11T08:06:25.370 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-05-11T08:06:25.370 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-05-11T08:06:25.370 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-11T08:06:25.370 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-11T08:06:25.371 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-11T08:06:25.371 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-05-11T08:06:25.371 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T08:06:25.371 DEBUG:teuthology.orchestra.run.smithi167:> ip addr 2024-05-11T08:06:25.390 INFO:teuthology.orchestra.run.smithi167.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-11T08:06:25.390 INFO:teuthology.orchestra.run.smithi167.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-11T08:06:25.390 INFO:teuthology.orchestra.run.smithi167.stdout: inet 127.0.0.1/8 scope host lo 2024-05-11T08:06:25.390 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2024-05-11T08:06:25.390 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 ::1/128 scope host 2024-05-11T08:06:25.390 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2024-05-11T08:06:25.390 INFO:teuthology.orchestra.run.smithi167.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-11T08:06:25.390 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:d9:35:e6 brd ff:ff:ff:ff:ff:ff 2024-05-11T08:06:25.390 INFO:teuthology.orchestra.run.smithi167.stdout: altname enp7s0f0 2024-05-11T08:06:25.390 INFO:teuthology.orchestra.run.smithi167.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-11T08:06:25.390 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:d9:35:e7 brd ff:ff:ff:ff:ff:ff 2024-05-11T08:06:25.390 INFO:teuthology.orchestra.run.smithi167.stdout: altname enp7s0f1 2024-05-11T08:06:25.390 INFO:teuthology.orchestra.run.smithi167.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-11T08:06:25.391 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:88:73:08 brd ff:ff:ff:ff:ff:ff 2024-05-11T08:06:25.391 INFO:teuthology.orchestra.run.smithi167.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-11T08:06:25.391 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:88:73:09 brd ff:ff:ff:ff:ff:ff 2024-05-11T08:06:25.391 INFO:teuthology.orchestra.run.smithi167.stdout: inet 172.21.15.167/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-05-11T08:06:25.391 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft 42698sec preferred_lft 42698sec 2024-05-11T08:06:25.391 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::ec4:7aff:fe88:7309/64 scope link 2024-05-11T08:06:25.391 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2024-05-11T08:06:25.391 INFO:teuthology.orchestra.run.smithi167.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-11T08:06:25.391 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether ce:35:1f:82:16:86 brd ff:ff:ff:ff:ff:ff 2024-05-11T08:06:25.391 INFO:teuthology.orchestra.run.smithi167.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-11T08:06:25.391 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2024-05-11T08:06:25.391 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::c0f2:32ff:fe0b:cf4f/64 scope link 2024-05-11T08:06:25.391 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2024-05-11T08:06:25.391 INFO:teuthology.orchestra.run.smithi167.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-11T08:06:25.391 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether ce:35:1f:82:16:86 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-11T08:06:25.392 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::cc35:1fff:fe82:1686/64 scope link tentative 2024-05-11T08:06:25.392 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2024-05-11T08:06:25.392 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T08:06:25.392 DEBUG:teuthology.orchestra.run.smithi167:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-11T08:06:25.454 INFO:teuthology.orchestra.run.smithi167.stdout:1 2024-05-11T08:06:25.455 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T08:06:25.456 DEBUG:teuthology.orchestra.run.smithi167:> route 2024-05-11T08:06:25.513 INFO:teuthology.orchestra.run.smithi167.stdout:Kernel IP routing table 2024-05-11T08:06:25.513 INFO:teuthology.orchestra.run.smithi167.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-11T08:06:25.513 INFO:teuthology.orchestra.run.smithi167.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-11T08:06:25.513 INFO:teuthology.orchestra.run.smithi167.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-11T08:06:25.513 INFO:teuthology.orchestra.run.smithi167.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-11T08:06:25.514 DEBUG:teuthology.orchestra.run.smithi167:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-11T08:06:25.514 DEBUG:teuthology.orchestra.run.smithi167:> set -e 2024-05-11T08:06:25.514 DEBUG:teuthology.orchestra.run.smithi167:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-11T08:06:25.514 DEBUG:teuthology.orchestra.run.smithi167:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-11T08:06:25.514 DEBUG:teuthology.orchestra.run.smithi167:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-11T08:06:25.514 DEBUG:teuthology.orchestra.run.smithi167:> ') 2024-05-11T08:06:25.602 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:25 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-11T08:06:25.636 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T08:06:25.642 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T08:06:25.642 DEBUG:teuthology.orchestra.run.smithi167:> ip netns list 2024-05-11T08:06:25.657 INFO:teuthology.orchestra.run.smithi167.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-11T08:06:25.658 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T08:06:25.658 DEBUG:teuthology.orchestra.run.smithi167:> ip netns list-id 2024-05-11T08:06:25.713 INFO:teuthology.orchestra.run.smithi167.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-05-11T08:06:25.714 DEBUG:teuthology.orchestra.run.smithi167:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-11T08:06:25.714 DEBUG:teuthology.orchestra.run.smithi167:> set -e 2024-05-11T08:06:25.714 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-11T08:06:25.714 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-05-11T08:06:25.714 DEBUG:teuthology.orchestra.run.smithi167:> ') 2024-05-11T08:06:25.802 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:25 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-11T08:06:25.826 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T08:06:25.833 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T08:06:25.833 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-05-11T08:06:25.857 INFO:teuthology.orchestra.run.smithi167.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-11T08:06:25.857 INFO:teuthology.orchestra.run.smithi167.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-11T08:06:25.857 INFO:teuthology.orchestra.run.smithi167.stdout: inet 127.0.0.1/8 scope host lo 2024-05-11T08:06:25.857 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2024-05-11T08:06:25.857 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 ::1/128 scope host 2024-05-11T08:06:25.857 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2024-05-11T08:06:25.858 INFO:teuthology.orchestra.run.smithi167.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-11T08:06:25.858 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether a2:3e:ee:2c:b9:10 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-05-11T08:06:25.858 INFO:teuthology.orchestra.run.smithi167.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-05-11T08:06:25.858 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2024-05-11T08:06:25.858 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::a03e:eeff:fe2c:b910/64 scope link 2024-05-11T08:06:25.858 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2024-05-11T08:06:25.859 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T08:06:25.859 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-05-11T08:06:25.870 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-11T08:06:25.902 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-11T08:06:25.924 INFO:teuthology.orchestra.run.smithi167.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-11T08:06:25.924 INFO:teuthology.orchestra.run.smithi167.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-11T08:06:25.924 INFO:teuthology.orchestra.run.smithi167.stdout: inet 127.0.0.1/8 scope host lo 2024-05-11T08:06:25.924 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2024-05-11T08:06:25.924 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 ::1/128 scope host 2024-05-11T08:06:25.925 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2024-05-11T08:06:25.925 INFO:teuthology.orchestra.run.smithi167.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-11T08:06:25.925 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether a2:3e:ee:2c:b9:10 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-05-11T08:06:25.925 INFO:teuthology.orchestra.run.smithi167.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-05-11T08:06:25.925 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2024-05-11T08:06:25.925 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::a03e:eeff:fe2c:b910/64 scope link 2024-05-11T08:06:25.925 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2024-05-11T08:06:25.926 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-05-11T08:06:25.926 DEBUG:teuthology.orchestra.run.smithi167:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-11T08:06:25.926 DEBUG:teuthology.orchestra.run.smithi167:> set -e 2024-05-11T08:06:25.926 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-05-11T08:06:25.926 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-05-11T08:06:25.926 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-05-11T08:06:25.927 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-05-11T08:06:25.927 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-05-11T08:06:25.927 DEBUG:teuthology.orchestra.run.smithi167:> ') 2024-05-11T08:06:26.037 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:26 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-11T08:06:26.096 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T08:06:26.102 DEBUG:teuthology.orchestra.run.smithi167:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-11T08:06:26.102 DEBUG:teuthology.orchestra.run.smithi167:> set -e 2024-05-11T08:06:26.102 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip link set brx.1 up 2024-05-11T08:06:26.102 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip link set dev brx.1 master ceph-brx 2024-05-11T08:06:26.102 DEBUG:teuthology.orchestra.run.smithi167:> ') 2024-05-11T08:06:26.151 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:26 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-11T08:06:26.173 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T08:06:26.179 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-11T08:06:26.179 DEBUG:teuthology.orchestra.run.smithi167:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-05-11T08:06:26.195 INFO:teuthology.orchestra.run.smithi167.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-05-11T08:06:26.195 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-11T08:06:26.195 DEBUG:teuthology.orchestra.run.smithi167:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-05-11T08:06:26.250 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-05-11T08:06:26.250 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T08:06:26.250 DEBUG:teuthology.orchestra.run.smithi167:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-05-11T08:06:26.334 INFO:teuthology.orchestra.run.smithi167.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-05-11T08:06:26.334 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-05-11T08:06:26.334 INFO:teuthology.orchestra.run.smithi167.stdout:invalid new device string format 2024-05-11T08:06:26.342 INFO:teuthology.orchestra.run.smithi167.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-05-11T08:06:26.343 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-05-11T08:06:26.343 INFO:teuthology.orchestra.run.smithi167.stdout:invalid new device string format 2024-05-11T08:06:26.343 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: resolved to: "172.21.15.167:3300,172.21.15.204:3300,172.21.15.204:3301" 2024-05-11T08:06:26.343 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-11T08:06:26.348 INFO:teuthology.orchestra.run.smithi167.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-05-11T08:06:26.348 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-05-11T08:06:26.348 INFO:teuthology.orchestra.run.smithi167.stdout:invalid new device string format 2024-05-11T08:06:26.348 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: resolved to: "172.21.15.167:3300,172.21.15.204:3300,172.21.15.204:3301" 2024-05-11T08:06:26.348 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-11T08:06:26.348 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: resolved to: "172.21.15.167:6789,172.21.15.204:6789,172.21.15.204:6790" 2024-05-11T08:06:26.348 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.167:6789,172.21.15.204:6789,172.21.15.204:6790:/ 2024-05-11T08:06:26.348 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs,key=2,fsid=2034778d-8ff1-4823-aa44-ded32b518241" will pass to kernel 2024-05-11T08:06:26.349 INFO:teuthology.orchestra.run.smithi167.stdout:mount: /home/ubuntu/cephtest/mnt.2 does not contain SELinux labels. 2024-05-11T08:06:26.349 INFO:teuthology.orchestra.run.smithi167.stdout: You just mounted a file system that supports labels which does not 2024-05-11T08:06:26.349 INFO:teuthology.orchestra.run.smithi167.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-11T08:06:26.349 INFO:teuthology.orchestra.run.smithi167.stdout: applications will generate AVC messages and not be allowed access to 2024-05-11T08:06:26.349 INFO:teuthology.orchestra.run.smithi167.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-11T08:06:26.351 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-11T08:06:26.351 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T08:06:26.351 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-05-11T08:06:26.376 DEBUG:teuthology.orchestra.run.smithi167:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-11T08:06:26.466 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:26 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-11T08:06:26.468 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T08:06:26.474 INFO:teuthology.orchestra.run.smithi167.stdout:client4746 2024-05-11T08:06:26.474 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-05-11T08:06:26.474 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T08:06:26.491 DEBUG:teuthology.orchestra.run.smithi167:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-11T08:06:26.580 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:26 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-11T08:06:26.581 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T08:06:26.587 INFO:teuthology.orchestra.run.smithi167.stdout:client4746 2024-05-11T08:06:26.587 DEBUG:teuthology.orchestra.run.smithi167:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-11T08:06:26.637 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:26 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-11T08:06:26.638 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T08:06:26.644 INFO:teuthology.orchestra.run.smithi167.stdout:client4746 2024-05-11T08:06:26.644 DEBUG:teuthology.orchestra.run.smithi167:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/2034778d-8ff1-4823-aa44-ded32b518241.client4746/status') 2024-05-11T08:06:26.692 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:26 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-11T08:06:26.703 INFO:teuthology.orchestra.run.smithi167.stdout:instance: client.4746 (3)192.168.0.2:0/3406918671 2024-05-11T08:06:26.703 INFO:teuthology.orchestra.run.smithi167.stdout:blocklisted: false 2024-05-11T08:06:26.703 INFO:teuthology.orchestra.run.smithi167.stderr:0+1 records in 2024-05-11T08:06:26.703 INFO:teuthology.orchestra.run.smithi167.stderr:0+1 records out 2024-05-11T08:06:26.703 INFO:teuthology.orchestra.run.smithi167.stderr:69 bytes copied, 2.0892e-05 s, 3.3 MB/s 2024-05-11T08:06:26.704 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T08:06:26.710 INFO:tasks.kclient:client.1 config is {'syntax': 'v1'} 2024-05-11T08:06:26.710 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-11T08:06:26.710 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-05-11T08:06:26.710 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-11T08:06:26.811 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:26.811+0000 7f7657314640 1 Processor -- start 2024-05-11T08:06:26.811 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:26.811+0000 7f7657314640 1 -- start start 2024-05-11T08:06:26.812 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:26.812+0000 7f7657314640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f765015b860 0x7f765006f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:26.812 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:26.812+0000 7f7657314640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f7650073210 0x7f76500b47d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:26.813 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:26.812+0000 7f7657314640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f765015bc40 con 0x7f76500736a0 2024-05-11T08:06:26.813 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:26.812+0000 7f7657314640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f765006e410 con 0x7f765015b860 2024-05-11T08:06:26.813 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:26.812+0000 7f7657314640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f765006e6f0 con 0x7f7650073210 2024-05-11T08:06:26.813 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:26.812+0000 7f76548af640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f76500736a0 0x7f7650072b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:47306/0 (socket says 172.21.15.167:47306) 2024-05-11T08:06:26.813 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:26.812+0000 7f76548af640 1 -- 172.21.15.167:0/284234482 learned_addr learned my addr 172.21.15.167:0/284234482 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T08:06:26.813 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:26.812+0000 7f76550b0640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f765015b860 0x7f765006f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:26.813 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:26.812+0000 7f76558b1640 1 --2- 172.21.15.167:0/284234482 >> v2:172.21.15.204:3300/0 conn(0x7f7650073210 0x7f76500b47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:26.813 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:26.812+0000 7f7647fff640 1 -- 172.21.15.167:0/284234482 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 630821972 0 0) 0x7f7640001110 con 0x7f76500736a0 2024-05-11T08:06:26.813 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:26.812+0000 7f7647fff640 1 -- 172.21.15.167:0/284234482 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f763c005730 con 0x7f76500736a0 2024-05-11T08:06:26.813 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:26.812+0000 7f76550b0640 1 -- 172.21.15.167:0/284234482 >> v2:172.21.15.204:3300/0 conn(0x7f7650073210 msgr2=0x7f76500b47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:26.813 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:26.812+0000 7f76550b0640 1 --2- 172.21.15.167:0/284234482 >> v2:172.21.15.204:3300/0 conn(0x7f7650073210 0x7f76500b47d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:26.813 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:26.812+0000 7f76550b0640 1 -- 172.21.15.167:0/284234482 >> v1:172.21.15.167:6789/0 conn(0x7f76500736a0 legacy=0x7f7650072b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:26.814 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:26.812+0000 7f76550b0640 1 -- 172.21.15.167:0/284234482 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f765006e9d0 con 0x7f765015b860 2024-05-11T08:06:26.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:26.813+0000 7f76550b0640 1 --2- 172.21.15.167:0/284234482 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f765015b860 0x7f765006f4b0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f76500d0880 tx=0x7f764c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9399fbb976551311 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:26.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:26.813+0000 7f7647fff640 1 -- 172.21.15.167:0/284234482 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f764c037070 con 0x7f765015b860 2024-05-11T08:06:26.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:26.813+0000 7f7647fff640 1 -- 172.21.15.167:0/284234482 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f764c032440 con 0x7f765015b860 2024-05-11T08:06:26.816 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:26.813+0000 7f7647fff640 1 -- 172.21.15.167:0/284234482 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f764c032810 con 0x7f765015b860 2024-05-11T08:06:26.816 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:26.814+0000 7f7657314640 1 -- 172.21.15.167:0/284234482 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f765015b860 msgr2=0x7f765006f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:26.816 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:26.814+0000 7f7657314640 1 --2- 172.21.15.167:0/284234482 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f765015b860 0x7f765006f4b0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f76500d0880 tx=0x7f764c02a1e0 comp rx=0 tx=0).stop 2024-05-11T08:06:26.816 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:26.814+0000 7f7657314640 1 -- 172.21.15.167:0/284234482 shutdown_connections 2024-05-11T08:06:26.816 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:26.814+0000 7f7657314640 1 --2- 172.21.15.167:0/284234482 >> v2:172.21.15.204:3300/0 conn(0x7f7650073210 0x7f76500b47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:26.816 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:26.814+0000 7f7657314640 1 --2- 172.21.15.167:0/284234482 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f765015b860 0x7f765006f4b0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:26.816 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:26.814+0000 7f7657314640 1 -- 172.21.15.167:0/284234482 >> 172.21.15.167:0/284234482 conn(0x7f76500cd8f0 msgr2=0x7f76500cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:26.816 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:26.814+0000 7f7657314640 1 -- 172.21.15.167:0/284234482 shutdown_connections 2024-05-11T08:06:26.816 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:26.814+0000 7f7657314640 1 -- 172.21.15.167:0/284234482 wait complete. 2024-05-11T08:06:26.816 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:26.815+0000 7f7657314640 1 Processor -- start 2024-05-11T08:06:26.816 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:26.815+0000 7f7657314640 1 -- start start 2024-05-11T08:06:26.819 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:26.816+0000 7f7657314640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7650073210 0x7f765013d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:26.819 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:26.816+0000 7f7657314640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f76500736a0 0x7f7650151660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:26.819 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:26.816+0000 7f7657314640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f765015b860 0x7f7650151ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:26.819 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:26.816+0000 7f7657314640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7650072640 con 0x7f765015b860 2024-05-11T08:06:26.819 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:26.816+0000 7f7657314640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f765006ef90 con 0x7f76500736a0 2024-05-11T08:06:26.819 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:26.816+0000 7f7657314640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f76500727c0 con 0x7f7650073210 2024-05-11T08:06:26.819 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:26.816+0000 7f76558b1640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f765015b860 0x7f7650151ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:26.819 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:26.816+0000 7f76558b1640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f765015b860 0x7f7650151ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:58556/0 (socket says 172.21.15.167:58556) 2024-05-11T08:06:26.819 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:26.816+0000 7f76558b1640 1 -- 172.21.15.167:0/1802464059 learned_addr learned my addr 172.21.15.167:0/1802464059 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:26.820 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:26.816+0000 7f76550b0640 1 --2- 172.21.15.167:0/1802464059 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7650073210 0x7f765013d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:26.820 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:26.816+0000 7f76558b1640 1 -- 172.21.15.167:0/1802464059 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7650073210 msgr2=0x7f765013d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:26.820 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:26.816+0000 7f76558b1640 1 --2- 172.21.15.167:0/1802464059 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7650073210 0x7f765013d450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:26.820 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:26.816+0000 7f76558b1640 1 -- 172.21.15.167:0/1802464059 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f76500736a0 msgr2=0x7f7650151660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:26.820 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:26.816+0000 7f76558b1640 1 --2- 172.21.15.167:0/1802464059 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f76500736a0 0x7f7650151660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:26.820 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:26.816+0000 7f76558b1640 1 -- 172.21.15.167:0/1802464059 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f765013fb90 con 0x7f765015b860 2024-05-11T08:06:26.820 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:26.816+0000 7f76558b1640 1 --2- 172.21.15.167:0/1802464059 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f765015b860 0x7f7650151ba0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f764800efc0 tx=0x7f764800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:26.820 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:26.817+0000 7f7645ffb640 1 -- 172.21.15.167:0/1802464059 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7648018020 con 0x7f765015b860 2024-05-11T08:06:26.820 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:26.817+0000 7f7645ffb640 1 -- 172.21.15.167:0/1802464059 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7648014480 con 0x7f765015b860 2024-05-11T08:06:26.820 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:26.817+0000 7f7657314640 1 -- 172.21.15.167:0/1802464059 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f765013eaf0 con 0x7f765015b860 2024-05-11T08:06:26.820 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:26.817+0000 7f7645ffb640 1 -- 172.21.15.167:0/1802464059 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7648014870 con 0x7f765015b860 2024-05-11T08:06:26.820 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:26.817+0000 7f7657314640 1 -- 172.21.15.167:0/1802464059 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f765013f0b0 con 0x7f765015b860 2024-05-11T08:06:26.820 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:26.818+0000 7f7657314640 1 -- 172.21.15.167:0/1802464059 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7650069590 con 0x7f765015b860 2024-05-11T08:06:26.820 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:26.818+0000 7f7645ffb640 1 -- 172.21.15.167:0/1802464059 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f7648005d10 con 0x7f765015b860 2024-05-11T08:06:26.820 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:26.819+0000 7f7645ffb640 1 --2- 172.21.15.167:0/1802464059 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f7634076300 0x7f76340787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:26.821 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:26.819+0000 7f7645ffb640 1 -- 172.21.15.167:0/1802464059 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f7648098420 con 0x7f765015b860 2024-05-11T08:06:26.821 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:26.819+0000 7f76550b0640 1 --2- 172.21.15.167:0/1802464059 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f7634076300 0x7f76340787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:26.821 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:26.819+0000 7f76550b0640 1 --2- 172.21.15.167:0/1802464059 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f7634076300 0x7f76340787c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f764c034000 tx=0x7f764c02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:26.823 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:26.823+0000 7f7645ffb640 1 -- 172.21.15.167:0/1802464059 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f7648022070 con 0x7f765015b860 2024-05-11T08:06:27.044 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:27.044+0000 7f7657314640 1 -- 172.21.15.167:0/1802464059 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f7650150de0 con 0x7f765015b860 2024-05-11T08:06:27.044 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:27.044+0000 7f7645ffb640 1 -- 172.21.15.167:0/1802464059 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7f7648010040 con 0x7f765015b860 2024-05-11T08:06:27.044 INFO:teuthology.orchestra.run.smithi167.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-05-11T08:06:27.047 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:27.046+0000 7f7657314640 1 -- 172.21.15.167:0/1802464059 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f7634076300 msgr2=0x7f76340787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:27.047 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:27.046+0000 7f7657314640 1 --2- 172.21.15.167:0/1802464059 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f7634076300 0x7f76340787c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f764c034000 tx=0x7f764c02e000 comp rx=0 tx=0).stop 2024-05-11T08:06:27.047 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:27.046+0000 7f7657314640 1 -- 172.21.15.167:0/1802464059 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f765015b860 msgr2=0x7f7650151ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:27.047 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:27.046+0000 7f7657314640 1 --2- 172.21.15.167:0/1802464059 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f765015b860 0x7f7650151ba0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f764800efc0 tx=0x7f764800c490 comp rx=0 tx=0).stop 2024-05-11T08:06:27.047 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:27.046+0000 7f7657314640 1 -- 172.21.15.167:0/1802464059 shutdown_connections 2024-05-11T08:06:27.047 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:27.046+0000 7f7657314640 1 --2- 172.21.15.167:0/1802464059 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f7634076300 0x7f76340787c0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:27.047 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:27.046+0000 7f7657314640 1 --2- 172.21.15.167:0/1802464059 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f765015b860 0x7f7650151ba0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:27.047 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:27.046+0000 7f7657314640 1 --2- 172.21.15.167:0/1802464059 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f76500736a0 0x7f7650151660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:27.047 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:27.046+0000 7f7657314640 1 --2- 172.21.15.167:0/1802464059 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7650073210 0x7f765013d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:27.047 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:27.046+0000 7f7657314640 1 -- 172.21.15.167:0/1802464059 >> 172.21.15.167:0/1802464059 conn(0x7f76500cd8f0 msgr2=0x7f765014f740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:27.047 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:27.046+0000 7f7657314640 1 -- 172.21.15.167:0/1802464059 shutdown_connections 2024-05-11T08:06:27.048 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:27.046+0000 7f7657314640 1 -- 172.21.15.167:0/1802464059 wait complete. 2024-05-11T08:06:27.058 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-11T08:06:27.058 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-11T08:06:27.058 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi204.front.sepia.ceph.com 2024-05-11T08:06:27.058 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-05-11T08:06:27.059 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-05-11T08:06:27.059 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-11T08:06:27.059 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-11T08:06:27.059 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-11T08:06:27.059 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-05-11T08:06:27.059 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T08:06:27.060 DEBUG:teuthology.orchestra.run.smithi204:> ip addr 2024-05-11T08:06:27.079 INFO:teuthology.orchestra.run.smithi204.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-11T08:06:27.079 INFO:teuthology.orchestra.run.smithi204.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-11T08:06:27.080 INFO:teuthology.orchestra.run.smithi204.stdout: inet 127.0.0.1/8 scope host lo 2024-05-11T08:06:27.080 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-05-11T08:06:27.080 INFO:teuthology.orchestra.run.smithi204.stdout: inet6 ::1/128 scope host 2024-05-11T08:06:27.080 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-05-11T08:06:27.080 INFO:teuthology.orchestra.run.smithi204.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-11T08:06:27.080 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 0c:c4:7a:d9:36:2a brd ff:ff:ff:ff:ff:ff 2024-05-11T08:06:27.080 INFO:teuthology.orchestra.run.smithi204.stdout: altname enp7s0f0 2024-05-11T08:06:27.080 INFO:teuthology.orchestra.run.smithi204.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-11T08:06:27.080 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 0c:c4:7a:d9:36:2b brd ff:ff:ff:ff:ff:ff 2024-05-11T08:06:27.080 INFO:teuthology.orchestra.run.smithi204.stdout: altname enp7s0f1 2024-05-11T08:06:27.080 INFO:teuthology.orchestra.run.smithi204.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-11T08:06:27.080 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 0c:c4:7a:88:77:aa brd ff:ff:ff:ff:ff:ff 2024-05-11T08:06:27.080 INFO:teuthology.orchestra.run.smithi204.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-11T08:06:27.080 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 0c:c4:7a:88:77:ab brd ff:ff:ff:ff:ff:ff 2024-05-11T08:06:27.081 INFO:teuthology.orchestra.run.smithi204.stdout: inet 172.21.15.204/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-05-11T08:06:27.081 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft 42701sec preferred_lft 42701sec 2024-05-11T08:06:27.081 INFO:teuthology.orchestra.run.smithi204.stdout: inet6 fe80::ec4:7aff:fe88:77ab/64 scope link 2024-05-11T08:06:27.081 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-05-11T08:06:27.081 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-11T08:06:27.081 DEBUG:teuthology.orchestra.run.smithi204:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-11T08:06:27.081 DEBUG:teuthology.orchestra.run.smithi204:> set -e 2024-05-11T08:06:27.081 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip link add name ceph-brx type bridge 2024-05-11T08:06:27.081 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip addr flush dev ceph-brx 2024-05-11T08:06:27.081 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip link set ceph-brx up 2024-05-11T08:06:27.081 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-11T08:06:27.081 DEBUG:teuthology.orchestra.run.smithi204:> ') 2024-05-11T08:06:27.177 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:06:27 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-11T08:06:27.295 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:06:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T08:06:27.301 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T08:06:27.301 DEBUG:teuthology.orchestra.run.smithi204:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-11T08:06:27.327 INFO:teuthology.orchestra.run.smithi204.stdout:1 2024-05-11T08:06:27.329 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T08:06:27.329 DEBUG:teuthology.orchestra.run.smithi204:> route 2024-05-11T08:06:27.402 INFO:teuthology.orchestra.run.smithi204.stdout:Kernel IP routing table 2024-05-11T08:06:27.402 INFO:teuthology.orchestra.run.smithi204.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-11T08:06:27.403 INFO:teuthology.orchestra.run.smithi204.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-11T08:06:27.403 INFO:teuthology.orchestra.run.smithi204.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-11T08:06:27.403 INFO:teuthology.orchestra.run.smithi204.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-11T08:06:27.403 DEBUG:teuthology.orchestra.run.smithi204:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-11T08:06:27.403 DEBUG:teuthology.orchestra.run.smithi204:> set -e 2024-05-11T08:06:27.403 DEBUG:teuthology.orchestra.run.smithi204:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-11T08:06:27.403 DEBUG:teuthology.orchestra.run.smithi204:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-11T08:06:27.403 DEBUG:teuthology.orchestra.run.smithi204:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-11T08:06:27.403 DEBUG:teuthology.orchestra.run.smithi204:> ') 2024-05-11T08:06:27.493 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:06:27 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-11T08:06:27.685 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:06:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T08:06:27.691 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T08:06:27.691 DEBUG:teuthology.orchestra.run.smithi204:> ip netns list 2024-05-11T08:06:27.707 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T08:06:27.707 DEBUG:teuthology.orchestra.run.smithi204:> ip netns list-id 2024-05-11T08:06:27.763 DEBUG:teuthology.orchestra.run.smithi204:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-11T08:06:27.763 DEBUG:teuthology.orchestra.run.smithi204:> set -e 2024-05-11T08:06:27.763 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-11T08:06:27.763 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-05-11T08:06:27.763 DEBUG:teuthology.orchestra.run.smithi204:> ') 2024-05-11T08:06:27.853 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:06:27 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-11T08:06:27.877 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:06:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T08:06:27.884 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-05-11T08:06:27.884 DEBUG:teuthology.orchestra.run.smithi204:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-11T08:06:27.884 DEBUG:teuthology.orchestra.run.smithi204:> set -e 2024-05-11T08:06:27.884 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-05-11T08:06:27.884 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-11T08:06:27.884 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-05-11T08:06:27.884 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-05-11T08:06:27.884 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-05-11T08:06:27.885 DEBUG:teuthology.orchestra.run.smithi204:> ') 2024-05-11T08:06:27.934 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:06:27 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-11T08:06:28.007 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:06:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T08:06:28.014 DEBUG:teuthology.orchestra.run.smithi204:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-11T08:06:28.014 DEBUG:teuthology.orchestra.run.smithi204:> set -e 2024-05-11T08:06:28.014 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip link set brx.0 up 2024-05-11T08:06:28.014 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip link set dev brx.0 master ceph-brx 2024-05-11T08:06:28.014 DEBUG:teuthology.orchestra.run.smithi204:> ') 2024-05-11T08:06:28.063 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:06:28 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-11T08:06:28.086 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:06:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T08:06:28.092 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-11T08:06:28.092 DEBUG:teuthology.orchestra.run.smithi204:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-05-11T08:06:28.108 INFO:teuthology.orchestra.run.smithi204.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-05-11T08:06:28.108 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-11T08:06:28.108 DEBUG:teuthology.orchestra.run.smithi204:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-05-11T08:06:28.163 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-05-11T08:06:28.163 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T08:06:28.163 DEBUG:teuthology.orchestra.run.smithi204:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-11T08:06:28.256 INFO:teuthology.orchestra.run.smithi204.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-11T08:06:28.256 INFO:teuthology.orchestra.run.smithi204.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-05-11T08:06:28.256 INFO:teuthology.orchestra.run.smithi204.stdout:invalid new device string format 2024-05-11T08:06:28.371 INFO:teuthology.orchestra.run.smithi204.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-11T08:06:28.371 INFO:teuthology.orchestra.run.smithi204.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-05-11T08:06:28.371 INFO:teuthology.orchestra.run.smithi204.stdout:invalid new device string format 2024-05-11T08:06:28.372 INFO:teuthology.orchestra.run.smithi204.stdout:mount.ceph: resolved to: "172.21.15.167:3300,172.21.15.204:3300,172.21.15.204:3301" 2024-05-11T08:06:28.372 INFO:teuthology.orchestra.run.smithi204.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-11T08:06:28.376 INFO:teuthology.orchestra.run.smithi204.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-05-11T08:06:28.376 INFO:teuthology.orchestra.run.smithi204.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-05-11T08:06:28.377 INFO:teuthology.orchestra.run.smithi204.stdout:invalid new device string format 2024-05-11T08:06:28.377 INFO:teuthology.orchestra.run.smithi204.stdout:mount.ceph: resolved to: "172.21.15.167:3300,172.21.15.204:3300,172.21.15.204:3301" 2024-05-11T08:06:28.377 INFO:teuthology.orchestra.run.smithi204.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-11T08:06:28.377 INFO:teuthology.orchestra.run.smithi204.stdout:mount.ceph: resolved to: "172.21.15.167:6789,172.21.15.204:6789,172.21.15.204:6790" 2024-05-11T08:06:28.377 INFO:teuthology.orchestra.run.smithi204.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.167:6789,172.21.15.204:6789,172.21.15.204:6790:/ 2024-05-11T08:06:28.377 INFO:teuthology.orchestra.run.smithi204.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=2034778d-8ff1-4823-aa44-ded32b518241" will pass to kernel 2024-05-11T08:06:28.378 INFO:teuthology.orchestra.run.smithi204.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-05-11T08:06:28.378 INFO:teuthology.orchestra.run.smithi204.stdout: You just mounted a file system that supports labels which does not 2024-05-11T08:06:28.378 INFO:teuthology.orchestra.run.smithi204.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-11T08:06:28.378 INFO:teuthology.orchestra.run.smithi204.stdout: applications will generate AVC messages and not be allowed access to 2024-05-11T08:06:28.378 INFO:teuthology.orchestra.run.smithi204.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-11T08:06:28.380 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-11T08:06:28.380 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T08:06:28.380 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-05-11T08:06:28.405 DEBUG:teuthology.orchestra.run.smithi204:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-11T08:06:28.495 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:06:28 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-11T08:06:28.497 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:06:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T08:06:28.503 INFO:teuthology.orchestra.run.smithi204.stdout:client4758 2024-05-11T08:06:28.503 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-11T08:06:28.503 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T08:06:28.519 DEBUG:teuthology.orchestra.run.smithi204:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-11T08:06:28.609 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:06:28 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-11T08:06:28.611 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:06:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T08:06:28.616 INFO:teuthology.orchestra.run.smithi204.stdout:client4758 2024-05-11T08:06:28.616 DEBUG:teuthology.orchestra.run.smithi204:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-11T08:06:28.665 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:06:28 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-11T08:06:28.667 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:06:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T08:06:28.672 INFO:teuthology.orchestra.run.smithi204.stdout:client4758 2024-05-11T08:06:28.672 DEBUG:teuthology.orchestra.run.smithi204:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/2034778d-8ff1-4823-aa44-ded32b518241.client4758/status') 2024-05-11T08:06:28.721 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:06:28 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-11T08:06:28.732 INFO:teuthology.orchestra.run.smithi204.stdout:instance: client.4758 (3)172.21.15.204:0/3915451121 2024-05-11T08:06:28.732 INFO:teuthology.orchestra.run.smithi204.stdout:blocklisted: false 2024-05-11T08:06:28.732 INFO:teuthology.orchestra.run.smithi204.stderr:0+1 records in 2024-05-11T08:06:28.732 INFO:teuthology.orchestra.run.smithi204.stderr:0+1 records out 2024-05-11T08:06:28.732 INFO:teuthology.orchestra.run.smithi204.stderr:71 bytes copied, 2.322e-05 s, 3.1 MB/s 2024-05-11T08:06:28.733 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:06:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T08:06:28.739 INFO:tasks.kclient:client.3 config is {'syntax': 'v1'} 2024-05-11T08:06:28.739 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-11T08:06:28.739 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-05-11T08:06:28.739 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-11T08:06:28.851 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:28.851+0000 7f6683b58640 1 Processor -- start 2024-05-11T08:06:28.851 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:28.851+0000 7f6683b58640 1 -- start start 2024-05-11T08:06:28.853 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:28.852+0000 7f6683b58640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f667c0cd610 0x7f667c163be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:28.853 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:28.852+0000 7f6683b58640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f667c0b7b90 0x7f667c0b8020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:28.853 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:28.852+0000 7f6683b58640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f667c0b6300 con 0x7f667c0cd9f0 2024-05-11T08:06:28.853 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:28.852+0000 7f6683b58640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f667c0b65e0 con 0x7f667c0b7b90 2024-05-11T08:06:28.853 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:28.852+0000 7f6683b58640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f667c163100 con 0x7f667c0cd610 2024-05-11T08:06:28.853 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:28.852+0000 7f66810f3640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f667c0cd9f0 0x7f667c160230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:47316/0 (socket says 172.21.15.167:47316) 2024-05-11T08:06:28.853 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:28.852+0000 7f66810f3640 1 -- 172.21.15.167:0/631594135 learned_addr learned my addr 172.21.15.167:0/631594135 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T08:06:28.853 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:28.852+0000 7f66820f5640 1 --2- 172.21.15.167:0/631594135 >> v2:172.21.15.204:3300/0 conn(0x7f667c0b7b90 0x7f667c0b8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:28.853 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:28.852+0000 7f66818f4640 1 --2- 172.21.15.167:0/631594135 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f667c0cd610 0x7f667c163be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:28.853 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:28.853+0000 7f66808f2640 1 -- 172.21.15.167:0/631594135 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1862357170 0 0) 0x7f667c0b6300 con 0x7f667c0cd9f0 2024-05-11T08:06:28.854 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:28.853+0000 7f66808f2640 1 -- 172.21.15.167:0/631594135 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6668003520 con 0x7f667c0cd9f0 2024-05-11T08:06:28.854 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:28.853+0000 7f66818f4640 1 -- 172.21.15.167:0/631594135 >> v2:172.21.15.204:3300/0 conn(0x7f667c0b7b90 msgr2=0x7f667c0b8020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:28.854 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:28.853+0000 7f66818f4640 1 --2- 172.21.15.167:0/631594135 >> v2:172.21.15.204:3300/0 conn(0x7f667c0b7b90 0x7f667c0b8020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:28.854 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:28.853+0000 7f66818f4640 1 -- 172.21.15.167:0/631594135 >> v1:172.21.15.167:6789/0 conn(0x7f667c0cd9f0 legacy=0x7f667c160230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:28.854 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:28.853+0000 7f66818f4640 1 -- 172.21.15.167:0/631594135 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f667c1633e0 con 0x7f667c0cd610 2024-05-11T08:06:28.854 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:28.853+0000 7f66820f5640 1 --2- 172.21.15.167:0/631594135 >> v2:172.21.15.204:3300/0 conn(0x7f667c0b7b90 0x7f667c0b8020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T08:06:28.855 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:28.854+0000 7f66818f4640 1 --2- 172.21.15.167:0/631594135 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f667c0cd610 0x7f667c163be0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f66780097a0 tx=0x7f667802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c508fe748ed285c6 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:28.855 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:28.854+0000 7f66808f2640 1 -- 172.21.15.167:0/631594135 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6678037070 con 0x7f667c0cd610 2024-05-11T08:06:28.855 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:28.854+0000 7f66808f2640 1 -- 172.21.15.167:0/631594135 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6678032440 con 0x7f667c0cd610 2024-05-11T08:06:28.855 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:28.854+0000 7f66808f2640 1 -- 172.21.15.167:0/631594135 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66780327f0 con 0x7f667c0cd610 2024-05-11T08:06:28.855 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:28.854+0000 7f6683b58640 1 -- 172.21.15.167:0/631594135 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f667c0cd610 msgr2=0x7f667c163be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:28.855 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:28.854+0000 7f6683b58640 1 --2- 172.21.15.167:0/631594135 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f667c0cd610 0x7f667c163be0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f66780097a0 tx=0x7f667802a180 comp rx=0 tx=0).stop 2024-05-11T08:06:28.855 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:28.854+0000 7f6683b58640 1 -- 172.21.15.167:0/631594135 shutdown_connections 2024-05-11T08:06:28.855 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:28.854+0000 7f6683b58640 1 --2- 172.21.15.167:0/631594135 >> v2:172.21.15.204:3300/0 conn(0x7f667c0b7b90 0x7f667c0b8020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:28.856 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:28.854+0000 7f6683b58640 1 --2- 172.21.15.167:0/631594135 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f667c0cd610 0x7f667c163be0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:28.856 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:28.854+0000 7f6683b58640 1 -- 172.21.15.167:0/631594135 >> 172.21.15.167:0/631594135 conn(0x7f667c072730 msgr2=0x7f667c0d2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:28.856 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:28.854+0000 7f6683b58640 1 -- 172.21.15.167:0/631594135 shutdown_connections 2024-05-11T08:06:28.856 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:28.854+0000 7f6683b58640 1 -- 172.21.15.167:0/631594135 wait complete. 2024-05-11T08:06:28.858 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:28.856+0000 7f6683b58640 1 Processor -- start 2024-05-11T08:06:28.858 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:28.856+0000 7f6683b58640 1 -- start start 2024-05-11T08:06:28.858 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:28.856+0000 7f6683b58640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f667c0b7b90 0x7f667c0cb0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:28.858 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:28.856+0000 7f6683b58640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f667c0cd610 0x7f667c15cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:28.858 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:28.856+0000 7f6683b58640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f667c0cd9f0 0x7f667c1f83e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:28.858 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:28.856+0000 7f6683b58640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f667c0d2a80 con 0x7f667c0cd610 2024-05-11T08:06:28.858 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:28.856+0000 7f6683b58640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f667c160940 con 0x7f667c0b7b90 2024-05-11T08:06:28.858 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:28.856+0000 7f6683b58640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f667c0cde50 con 0x7f667c0cd9f0 2024-05-11T08:06:28.858 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:28.857+0000 7f66810f3640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f667c0cd610 0x7f667c15cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:28.858 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:28.857+0000 7f66810f3640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f667c0cd610 0x7f667c15cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:58572/0 (socket says 172.21.15.167:58572) 2024-05-11T08:06:28.858 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:28.857+0000 7f66810f3640 1 -- 172.21.15.167:0/1302131754 learned_addr learned my addr 172.21.15.167:0/1302131754 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:28.859 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:28.857+0000 7f66820f5640 1 --2- 172.21.15.167:0/1302131754 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f667c0cd9f0 0x7f667c1f83e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:28.859 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:28.857+0000 7f66818f4640 1 --2- 172.21.15.167:0/1302131754 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f667c0b7b90 0x7f667c0cb0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:28.859 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:28.857+0000 7f66810f3640 1 -- 172.21.15.167:0/1302131754 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f667c0cd9f0 msgr2=0x7f667c1f83e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:28.859 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:28.857+0000 7f66810f3640 1 --2- 172.21.15.167:0/1302131754 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f667c0cd9f0 0x7f667c1f83e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:28.859 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:28.857+0000 7f66810f3640 1 -- 172.21.15.167:0/1302131754 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f667c0b7b90 msgr2=0x7f667c0cb0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:28.859 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:28.857+0000 7f66810f3640 1 --2- 172.21.15.167:0/1302131754 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f667c0b7b90 0x7f667c0cb0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:28.859 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:28.857+0000 7f66810f3640 1 -- 172.21.15.167:0/1302131754 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f667c15d3e0 con 0x7f667c0cd610 2024-05-11T08:06:28.859 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:28.857+0000 7f66810f3640 1 --2- 172.21.15.167:0/1302131754 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f667c0cd610 0x7f667c15cea0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f666c00c410 tx=0x7f666c00c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:28.859 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:28.857+0000 7f66727fc640 1 -- 172.21.15.167:0/1302131754 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f666c017070 con 0x7f667c0cd610 2024-05-11T08:06:28.859 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:28.857+0000 7f66727fc640 1 -- 172.21.15.167:0/1302131754 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f666c005ce0 con 0x7f667c0cd610 2024-05-11T08:06:28.859 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:28.857+0000 7f6683b58640 1 -- 172.21.15.167:0/1302131754 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f667c0cb770 con 0x7f667c0cd610 2024-05-11T08:06:28.859 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:28.857+0000 7f66727fc640 1 -- 172.21.15.167:0/1302131754 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f666c0072d0 con 0x7f667c0cd610 2024-05-11T08:06:28.859 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:28.857+0000 7f6683b58640 1 -- 172.21.15.167:0/1302131754 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f667c0cbd30 con 0x7f667c0cd610 2024-05-11T08:06:28.860 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:28.858+0000 7f6683b58640 1 -- 172.21.15.167:0/1302131754 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f667c0cc4f0 con 0x7f667c0cd610 2024-05-11T08:06:28.860 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:28.859+0000 7f66727fc640 1 -- 172.21.15.167:0/1302131754 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f666c01a070 con 0x7f667c0cd610 2024-05-11T08:06:28.860 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:28.860+0000 7f66727fc640 1 --2- 172.21.15.167:0/1302131754 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f66600762d0 0x7f6660078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:28.861 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:28.860+0000 7f66818f4640 1 --2- 172.21.15.167:0/1302131754 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f66600762d0 0x7f6660078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:28.861 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:28.860+0000 7f66727fc640 1 -- 172.21.15.167:0/1302131754 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f666c096b30 con 0x7f667c0cd610 2024-05-11T08:06:28.861 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:28.860+0000 7f66818f4640 1 --2- 172.21.15.167:0/1302131754 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f66600762d0 0x7f6660078790 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f6678034000 tx=0x7f667802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:28.865 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:28.864+0000 7f66727fc640 1 -- 172.21.15.167:0/1302131754 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f667c0cc4f0 con 0x7f667c0cd610 2024-05-11T08:06:29.083 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:29.083+0000 7f6683b58640 1 -- 172.21.15.167:0/1302131754 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f667c069590 con 0x7f667c0cd610 2024-05-11T08:06:29.084 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:29.084+0000 7f66727fc640 1 -- 172.21.15.167:0/1302131754 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7f666c05fa80 con 0x7f667c0cd610 2024-05-11T08:06:29.084 INFO:teuthology.orchestra.run.smithi167.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-05-11T08:06:29.086 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:29.086+0000 7f6683b58640 1 -- 172.21.15.167:0/1302131754 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f66600762d0 msgr2=0x7f6660078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:29.086 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:29.086+0000 7f6683b58640 1 --2- 172.21.15.167:0/1302131754 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f66600762d0 0x7f6660078790 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f6678034000 tx=0x7f667802e000 comp rx=0 tx=0).stop 2024-05-11T08:06:29.086 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:29.086+0000 7f6683b58640 1 -- 172.21.15.167:0/1302131754 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f667c0cd610 msgr2=0x7f667c15cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:29.086 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:29.086+0000 7f6683b58640 1 --2- 172.21.15.167:0/1302131754 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f667c0cd610 0x7f667c15cea0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f666c00c410 tx=0x7f666c00c810 comp rx=0 tx=0).stop 2024-05-11T08:06:29.086 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:29.086+0000 7f6683b58640 1 -- 172.21.15.167:0/1302131754 shutdown_connections 2024-05-11T08:06:29.086 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:29.086+0000 7f6683b58640 1 --2- 172.21.15.167:0/1302131754 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f66600762d0 0x7f6660078790 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:29.086 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:29.086+0000 7f6683b58640 1 --2- 172.21.15.167:0/1302131754 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f667c0cd9f0 0x7f667c1f83e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:29.087 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:29.086+0000 7f6683b58640 1 --2- 172.21.15.167:0/1302131754 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f667c0cd610 0x7f667c15cea0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:29.087 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:29.086+0000 7f6683b58640 1 --2- 172.21.15.167:0/1302131754 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f667c0b7b90 0x7f667c0cb0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:29.087 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:29.086+0000 7f6683b58640 1 -- 172.21.15.167:0/1302131754 >> 172.21.15.167:0/1302131754 conn(0x7f667c072730 msgr2=0x7f667c15da90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:29.087 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:29.086+0000 7f6683b58640 1 -- 172.21.15.167:0/1302131754 shutdown_connections 2024-05-11T08:06:29.087 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:29.086+0000 7f6683b58640 1 -- 172.21.15.167:0/1302131754 wait complete. 2024-05-11T08:06:29.097 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-11T08:06:29.097 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-11T08:06:29.097 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi204.front.sepia.ceph.com 2024-05-11T08:06:29.097 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-05-11T08:06:29.097 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-05-11T08:06:29.097 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-11T08:06:29.097 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-11T08:06:29.097 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-11T08:06:29.097 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-05-11T08:06:29.097 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T08:06:29.097 DEBUG:teuthology.orchestra.run.smithi204:> ip addr 2024-05-11T08:06:29.114 INFO:teuthology.orchestra.run.smithi204.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-11T08:06:29.114 INFO:teuthology.orchestra.run.smithi204.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-11T08:06:29.114 INFO:teuthology.orchestra.run.smithi204.stdout: inet 127.0.0.1/8 scope host lo 2024-05-11T08:06:29.114 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-05-11T08:06:29.114 INFO:teuthology.orchestra.run.smithi204.stdout: inet6 ::1/128 scope host 2024-05-11T08:06:29.114 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-05-11T08:06:29.114 INFO:teuthology.orchestra.run.smithi204.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-11T08:06:29.114 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 0c:c4:7a:d9:36:2a brd ff:ff:ff:ff:ff:ff 2024-05-11T08:06:29.115 INFO:teuthology.orchestra.run.smithi204.stdout: altname enp7s0f0 2024-05-11T08:06:29.115 INFO:teuthology.orchestra.run.smithi204.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-11T08:06:29.115 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 0c:c4:7a:d9:36:2b brd ff:ff:ff:ff:ff:ff 2024-05-11T08:06:29.115 INFO:teuthology.orchestra.run.smithi204.stdout: altname enp7s0f1 2024-05-11T08:06:29.115 INFO:teuthology.orchestra.run.smithi204.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-11T08:06:29.115 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 0c:c4:7a:88:77:aa brd ff:ff:ff:ff:ff:ff 2024-05-11T08:06:29.115 INFO:teuthology.orchestra.run.smithi204.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-11T08:06:29.115 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 0c:c4:7a:88:77:ab brd ff:ff:ff:ff:ff:ff 2024-05-11T08:06:29.115 INFO:teuthology.orchestra.run.smithi204.stdout: inet 172.21.15.204/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-05-11T08:06:29.115 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft 42699sec preferred_lft 42699sec 2024-05-11T08:06:29.115 INFO:teuthology.orchestra.run.smithi204.stdout: inet6 fe80::ec4:7aff:fe88:77ab/64 scope link 2024-05-11T08:06:29.115 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-05-11T08:06:29.115 INFO:teuthology.orchestra.run.smithi204.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-11T08:06:29.115 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether d2:93:a6:6a:72:07 brd ff:ff:ff:ff:ff:ff 2024-05-11T08:06:29.116 INFO:teuthology.orchestra.run.smithi204.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-11T08:06:29.116 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-05-11T08:06:29.116 INFO:teuthology.orchestra.run.smithi204.stdout: inet6 fe80::7c06:dfff:fef5:c7be/64 scope link tentative 2024-05-11T08:06:29.116 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-05-11T08:06:29.116 INFO:teuthology.orchestra.run.smithi204.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-11T08:06:29.116 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether d2:93:a6:6a:72:07 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-11T08:06:29.116 INFO:teuthology.orchestra.run.smithi204.stdout: inet6 fe80::d093:a6ff:fe6a:7207/64 scope link tentative 2024-05-11T08:06:29.116 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-05-11T08:06:29.116 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T08:06:29.116 DEBUG:teuthology.orchestra.run.smithi204:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-11T08:06:29.179 INFO:teuthology.orchestra.run.smithi204.stdout:1 2024-05-11T08:06:29.181 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T08:06:29.181 DEBUG:teuthology.orchestra.run.smithi204:> route 2024-05-11T08:06:29.237 INFO:teuthology.orchestra.run.smithi204.stdout:Kernel IP routing table 2024-05-11T08:06:29.238 INFO:teuthology.orchestra.run.smithi204.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-11T08:06:29.238 INFO:teuthology.orchestra.run.smithi204.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-11T08:06:29.238 INFO:teuthology.orchestra.run.smithi204.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-11T08:06:29.238 INFO:teuthology.orchestra.run.smithi204.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-11T08:06:29.238 DEBUG:teuthology.orchestra.run.smithi204:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-11T08:06:29.238 DEBUG:teuthology.orchestra.run.smithi204:> set -e 2024-05-11T08:06:29.238 DEBUG:teuthology.orchestra.run.smithi204:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-11T08:06:29.238 DEBUG:teuthology.orchestra.run.smithi204:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-11T08:06:29.238 DEBUG:teuthology.orchestra.run.smithi204:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-11T08:06:29.239 DEBUG:teuthology.orchestra.run.smithi204:> ') 2024-05-11T08:06:29.327 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:06:29 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-11T08:06:29.360 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:06:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T08:06:29.365 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T08:06:29.366 DEBUG:teuthology.orchestra.run.smithi204:> ip netns list 2024-05-11T08:06:29.382 INFO:teuthology.orchestra.run.smithi204.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-05-11T08:06:29.382 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T08:06:29.383 DEBUG:teuthology.orchestra.run.smithi204:> ip netns list-id 2024-05-11T08:06:29.437 INFO:teuthology.orchestra.run.smithi204.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-05-11T08:06:29.437 DEBUG:teuthology.orchestra.run.smithi204:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-11T08:06:29.437 DEBUG:teuthology.orchestra.run.smithi204:> set -e 2024-05-11T08:06:29.438 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-05-11T08:06:29.438 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-05-11T08:06:29.438 DEBUG:teuthology.orchestra.run.smithi204:> ') 2024-05-11T08:06:29.527 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:06:29 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-11T08:06:29.550 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:06:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T08:06:29.556 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T08:06:29.557 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-05-11T08:06:29.582 INFO:teuthology.orchestra.run.smithi204.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-11T08:06:29.582 INFO:teuthology.orchestra.run.smithi204.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-11T08:06:29.582 INFO:teuthology.orchestra.run.smithi204.stdout: inet 127.0.0.1/8 scope host lo 2024-05-11T08:06:29.582 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-05-11T08:06:29.582 INFO:teuthology.orchestra.run.smithi204.stdout: inet6 ::1/128 scope host 2024-05-11T08:06:29.582 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-05-11T08:06:29.582 INFO:teuthology.orchestra.run.smithi204.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-11T08:06:29.582 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether ca:8c:5e:cf:0f:1d brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-05-11T08:06:29.582 INFO:teuthology.orchestra.run.smithi204.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-05-11T08:06:29.582 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-05-11T08:06:29.582 INFO:teuthology.orchestra.run.smithi204.stdout: inet6 fe80::c88c:5eff:fecf:f1d/64 scope link tentative 2024-05-11T08:06:29.583 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-05-11T08:06:29.583 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T08:06:29.584 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-05-11T08:06:29.649 INFO:teuthology.orchestra.run.smithi204.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-11T08:06:29.649 INFO:teuthology.orchestra.run.smithi204.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-11T08:06:29.649 INFO:teuthology.orchestra.run.smithi204.stdout: inet 127.0.0.1/8 scope host lo 2024-05-11T08:06:29.649 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-05-11T08:06:29.649 INFO:teuthology.orchestra.run.smithi204.stdout: inet6 ::1/128 scope host 2024-05-11T08:06:29.649 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-05-11T08:06:29.649 INFO:teuthology.orchestra.run.smithi204.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-11T08:06:29.649 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether ca:8c:5e:cf:0f:1d brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-05-11T08:06:29.649 INFO:teuthology.orchestra.run.smithi204.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-05-11T08:06:29.649 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-05-11T08:06:29.649 INFO:teuthology.orchestra.run.smithi204.stdout: inet6 fe80::c88c:5eff:fecf:f1d/64 scope link tentative 2024-05-11T08:06:29.649 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-05-11T08:06:29.650 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-05-11T08:06:29.650 DEBUG:teuthology.orchestra.run.smithi204:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-11T08:06:29.651 DEBUG:teuthology.orchestra.run.smithi204:> set -e 2024-05-11T08:06:29.651 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-05-11T08:06:29.651 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-05-11T08:06:29.651 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-05-11T08:06:29.651 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-05-11T08:06:29.651 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-05-11T08:06:29.651 DEBUG:teuthology.orchestra.run.smithi204:> ') 2024-05-11T08:06:29.739 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:06:29 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-11T08:06:29.797 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:06:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T08:06:29.803 DEBUG:teuthology.orchestra.run.smithi204:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-11T08:06:29.803 DEBUG:teuthology.orchestra.run.smithi204:> set -e 2024-05-11T08:06:29.803 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip link set brx.1 up 2024-05-11T08:06:29.803 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip link set dev brx.1 master ceph-brx 2024-05-11T08:06:29.803 DEBUG:teuthology.orchestra.run.smithi204:> ') 2024-05-11T08:06:29.853 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:06:29 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-11T08:06:29.875 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:06:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T08:06:29.881 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-11T08:06:29.881 DEBUG:teuthology.orchestra.run.smithi204:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-05-11T08:06:29.896 INFO:teuthology.orchestra.run.smithi204.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-05-11T08:06:29.897 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-11T08:06:29.897 DEBUG:teuthology.orchestra.run.smithi204:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-05-11T08:06:29.952 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-05-11T08:06:29.952 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T08:06:29.952 DEBUG:teuthology.orchestra.run.smithi204:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-05-11T08:06:30.036 INFO:teuthology.orchestra.run.smithi204.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-05-11T08:06:30.036 INFO:teuthology.orchestra.run.smithi204.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-05-11T08:06:30.036 INFO:teuthology.orchestra.run.smithi204.stdout:invalid new device string format 2024-05-11T08:06:30.044 INFO:teuthology.orchestra.run.smithi204.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-05-11T08:06:30.044 INFO:teuthology.orchestra.run.smithi204.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-05-11T08:06:30.045 INFO:teuthology.orchestra.run.smithi204.stdout:invalid new device string format 2024-05-11T08:06:30.045 INFO:teuthology.orchestra.run.smithi204.stdout:mount.ceph: resolved to: "172.21.15.167:3300,172.21.15.204:3300,172.21.15.204:3301" 2024-05-11T08:06:30.045 INFO:teuthology.orchestra.run.smithi204.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-11T08:06:30.050 INFO:teuthology.orchestra.run.smithi204.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-05-11T08:06:30.050 INFO:teuthology.orchestra.run.smithi204.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-05-11T08:06:30.050 INFO:teuthology.orchestra.run.smithi204.stdout:invalid new device string format 2024-05-11T08:06:30.050 INFO:teuthology.orchestra.run.smithi204.stdout:mount.ceph: resolved to: "172.21.15.167:3300,172.21.15.204:3300,172.21.15.204:3301" 2024-05-11T08:06:30.050 INFO:teuthology.orchestra.run.smithi204.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-11T08:06:30.050 INFO:teuthology.orchestra.run.smithi204.stdout:mount.ceph: resolved to: "172.21.15.167:6789,172.21.15.204:6789,172.21.15.204:6790" 2024-05-11T08:06:30.050 INFO:teuthology.orchestra.run.smithi204.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.167:6789,172.21.15.204:6789,172.21.15.204:6790:/ 2024-05-11T08:06:30.051 INFO:teuthology.orchestra.run.smithi204.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs,key=3,fsid=2034778d-8ff1-4823-aa44-ded32b518241" will pass to kernel 2024-05-11T08:06:30.051 INFO:teuthology.orchestra.run.smithi204.stdout:mount: /home/ubuntu/cephtest/mnt.3 does not contain SELinux labels. 2024-05-11T08:06:30.051 INFO:teuthology.orchestra.run.smithi204.stdout: You just mounted a file system that supports labels which does not 2024-05-11T08:06:30.051 INFO:teuthology.orchestra.run.smithi204.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-11T08:06:30.051 INFO:teuthology.orchestra.run.smithi204.stdout: applications will generate AVC messages and not be allowed access to 2024-05-11T08:06:30.051 INFO:teuthology.orchestra.run.smithi204.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-11T08:06:30.053 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-11T08:06:30.053 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T08:06:30.053 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-05-11T08:06:30.077 DEBUG:teuthology.orchestra.run.smithi204:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-11T08:06:30.167 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:06:30 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-11T08:06:30.168 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:06:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T08:06:30.175 INFO:teuthology.orchestra.run.smithi204.stdout:client4767 2024-05-11T08:06:30.175 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-11T08:06:30.175 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T08:06:30.191 DEBUG:teuthology.orchestra.run.smithi204:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-11T08:06:30.281 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:06:30 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-11T08:06:30.282 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:06:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T08:06:30.288 INFO:teuthology.orchestra.run.smithi204.stdout:client4767 2024-05-11T08:06:30.288 DEBUG:teuthology.orchestra.run.smithi204:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-11T08:06:30.337 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:06:30 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-11T08:06:30.339 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:06:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T08:06:30.345 INFO:teuthology.orchestra.run.smithi204.stdout:client4767 2024-05-11T08:06:30.345 DEBUG:teuthology.orchestra.run.smithi204:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/2034778d-8ff1-4823-aa44-ded32b518241.client4767/status') 2024-05-11T08:06:30.395 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:06:30 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-11T08:06:30.404 INFO:teuthology.orchestra.run.smithi204.stdout:instance: client.4767 (3)172.21.15.204:0/4013607971 2024-05-11T08:06:30.405 INFO:teuthology.orchestra.run.smithi204.stdout:blocklisted: false 2024-05-11T08:06:30.405 INFO:teuthology.orchestra.run.smithi204.stderr:0+1 records in 2024-05-11T08:06:30.405 INFO:teuthology.orchestra.run.smithi204.stderr:0+1 records out 2024-05-11T08:06:30.405 INFO:teuthology.orchestra.run.smithi204.stderr:71 bytes copied, 2.2113e-05 s, 3.2 MB/s 2024-05-11T08:06:30.405 INFO:teuthology.orchestra.run.smithi204.stderr:2024-05-11T08:06:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T08:06:30.412 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-05-11T08:06:30.463 INFO:tasks.cephfs_test_runner:Starting test: test_backtrace (tasks.cephfs.test_backtrace.TestBacktrace) That the 'parent' 'layout' and 'symlink' xattrs on the head objects of files 2024-05-11T08:06:30.463 INFO:tasks.cephfs_test_runner:test_backtrace (tasks.cephfs.test_backtrace.TestBacktrace) 2024-05-11T08:06:30.463 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-11T08:06:30.576 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:30.576+0000 7f630c42a640 1 Processor -- start 2024-05-11T08:06:30.576 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:30.576+0000 7f630c42a640 1 -- start start 2024-05-11T08:06:30.579 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:30.577+0000 7f630c42a640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f63040b8a40 0x7f6304160540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:30.579 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:30.577+0000 7f630c42a640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f63040b5e90 0x7f63040cfba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:30.579 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:30.577+0000 7f630c42a640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f63040d00e0 0x7f63040d0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:30.579 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:30.577+0000 7f630c42a640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f6304160a80 con 0x7f63040b5e90 2024-05-11T08:06:30.579 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:30.577+0000 7f630c42a640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f63040d24a0 con 0x7f63040d00e0 2024-05-11T08:06:30.580 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:30.577+0000 7f630c42a640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f63040cf3e0 con 0x7f63040b8a40 2024-05-11T08:06:30.580 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:30.577+0000 7f63099c5640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f63040b5e90 0x7f63040cfba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:30.580 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:30.577+0000 7f63099c5640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f63040b5e90 0x7f63040cfba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:58576/0 (socket says 172.21.15.167:58576) 2024-05-11T08:06:30.580 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:30.577+0000 7f63099c5640 1 -- 172.21.15.167:0/292451081 learned_addr learned my addr 172.21.15.167:0/292451081 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:30.580 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:30.577+0000 7f630a9c7640 1 --2- 172.21.15.167:0/292451081 >> v2:172.21.15.204:3300/0 conn(0x7f63040d00e0 0x7f63040d0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:30.580 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:30.577+0000 7f630a1c6640 1 --2- 172.21.15.167:0/292451081 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f63040b8a40 0x7f6304160540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:30.580 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:30.578+0000 7f63099c5640 1 -- 172.21.15.167:0/292451081 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f63040b8a40 msgr2=0x7f6304160540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:30.580 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:30.578+0000 7f63099c5640 1 --2- 172.21.15.167:0/292451081 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f63040b8a40 0x7f6304160540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:30.580 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:30.578+0000 7f63099c5640 1 -- 172.21.15.167:0/292451081 >> v2:172.21.15.204:3300/0 conn(0x7f63040d00e0 msgr2=0x7f63040d0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:30.580 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:30.578+0000 7f63099c5640 1 --2- 172.21.15.167:0/292451081 >> v2:172.21.15.204:3300/0 conn(0x7f63040d00e0 0x7f63040d0ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:30.580 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:30.578+0000 7f63099c5640 1 -- 172.21.15.167:0/292451081 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63040d2740 con 0x7f63040b5e90 2024-05-11T08:06:30.580 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:30.578+0000 7f630a9c7640 1 --2- 172.21.15.167:0/292451081 >> v2:172.21.15.204:3300/0 conn(0x7f63040d00e0 0x7f63040d0ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T08:06:30.581 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:30.578+0000 7f630a1c6640 1 --2- 172.21.15.167:0/292451081 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f63040b8a40 0x7f6304160540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T08:06:30.581 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:30.578+0000 7f63099c5640 1 --2- 172.21.15.167:0/292451081 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f63040b5e90 0x7f63040cfba0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f63040d3f70 tx=0x7f62f402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86361beb7befe9c9 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:30.581 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:30.578+0000 7f63091c4640 1 -- 172.21.15.167:0/292451081 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62f4037070 con 0x7f63040b5e90 2024-05-11T08:06:30.581 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:30.578+0000 7f63091c4640 1 -- 172.21.15.167:0/292451081 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62f4032440 con 0x7f63040b5e90 2024-05-11T08:06:30.581 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:30.578+0000 7f63091c4640 1 -- 172.21.15.167:0/292451081 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62f4032810 con 0x7f63040b5e90 2024-05-11T08:06:30.581 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:30.579+0000 7f630c42a640 1 -- 172.21.15.167:0/292451081 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f63040b5e90 msgr2=0x7f63040cfba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:30.581 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:30.579+0000 7f630c42a640 1 --2- 172.21.15.167:0/292451081 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f63040b5e90 0x7f63040cfba0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f63040d3f70 tx=0x7f62f402a140 comp rx=0 tx=0).stop 2024-05-11T08:06:30.581 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:30.579+0000 7f630c42a640 1 -- 172.21.15.167:0/292451081 shutdown_connections 2024-05-11T08:06:30.581 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:30.579+0000 7f630c42a640 1 --2- 172.21.15.167:0/292451081 >> v2:172.21.15.204:3300/0 conn(0x7f63040d00e0 0x7f63040d0ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:30.581 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:30.579+0000 7f630c42a640 1 --2- 172.21.15.167:0/292451081 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f63040b5e90 0x7f63040cfba0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:30.581 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:30.579+0000 7f630c42a640 1 --2- 172.21.15.167:0/292451081 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f63040b8a40 0x7f6304160540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:30.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:30.579+0000 7f630c42a640 1 -- 172.21.15.167:0/292451081 >> 172.21.15.167:0/292451081 conn(0x7f6304072730 msgr2=0x7f63040d2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:30.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:30.579+0000 7f630c42a640 1 -- 172.21.15.167:0/292451081 shutdown_connections 2024-05-11T08:06:30.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:30.579+0000 7f630c42a640 1 -- 172.21.15.167:0/292451081 wait complete. 2024-05-11T08:06:30.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:30.580+0000 7f630c42a640 1 Processor -- start 2024-05-11T08:06:30.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:30.580+0000 7f630c42a640 1 -- start start 2024-05-11T08:06:30.585 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:30.581+0000 7f630c42a640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f63040b5e90 0x7f630414c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:30.585 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:30.581+0000 7f630c42a640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f63040b8a40 0x7f630415f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:30.585 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:30.581+0000 7f630c42a640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f63040d00e0 0x7f63040cc520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:30.585 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:30.581+0000 7f630c42a640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f63040b8e20 con 0x7f63040b8a40 2024-05-11T08:06:30.585 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:30.581+0000 7f630c42a640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f63040cf680 con 0x7f63040b5e90 2024-05-11T08:06:30.585 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:30.581+0000 7f630c42a640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f63040b8fa0 con 0x7f63040d00e0 2024-05-11T08:06:30.585 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:30.581+0000 7f63099c5640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f63040b8a40 0x7f630415f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:30.585 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:30.581+0000 7f63099c5640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f63040b8a40 0x7f630415f2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:58580/0 (socket says 172.21.15.167:58580) 2024-05-11T08:06:30.585 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:30.581+0000 7f63099c5640 1 -- 172.21.15.167:0/3761658690 learned_addr learned my addr 172.21.15.167:0/3761658690 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:30.585 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:30.581+0000 7f630a9c7640 1 --2- 172.21.15.167:0/3761658690 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f63040d00e0 0x7f63040cc520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:30.585 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:30.581+0000 7f630a1c6640 1 --2- 172.21.15.167:0/3761658690 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f63040b5e90 0x7f630414c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:30.585 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:30.582+0000 7f63099c5640 1 -- 172.21.15.167:0/3761658690 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f63040d00e0 msgr2=0x7f63040cc520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:30.585 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:30.582+0000 7f63099c5640 1 --2- 172.21.15.167:0/3761658690 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f63040d00e0 0x7f63040cc520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:30.586 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:30.582+0000 7f63099c5640 1 -- 172.21.15.167:0/3761658690 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f63040b5e90 msgr2=0x7f630414c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:30.586 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:30.582+0000 7f63099c5640 1 --2- 172.21.15.167:0/3761658690 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f63040b5e90 0x7f630414c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:30.586 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:30.582+0000 7f63099c5640 1 -- 172.21.15.167:0/3761658690 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f630414c930 con 0x7f63040b8a40 2024-05-11T08:06:30.586 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:30.582+0000 7f630a9c7640 1 --2- 172.21.15.167:0/3761658690 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f63040d00e0 0x7f63040cc520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T08:06:30.586 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:30.582+0000 7f630a1c6640 1 --2- 172.21.15.167:0/3761658690 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f63040b5e90 0x7f630414c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T08:06:30.586 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:30.582+0000 7f63099c5640 1 --2- 172.21.15.167:0/3761658690 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f63040b8a40 0x7f630415f2e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f62f402a180 tx=0x7f62f40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:30.586 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:30.582+0000 7f62faffd640 1 -- 172.21.15.167:0/3761658690 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62f4041020 con 0x7f63040b8a40 2024-05-11T08:06:30.586 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:30.582+0000 7f62faffd640 1 -- 172.21.15.167:0/3761658690 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62f40306c0 con 0x7f63040b8a40 2024-05-11T08:06:30.586 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:30.582+0000 7f630c42a640 1 -- 172.21.15.167:0/3761658690 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f630415ac80 con 0x7f63040b8a40 2024-05-11T08:06:30.586 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:30.582+0000 7f62faffd640 1 -- 172.21.15.167:0/3761658690 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62f4037040 con 0x7f63040b8a40 2024-05-11T08:06:30.586 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:30.582+0000 7f630c42a640 1 -- 172.21.15.167:0/3761658690 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f630415b240 con 0x7f63040b8a40 2024-05-11T08:06:30.586 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:30.583+0000 7f630c42a640 1 -- 172.21.15.167:0/3761658690 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6304069590 con 0x7f63040b8a40 2024-05-11T08:06:30.586 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:30.583+0000 7f62faffd640 1 -- 172.21.15.167:0/3761658690 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f62f4044020 con 0x7f63040b8a40 2024-05-11T08:06:30.587 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:30.584+0000 7f62faffd640 1 --2- 172.21.15.167:0/3761658690 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f62e80762d0 0x7f62e8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:30.587 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:30.584+0000 7f62faffd640 1 -- 172.21.15.167:0/3761658690 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f62f40b80b0 con 0x7f63040b8a40 2024-05-11T08:06:30.587 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:30.584+0000 7f630a1c6640 1 --2- 172.21.15.167:0/3761658690 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f62e80762d0 0x7f62e8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:30.587 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:30.584+0000 7f630a1c6640 1 --2- 172.21.15.167:0/3761658690 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f62e80762d0 0x7f62e8078790 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f63000044f0 tx=0x7f6300003a30 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:30.588 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:30.588+0000 7f62faffd640 1 -- 172.21.15.167:0/3761658690 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f62f404b030 con 0x7f63040b8a40 2024-05-11T08:06:30.769 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:30.769+0000 7f630c42a640 1 -- 172.21.15.167:0/3761658690 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6304071960 con 0x7f63040b8a40 2024-05-11T08:06:30.770 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:30.770+0000 7f62faffd640 1 -- 172.21.15.167:0/3761658690 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18562 (secure 0 0 0) 0x7f630414c930 con 0x7f63040b8a40 2024-05-11T08:06:30.770 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T08:06:30.771 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":27,"fsid":"2034778d-8ff1-4823-aa44-ded32b518241","created":"2024-05-11T08:05:31.612421+0000","modified":"2024-05-11T08:05:59.756806+0000","last_up_change":"2024-05-11T08:05:42.557382+0000","last_in_change":"2024-05-11T08:05:33.757311+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-11T08:05:43.931263+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-11T08:05:52.573726+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-11T08:05:53.670746+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-11T08:05:57.380272+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"587eaced-5ded-406d-884b-30c1caa8686b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":3863307356},{"type":"v1","addr":"172.21.15.167:6806","nonce":3863307356}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":3863307356},{"type":"v1","addr":"172.21.15.167:6810","nonce":3863307356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":3863307356},{"type":"v1","addr":"172.21.15.167:6815","nonce":3863307356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":3863307356},{"type":"v1","addr":"172.21.15.167:6813","nonce":3863307356}]},"public_addr":"172.21.15.167:6806/3863307356","cluster_addr":"172.21.15.167:6810/3863307356","heartbeat_back_addr":"172.21.15.167:6815/3863307356","heartbeat_front_addr":"172.21.15.167:6813/3863307356","state":["exists","up"]},{"osd":1,"uuid":"37c59104-2dd0-4cf9-85ea-56636f6b02a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":1585024048},{"type":"v1","addr":"172.21.15.167:6817","nonce":1585024048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1585024048},{"type":"v1","addr":"172.21.15.167:6819","nonce":1585024048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1585024048},{"type":"v1","addr":"172.21.15.167:6823","nonce":1585024048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":1585024048},{"type":"v1","addr":"172.21.15.167:6821","nonce":1585024048}]},"public_addr":"172.21.15.167:6817/1585024048","cluster_addr":"172.21.15.167:6819/1585024048","heartbeat_back_addr":"172.21.15.167:6823/1585024048","heartbeat_front_addr":"172.21.15.167:6821/1585024048","state":["exists","up"]},{"osd":2,"uuid":"2bb4f7f5-c698-451b-b4bb-78e716921291","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":3242166900},{"type":"v1","addr":"172.21.15.167:6801","nonce":3242166900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":3242166900},{"type":"v1","addr":"172.21.15.167:6803","nonce":3242166900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6809","nonce":3242166900},{"type":"v1","addr":"172.21.15.167:6811","nonce":3242166900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6805","nonce":3242166900},{"type":"v1","addr":"172.21.15.167:6807","nonce":3242166900}]},"public_addr":"172.21.15.167:6801/3242166900","cluster_addr":"172.21.15.167:6803/3242166900","heartbeat_back_addr":"172.21.15.167:6811/3242166900","heartbeat_front_addr":"172.21.15.167:6807/3242166900","state":["exists","up"]},{"osd":3,"uuid":"9dd26759-a338-49d2-8103-45d068b9c000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":453689293},{"type":"v1","addr":"172.21.15.167:6825","nonce":453689293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":453689293},{"type":"v1","addr":"172.21.15.167:6827","nonce":453689293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":453689293},{"type":"v1","addr":"172.21.15.167:6831","nonce":453689293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":453689293},{"type":"v1","addr":"172.21.15.167:6829","nonce":453689293}]},"public_addr":"172.21.15.167:6825/453689293","cluster_addr":"172.21.15.167:6827/453689293","heartbeat_back_addr":"172.21.15.167:6831/453689293","heartbeat_front_addr":"172.21.15.167:6829/453689293","state":["exists","up"]},{"osd":4,"uuid":"cb27250d-2a80-4c7b-8cef-3a7af05c8a16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1311831500},{"type":"v1","addr":"172.21.15.204:6801","nonce":1311831500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1311831500},{"type":"v1","addr":"172.21.15.204:6803","nonce":1311831500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1311831500},{"type":"v1","addr":"172.21.15.204:6807","nonce":1311831500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1311831500},{"type":"v1","addr":"172.21.15.204:6805","nonce":1311831500}]},"public_addr":"172.21.15.204:6801/1311831500","cluster_addr":"172.21.15.204:6803/1311831500","heartbeat_back_addr":"172.21.15.204:6807/1311831500","heartbeat_front_addr":"172.21.15.204:6805/1311831500","state":["exists","up"]},{"osd":5,"uuid":"92998541-ffec-430b-ab76-8c3cb7c2752d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2921847521},{"type":"v1","addr":"172.21.15.204:6825","nonce":2921847521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2921847521},{"type":"v1","addr":"172.21.15.204:6827","nonce":2921847521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2921847521},{"type":"v1","addr":"172.21.15.204:6831","nonce":2921847521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2921847521},{"type":"v1","addr":"172.21.15.204:6829","nonce":2921847521}]},"public_addr":"172.21.15.204:6825/2921847521","cluster_addr":"172.21.15.204:6827/2921847521","heartbeat_back_addr":"172.21.15.204:6831/2921847521","heartbeat_front_addr":"172.21.15.204:6829/2921847521","state":["exists","up"]},{"osd":6,"uuid":"9543479c-2ec5-42a8-8933-707bfed2e048","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":2889851077},{"type":"v1","addr":"172.21.15.204:6817","nonce":2889851077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":2889851077},{"type":"v1","addr":"172.21.15.204:6819","nonce":2889851077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":2889851077},{"type":"v1","addr":"172.21.15.204:6823","nonce":2889851077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":2889851077},{"type":"v1","addr":"172.21.15.204:6821","nonce":2889851077}]},"public_addr":"172.21.15.204:6817/2889851077","cluster_addr":"172.21.15.204:6819/2889851077","heartbeat_back_addr":"172.21.15.204:6823/2889851077","heartbeat_front_addr":"172.21.15.204:6821/2889851077","state":["exists","up"]},{"osd":7,"uuid":"c45a5647-8358-409e-af91-c05dda65f0b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":1949534184},{"type":"v1","addr":"172.21.15.204:6809","nonce":1949534184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":1949534184},{"type":"v1","addr":"172.21.15.204:6811","nonce":1949534184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1949534184},{"type":"v1","addr":"172.21.15.204:6815","nonce":1949534184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1949534184},{"type":"v1","addr":"172.21.15.204:6813","nonce":1949534184}]},"public_addr":"172.21.15.204:6809/1949534184","cluster_addr":"172.21.15.204:6811/1949534184","heartbeat_back_addr":"172.21.15.204:6815/1949534184","heartbeat_front_addr":"172.21.15.204:6813/1949534184","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.952253+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.940689+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.860682+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.929350+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.901627+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.957451+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.987249+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.951197+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-11T08:06:30.773 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:30.772+0000 7f630c42a640 1 -- 172.21.15.167:0/3761658690 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f62e80762d0 msgr2=0x7f62e8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:30.773 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:30.772+0000 7f630c42a640 1 --2- 172.21.15.167:0/3761658690 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f62e80762d0 0x7f62e8078790 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f63000044f0 tx=0x7f6300003a30 comp rx=0 tx=0).stop 2024-05-11T08:06:30.773 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:30.772+0000 7f630c42a640 1 -- 172.21.15.167:0/3761658690 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f63040b8a40 msgr2=0x7f630415f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:30.773 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:30.772+0000 7f630c42a640 1 --2- 172.21.15.167:0/3761658690 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f63040b8a40 0x7f630415f2e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f62f402a180 tx=0x7f62f40046c0 comp rx=0 tx=0).stop 2024-05-11T08:06:30.773 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:30.772+0000 7f630c42a640 1 -- 172.21.15.167:0/3761658690 shutdown_connections 2024-05-11T08:06:30.773 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:30.772+0000 7f630c42a640 1 --2- 172.21.15.167:0/3761658690 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f62e80762d0 0x7f62e8078790 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:30.773 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:30.772+0000 7f630c42a640 1 --2- 172.21.15.167:0/3761658690 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f63040d00e0 0x7f63040cc520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:30.773 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:30.772+0000 7f630c42a640 1 --2- 172.21.15.167:0/3761658690 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f63040b8a40 0x7f630415f2e0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:30.773 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:30.772+0000 7f630c42a640 1 --2- 172.21.15.167:0/3761658690 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f63040b5e90 0x7f630414c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:30.773 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:30.772+0000 7f630c42a640 1 -- 172.21.15.167:0/3761658690 >> 172.21.15.167:0/3761658690 conn(0x7f6304072730 msgr2=0x7f63040d2fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:30.773 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:30.772+0000 7f630c42a640 1 -- 172.21.15.167:0/3761658690 shutdown_connections 2024-05-11T08:06:30.773 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:30.772+0000 7f630c42a640 1 -- 172.21.15.167:0/3761658690 wait complete. 2024-05-11T08:06:30.784 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-11T08:05:43.931263+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-11T08:05:52.573726+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-11T08:05:53.670746+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-11T08:05:57.380272+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-11T08:06:30.784 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-11T08:06:30.888 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:30.888+0000 7f55ce704640 1 Processor -- start 2024-05-11T08:06:30.889 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:30.888+0000 7f55ce704640 1 -- start start 2024-05-11T08:06:30.890 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:30.889+0000 7f55ce704640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f55c815f910 0x7f55c806c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:30.890 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:30.889+0000 7f55ce704640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f55c80d40a0 0x7f55c8168ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:30.890 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:30.889+0000 7f55ce704640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55c806dcc0 con 0x7f55c806cc30 2024-05-11T08:06:30.890 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:30.889+0000 7f55ce704640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f55c8156820 con 0x7f55c815f910 2024-05-11T08:06:30.890 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:30.889+0000 7f55ce704640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f55c806d140 con 0x7f55c80d40a0 2024-05-11T08:06:30.890 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:30.889+0000 7f55c7fff640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f55c815f910 0x7f55c806c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:30.890 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:30.889+0000 7f55c7fff640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f55c815f910 0x7f55c806c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:58588/0 (socket says 172.21.15.167:58588) 2024-05-11T08:06:30.890 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:30.889+0000 7f55c7fff640 1 -- 172.21.15.167:0/756090607 learned_addr learned my addr 172.21.15.167:0/756090607 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:30.890 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:30.889+0000 7f55ccca1640 1 --2- 172.21.15.167:0/756090607 >> v2:172.21.15.204:3300/0 conn(0x7f55c80d40a0 0x7f55c8168ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:30.891 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:30.890+0000 7f55c7fff640 1 -- 172.21.15.167:0/756090607 >> v2:172.21.15.204:3300/0 conn(0x7f55c80d40a0 msgr2=0x7f55c8168ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:30.891 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:30.890+0000 7f55c7fff640 1 --2- 172.21.15.167:0/756090607 >> v2:172.21.15.204:3300/0 conn(0x7f55c80d40a0 0x7f55c8168ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:30.891 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:30.890+0000 7f55c7fff640 1 -- 172.21.15.167:0/756090607 >> v1:172.21.15.204:6789/0 conn(0x7f55c806cc30 legacy=0x7f55c815fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:30.891 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:30.890+0000 7f55c7fff640 1 -- 172.21.15.167:0/756090607 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55c806d420 con 0x7f55c815f910 2024-05-11T08:06:30.891 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:30.890+0000 7f55ccca1640 1 --2- 172.21.15.167:0/756090607 >> v2:172.21.15.204:3300/0 conn(0x7f55c80d40a0 0x7f55c8168ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T08:06:30.892 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:30.890+0000 7f55c7fff640 1 --2- 172.21.15.167:0/756090607 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f55c815f910 0x7f55c806c6f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f55c80c8ce0 tx=0x7f55b802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fc29660e134d5ec server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:30.892 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:30.890+0000 7f55c6ffd640 1 -- 172.21.15.167:0/756090607 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55b8037070 con 0x7f55c815f910 2024-05-11T08:06:30.892 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:30.890+0000 7f55c6ffd640 1 -- 172.21.15.167:0/756090607 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55b8032440 con 0x7f55c815f910 2024-05-11T08:06:30.893 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:30.890+0000 7f55c6ffd640 1 -- 172.21.15.167:0/756090607 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55b8032810 con 0x7f55c815f910 2024-05-11T08:06:30.893 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:30.891+0000 7f55ce704640 1 -- 172.21.15.167:0/756090607 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f55c815f910 msgr2=0x7f55c806c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:30.893 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:30.891+0000 7f55ce704640 1 --2- 172.21.15.167:0/756090607 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f55c815f910 0x7f55c806c6f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f55c80c8ce0 tx=0x7f55b802a270 comp rx=0 tx=0).stop 2024-05-11T08:06:30.893 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:30.891+0000 7f55ce704640 1 -- 172.21.15.167:0/756090607 shutdown_connections 2024-05-11T08:06:30.893 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:30.891+0000 7f55ce704640 1 --2- 172.21.15.167:0/756090607 >> v2:172.21.15.204:3300/0 conn(0x7f55c80d40a0 0x7f55c8168ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:30.893 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:30.891+0000 7f55ce704640 1 --2- 172.21.15.167:0/756090607 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f55c815f910 0x7f55c806c6f0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:30.893 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:30.891+0000 7f55ce704640 1 -- 172.21.15.167:0/756090607 >> 172.21.15.167:0/756090607 conn(0x7f55c80d51e0 msgr2=0x7f55c80d55f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:30.893 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:30.891+0000 7f55ce704640 1 -- 172.21.15.167:0/756090607 shutdown_connections 2024-05-11T08:06:30.893 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:30.891+0000 7f55ce704640 1 -- 172.21.15.167:0/756090607 wait complete. 2024-05-11T08:06:30.893 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:30.892+0000 7f55ce704640 1 Processor -- start 2024-05-11T08:06:30.893 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:30.892+0000 7f55ce704640 1 -- start start 2024-05-11T08:06:30.895 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:30.893+0000 7f55ce704640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f55c806cc30 0x7f55c81f7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:30.895 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:30.893+0000 7f55ce704640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f55c80d40a0 0x7f55c80c8980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:30.896 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:30.893+0000 7f55ce704640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f55c815f910 0x7f55c8161620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:30.896 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:30.893+0000 7f55ce704640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f55c80d0060 con 0x7f55c815f910 2024-05-11T08:06:30.896 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:30.893+0000 7f55ce704640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f55c806d9e0 con 0x7f55c80d40a0 2024-05-11T08:06:30.896 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:30.893+0000 7f55ce704640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f55c80d01e0 con 0x7f55c806cc30 2024-05-11T08:06:30.896 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:30.893+0000 7f55ccca1640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f55c815f910 0x7f55c8161620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:30.896 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:30.893+0000 7f55ccca1640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f55c815f910 0x7f55c8161620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:58598/0 (socket says 172.21.15.167:58598) 2024-05-11T08:06:30.896 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:30.893+0000 7f55ccca1640 1 -- 172.21.15.167:0/2431807767 learned_addr learned my addr 172.21.15.167:0/2431807767 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:30.896 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:30.893+0000 7f55c7fff640 1 --2- 172.21.15.167:0/2431807767 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f55c806cc30 0x7f55c81f7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:30.896 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:30.893+0000 7f55c77fe640 1 --2- 172.21.15.167:0/2431807767 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f55c80d40a0 0x7f55c80c8980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:30.896 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:30.894+0000 7f55ccca1640 1 -- 172.21.15.167:0/2431807767 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f55c806cc30 msgr2=0x7f55c81f7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:30.896 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:30.894+0000 7f55ccca1640 1 --2- 172.21.15.167:0/2431807767 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f55c806cc30 0x7f55c81f7fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:30.896 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:30.894+0000 7f55ccca1640 1 -- 172.21.15.167:0/2431807767 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f55c80d40a0 msgr2=0x7f55c80c8980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:30.896 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:30.894+0000 7f55ccca1640 1 --2- 172.21.15.167:0/2431807767 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f55c80d40a0 0x7f55c80c8980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:30.897 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:30.894+0000 7f55ccca1640 1 -- 172.21.15.167:0/2431807767 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55c81f8520 con 0x7f55c815f910 2024-05-11T08:06:30.897 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:30.894+0000 7f55c7fff640 1 --2- 172.21.15.167:0/2431807767 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f55c806cc30 0x7f55c81f7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T08:06:30.897 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:30.894+0000 7f55c77fe640 1 --2- 172.21.15.167:0/2431807767 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f55c80d40a0 0x7f55c80c8980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T08:06:30.897 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:30.894+0000 7f55ccca1640 1 --2- 172.21.15.167:0/2431807767 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f55c815f910 0x7f55c8161620 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f55c0007ec0 tx=0x7f55c000d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:30.897 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:30.894+0000 7f55c4ff9640 1 -- 172.21.15.167:0/2431807767 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55c001a070 con 0x7f55c815f910 2024-05-11T08:06:30.897 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:30.894+0000 7f55c4ff9640 1 -- 172.21.15.167:0/2431807767 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55c0003b90 con 0x7f55c815f910 2024-05-11T08:06:30.897 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:30.894+0000 7f55ce704640 1 -- 172.21.15.167:0/2431807767 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55c80c9440 con 0x7f55c815f910 2024-05-11T08:06:30.897 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:30.894+0000 7f55c4ff9640 1 -- 172.21.15.167:0/2431807767 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55c0004030 con 0x7f55c815f910 2024-05-11T08:06:30.897 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:30.894+0000 7f55ce704640 1 -- 172.21.15.167:0/2431807767 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55c80c9a00 con 0x7f55c815f910 2024-05-11T08:06:30.897 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:30.895+0000 7f55ce704640 1 -- 172.21.15.167:0/2431807767 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55c8069520 con 0x7f55c815f910 2024-05-11T08:06:30.897 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:30.896+0000 7f55c4ff9640 1 -- 172.21.15.167:0/2431807767 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f55c001d070 con 0x7f55c815f910 2024-05-11T08:06:30.898 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:30.896+0000 7f55c4ff9640 1 --2- 172.21.15.167:0/2431807767 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f55b40762d0 0x7f55b4078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:30.898 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:30.896+0000 7f55c7fff640 1 --2- 172.21.15.167:0/2431807767 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f55b40762d0 0x7f55b4078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:30.898 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:30.896+0000 7f55c4ff9640 1 -- 172.21.15.167:0/2431807767 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f55c0097be0 con 0x7f55c815f910 2024-05-11T08:06:30.898 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:30.896+0000 7f55c7fff640 1 --2- 172.21.15.167:0/2431807767 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f55b40762d0 0x7f55b4078790 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f55b8034000 tx=0x7f55b802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:30.900 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:30.900+0000 7f55c4ff9640 1 -- 172.21.15.167:0/2431807767 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f55c0010070 con 0x7f55c815f910 2024-05-11T08:06:31.083 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.082+0000 7f55ce704640 1 -- 172.21.15.167:0/2431807767 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f55c80ba520 con 0x7f55c815f910 2024-05-11T08:06:31.083 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.083+0000 7f55c4ff9640 1 -- 172.21.15.167:0/2431807767 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7f55c0061290 con 0x7f55c815f910 2024-05-11T08:06:31.083 INFO:teuthology.orchestra.run.smithi167.stdout:pg_num: 1 2024-05-11T08:06:31.085 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.085+0000 7f55ce704640 1 -- 172.21.15.167:0/2431807767 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f55b40762d0 msgr2=0x7f55b4078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:31.085 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.085+0000 7f55ce704640 1 --2- 172.21.15.167:0/2431807767 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f55b40762d0 0x7f55b4078790 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f55b8034000 tx=0x7f55b802e000 comp rx=0 tx=0).stop 2024-05-11T08:06:31.085 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.085+0000 7f55ce704640 1 -- 172.21.15.167:0/2431807767 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f55c815f910 msgr2=0x7f55c8161620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:31.086 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.085+0000 7f55ce704640 1 --2- 172.21.15.167:0/2431807767 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f55c815f910 0x7f55c8161620 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f55c0007ec0 tx=0x7f55c000d4a0 comp rx=0 tx=0).stop 2024-05-11T08:06:31.086 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.085+0000 7f55ce704640 1 -- 172.21.15.167:0/2431807767 shutdown_connections 2024-05-11T08:06:31.086 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.085+0000 7f55ce704640 1 --2- 172.21.15.167:0/2431807767 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f55b40762d0 0x7f55b4078790 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:31.086 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.085+0000 7f55ce704640 1 --2- 172.21.15.167:0/2431807767 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f55c815f910 0x7f55c8161620 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:31.086 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.085+0000 7f55ce704640 1 --2- 172.21.15.167:0/2431807767 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f55c80d40a0 0x7f55c80c8980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:31.086 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.085+0000 7f55ce704640 1 --2- 172.21.15.167:0/2431807767 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f55c806cc30 0x7f55c81f7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:31.086 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.085+0000 7f55ce704640 1 -- 172.21.15.167:0/2431807767 >> 172.21.15.167:0/2431807767 conn(0x7f55c80d51e0 msgr2=0x7f55c80d55c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:31.086 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.085+0000 7f55ce704640 1 -- 172.21.15.167:0/2431807767 shutdown_connections 2024-05-11T08:06:31.086 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.085+0000 7f55ce704640 1 -- 172.21.15.167:0/2431807767 wait complete. 2024-05-11T08:06:31.097 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-11T08:06:31.201 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.201+0000 7f57de924640 1 Processor -- start 2024-05-11T08:06:31.201 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.201+0000 7f57de924640 1 -- start start 2024-05-11T08:06:31.203 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.202+0000 7f57de924640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f57d81627f0 0x7f57d8160020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:31.203 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.202+0000 7f57de924640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f57d8163a20 0x7f57d80710f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:31.203 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.202+0000 7f57de924640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57d80b4fe0 con 0x7f57d8071630 2024-05-11T08:06:31.203 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.202+0000 7f57de924640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f57d80b52c0 con 0x7f57d81627f0 2024-05-11T08:06:31.203 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.202+0000 7f57de924640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f57d80b55a0 con 0x7f57d8163a20 2024-05-11T08:06:31.203 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.202+0000 7f57d7fff640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f57d81627f0 0x7f57d8160020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:31.203 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.202+0000 7f57d7fff640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f57d81627f0 0x7f57d8160020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:58608/0 (socket says 172.21.15.167:58608) 2024-05-11T08:06:31.203 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.202+0000 7f57d7fff640 1 -- 172.21.15.167:0/7745435 learned_addr learned my addr 172.21.15.167:0/7745435 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:31.203 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.202+0000 7f57d7fff640 1 -- 172.21.15.167:0/7745435 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f57d8163a20 msgr2=0x7f57d80710f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:31.203 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.202+0000 7f57d7fff640 1 --2- 172.21.15.167:0/7745435 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f57d8163a20 0x7f57d80710f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:31.204 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.202+0000 7f57d7fff640 1 -- 172.21.15.167:0/7745435 >> v1:172.21.15.204:6789/0 conn(0x7f57d8071630 legacy=0x7f57d80d0450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:31.204 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.202+0000 7f57d7fff640 1 -- 172.21.15.167:0/7745435 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57d80b5880 con 0x7f57d81627f0 2024-05-11T08:06:31.204 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.203+0000 7f57d7fff640 1 --2- 172.21.15.167:0/7745435 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f57d81627f0 0x7f57d8160020 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f57c8009870 tx=0x7f57c802a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4063ab561d7dd181 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:31.204 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.203+0000 7f57d77fe640 1 -- 172.21.15.167:0/7745435 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57c8037070 con 0x7f57d81627f0 2024-05-11T08:06:31.204 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.203+0000 7f57d77fe640 1 -- 172.21.15.167:0/7745435 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57c8032440 con 0x7f57d81627f0 2024-05-11T08:06:31.204 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.203+0000 7f57d77fe640 1 -- 172.21.15.167:0/7745435 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57c8032810 con 0x7f57d81627f0 2024-05-11T08:06:31.205 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.203+0000 7f57de924640 1 -- 172.21.15.167:0/7745435 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f57d81627f0 msgr2=0x7f57d8160020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:31.205 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.203+0000 7f57de924640 1 --2- 172.21.15.167:0/7745435 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f57d81627f0 0x7f57d8160020 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f57c8009870 tx=0x7f57c802a130 comp rx=0 tx=0).stop 2024-05-11T08:06:31.205 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.204+0000 7f57de924640 1 -- 172.21.15.167:0/7745435 shutdown_connections 2024-05-11T08:06:31.205 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.204+0000 7f57de924640 1 --2- 172.21.15.167:0/7745435 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f57d8163a20 0x7f57d80710f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:31.205 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.204+0000 7f57de924640 1 --2- 172.21.15.167:0/7745435 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f57d81627f0 0x7f57d8160020 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:31.205 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.204+0000 7f57de924640 1 -- 172.21.15.167:0/7745435 >> 172.21.15.167:0/7745435 conn(0x7f57d80d2000 msgr2=0x7f57d80d2410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:31.205 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.204+0000 7f57de924640 1 -- 172.21.15.167:0/7745435 shutdown_connections 2024-05-11T08:06:31.205 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.204+0000 7f57de924640 1 -- 172.21.15.167:0/7745435 wait complete. 2024-05-11T08:06:31.205 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.205+0000 7f57de924640 1 Processor -- start 2024-05-11T08:06:31.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.205+0000 7f57de924640 1 -- start start 2024-05-11T08:06:31.208 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.205+0000 7f57de924640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f57d8071630 0x7f57d814e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:31.208 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.205+0000 7f57de924640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f57d81627f0 0x7f57d814f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:31.208 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.205+0000 7f57de924640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f57d8163a20 0x7f57d814fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:31.209 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.205+0000 7f57de924640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f57d81606e0 con 0x7f57d8163a20 2024-05-11T08:06:31.209 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.205+0000 7f57de924640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f57d8160560 con 0x7f57d8071630 2024-05-11T08:06:31.209 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.205+0000 7f57de924640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f57d8161940 con 0x7f57d81627f0 2024-05-11T08:06:31.209 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.206+0000 7f57dcec1640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f57d8163a20 0x7f57d814fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:31.209 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.206+0000 7f57dcec1640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f57d8163a20 0x7f57d814fef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:58618/0 (socket says 172.21.15.167:58618) 2024-05-11T08:06:31.209 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.206+0000 7f57dcec1640 1 -- 172.21.15.167:0/232503305 learned_addr learned my addr 172.21.15.167:0/232503305 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:31.209 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.206+0000 7f57d7fff640 1 --2- 172.21.15.167:0/232503305 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f57d8071630 0x7f57d814e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:31.209 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.206+0000 7f57cffff640 1 --2- 172.21.15.167:0/232503305 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f57d81627f0 0x7f57d814f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:31.209 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.206+0000 7f57dcec1640 1 -- 172.21.15.167:0/232503305 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f57d81627f0 msgr2=0x7f57d814f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:31.209 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.206+0000 7f57dcec1640 1 --2- 172.21.15.167:0/232503305 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f57d81627f0 0x7f57d814f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:31.209 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.206+0000 7f57dcec1640 1 -- 172.21.15.167:0/232503305 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f57d8071630 msgr2=0x7f57d814e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:31.209 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.206+0000 7f57dcec1640 1 --2- 172.21.15.167:0/232503305 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f57d8071630 0x7f57d814e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:31.209 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.206+0000 7f57dcec1640 1 -- 172.21.15.167:0/232503305 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57d8151e80 con 0x7f57d8163a20 2024-05-11T08:06:31.210 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.206+0000 7f57cffff640 1 --2- 172.21.15.167:0/232503305 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f57d81627f0 0x7f57d814f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T08:06:31.210 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.206+0000 7f57dcec1640 1 --2- 172.21.15.167:0/232503305 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f57d8163a20 0x7f57d814fef0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f57d000c780 tx=0x7f57d000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:31.210 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.206+0000 7f57d57fa640 1 -- 172.21.15.167:0/232503305 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57d0017070 con 0x7f57d8163a20 2024-05-11T08:06:31.210 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.206+0000 7f57d57fa640 1 -- 172.21.15.167:0/232503305 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57d0003da0 con 0x7f57d8163a20 2024-05-11T08:06:31.210 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.206+0000 7f57de924640 1 -- 172.21.15.167:0/232503305 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57d8152160 con 0x7f57d8163a20 2024-05-11T08:06:31.210 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.206+0000 7f57d57fa640 1 -- 172.21.15.167:0/232503305 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57d00052b0 con 0x7f57d8163a20 2024-05-11T08:06:31.210 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.206+0000 7f57de924640 1 -- 172.21.15.167:0/232503305 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57d814e930 con 0x7f57d8163a20 2024-05-11T08:06:31.210 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.207+0000 7f57de924640 1 -- 172.21.15.167:0/232503305 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57d8069bc0 con 0x7f57d8163a20 2024-05-11T08:06:31.210 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.208+0000 7f57d57fa640 1 -- 172.21.15.167:0/232503305 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f57d001a070 con 0x7f57d8163a20 2024-05-11T08:06:31.210 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.208+0000 7f57d57fa640 1 --2- 172.21.15.167:0/232503305 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f57b80762d0 0x7f57b8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:31.210 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.208+0000 7f57d7fff640 1 --2- 172.21.15.167:0/232503305 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f57b80762d0 0x7f57b8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:31.210 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.209+0000 7f57d57fa640 1 -- 172.21.15.167:0/232503305 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f57d0096e20 con 0x7f57d8163a20 2024-05-11T08:06:31.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.209+0000 7f57d7fff640 1 --2- 172.21.15.167:0/232503305 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f57b80762d0 0x7f57b8078790 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f57c8034000 tx=0x7f57c802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:31.213 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.213+0000 7f57d57fa640 1 -- 172.21.15.167:0/232503305 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f57d005fe80 con 0x7f57d8163a20 2024-05-11T08:06:31.394 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.394+0000 7f57de924640 1 -- 172.21.15.167:0/232503305 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f57d814ebf0 con 0x7f57d8163a20 2024-05-11T08:06:31.394 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.394+0000 7f57d57fa640 1 -- 172.21.15.167:0/232503305 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7f57d0064180 con 0x7f57d8163a20 2024-05-11T08:06:31.394 INFO:teuthology.orchestra.run.smithi167.stdout:pg_num: 32 2024-05-11T08:06:31.396 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.396+0000 7f57de924640 1 -- 172.21.15.167:0/232503305 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f57b80762d0 msgr2=0x7f57b8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:31.396 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.396+0000 7f57de924640 1 --2- 172.21.15.167:0/232503305 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f57b80762d0 0x7f57b8078790 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f57c8034000 tx=0x7f57c802e000 comp rx=0 tx=0).stop 2024-05-11T08:06:31.396 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.396+0000 7f57de924640 1 -- 172.21.15.167:0/232503305 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f57d8163a20 msgr2=0x7f57d814fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:31.396 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.396+0000 7f57de924640 1 --2- 172.21.15.167:0/232503305 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f57d8163a20 0x7f57d814fef0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f57d000c780 tx=0x7f57d000cc40 comp rx=0 tx=0).stop 2024-05-11T08:06:31.397 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.396+0000 7f57de924640 1 -- 172.21.15.167:0/232503305 shutdown_connections 2024-05-11T08:06:31.397 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.396+0000 7f57de924640 1 --2- 172.21.15.167:0/232503305 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f57b80762d0 0x7f57b8078790 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:31.397 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.396+0000 7f57de924640 1 --2- 172.21.15.167:0/232503305 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f57d8163a20 0x7f57d814fef0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:31.397 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.396+0000 7f57de924640 1 --2- 172.21.15.167:0/232503305 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f57d81627f0 0x7f57d814f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:31.397 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.396+0000 7f57de924640 1 --2- 172.21.15.167:0/232503305 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f57d8071630 0x7f57d814e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:31.397 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.396+0000 7f57de924640 1 -- 172.21.15.167:0/232503305 >> 172.21.15.167:0/232503305 conn(0x7f57d80d2000 msgr2=0x7f57d815bb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:31.397 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.396+0000 7f57de924640 1 -- 172.21.15.167:0/232503305 shutdown_connections 2024-05-11T08:06:31.397 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.396+0000 7f57de924640 1 -- 172.21.15.167:0/232503305 wait complete. 2024-05-11T08:06:31.407 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-11T08:06:31.511 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.511+0000 7fc332a3e640 1 Processor -- start 2024-05-11T08:06:31.511 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.511+0000 7fc332a3e640 1 -- start start 2024-05-11T08:06:31.513 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.512+0000 7fc332a3e640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fc32c15ac80 0x7fc32c074260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:31.513 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.512+0000 7fc332a3e640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc32c0b3d50 0x7fc32c160c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:31.513 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.512+0000 7fc332a3e640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc32c0b65e0 con 0x7fc32c1611b0 2024-05-11T08:06:31.514 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.512+0000 7fc332a3e640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7fc32c06e3c0 con 0x7fc32c15ac80 2024-05-11T08:06:31.514 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.512+0000 7fc332a3e640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fc32c15b060 con 0x7fc32c0b3d50 2024-05-11T08:06:31.514 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.512+0000 7fc32bfff640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fc32c15ac80 0x7fc32c074260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:31.514 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.512+0000 7fc32bfff640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fc32c15ac80 0x7fc32c074260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:58622/0 (socket says 172.21.15.167:58622) 2024-05-11T08:06:31.514 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.512+0000 7fc32bfff640 1 -- 172.21.15.167:0/1223965740 learned_addr learned my addr 172.21.15.167:0/1223965740 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:31.514 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.513+0000 7fc32bfff640 1 -- 172.21.15.167:0/1223965740 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc32c0b3d50 msgr2=0x7fc32c160c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:31.514 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.513+0000 7fc323fff640 1 --2- 172.21.15.167:0/1223965740 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc32c0b3d50 0x7fc32c160c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:31.514 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.513+0000 7fc32bfff640 1 --2- 172.21.15.167:0/1223965740 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc32c0b3d50 0x7fc32c160c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:31.514 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.513+0000 7fc32bfff640 1 -- 172.21.15.167:0/1223965740 >> v1:172.21.15.204:6789/0 conn(0x7fc32c1611b0 legacy=0x7fc32c15d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:31.514 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.513+0000 7fc32bfff640 1 -- 172.21.15.167:0/1223965740 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc32c0b68c0 con 0x7fc32c15ac80 2024-05-11T08:06:31.514 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.513+0000 7fc323fff640 1 --2- 172.21.15.167:0/1223965740 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc32c0b3d50 0x7fc32c160c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T08:06:31.514 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.513+0000 7fc32bfff640 1 --2- 172.21.15.167:0/1223965740 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc32c15ac80 0x7fc32c074260 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fc31c009870 tx=0x7fc31c02a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db62ce68fa32d6e0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:31.514 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.514+0000 7fc32b7fe640 1 -- 172.21.15.167:0/1223965740 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc31c037070 con 0x7fc32c15ac80 2024-05-11T08:06:31.515 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.514+0000 7fc32b7fe640 1 -- 172.21.15.167:0/1223965740 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc31c032440 con 0x7fc32c15ac80 2024-05-11T08:06:31.515 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.514+0000 7fc32b7fe640 1 -- 172.21.15.167:0/1223965740 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc31c032810 con 0x7fc32c15ac80 2024-05-11T08:06:31.517 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.514+0000 7fc332a3e640 1 -- 172.21.15.167:0/1223965740 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc32c15ac80 msgr2=0x7fc32c074260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:31.518 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.514+0000 7fc332a3e640 1 --2- 172.21.15.167:0/1223965740 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc32c15ac80 0x7fc32c074260 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fc31c009870 tx=0x7fc31c02a1c0 comp rx=0 tx=0).stop 2024-05-11T08:06:31.518 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.514+0000 7fc332a3e640 1 -- 172.21.15.167:0/1223965740 shutdown_connections 2024-05-11T08:06:31.518 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.514+0000 7fc332a3e640 1 --2- 172.21.15.167:0/1223965740 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc32c0b3d50 0x7fc32c160c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:31.518 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.514+0000 7fc332a3e640 1 --2- 172.21.15.167:0/1223965740 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc32c15ac80 0x7fc32c074260 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:31.518 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.514+0000 7fc332a3e640 1 -- 172.21.15.167:0/1223965740 >> 172.21.15.167:0/1223965740 conn(0x7fc32c06c450 msgr2=0x7fc32c06c860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:31.518 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.514+0000 7fc332a3e640 1 -- 172.21.15.167:0/1223965740 shutdown_connections 2024-05-11T08:06:31.518 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.514+0000 7fc332a3e640 1 -- 172.21.15.167:0/1223965740 wait complete. 2024-05-11T08:06:31.518 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.515+0000 7fc332a3e640 1 Processor -- start 2024-05-11T08:06:31.518 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.515+0000 7fc332a3e640 1 -- start start 2024-05-11T08:06:31.518 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.516+0000 7fc332a3e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc32c0b3d50 0x7fc32c15e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:31.518 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.516+0000 7fc332a3e640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc32c15ac80 0x7fc32c15c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:31.518 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.516+0000 7fc332a3e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc32c1611b0 0x7fc32c15c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:31.519 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.516+0000 7fc332a3e640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc32c0b44f0 con 0x7fc32c1611b0 2024-05-11T08:06:31.519 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.516+0000 7fc332a3e640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc32c0b41b0 con 0x7fc32c0b3d50 2024-05-11T08:06:31.519 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.516+0000 7fc332a3e640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fc32c073960 con 0x7fc32c15ac80 2024-05-11T08:06:31.519 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.516+0000 7fc330fdb640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc32c1611b0 0x7fc32c15c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:31.519 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.516+0000 7fc330fdb640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc32c1611b0 0x7fc32c15c9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:58624/0 (socket says 172.21.15.167:58624) 2024-05-11T08:06:31.519 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.516+0000 7fc330fdb640 1 -- 172.21.15.167:0/3806047654 learned_addr learned my addr 172.21.15.167:0/3806047654 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:31.519 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.516+0000 7fc323fff640 1 --2- 172.21.15.167:0/3806047654 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc32c15ac80 0x7fc32c15c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:31.519 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.516+0000 7fc32bfff640 1 --2- 172.21.15.167:0/3806047654 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc32c0b3d50 0x7fc32c15e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:31.519 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.516+0000 7fc330fdb640 1 -- 172.21.15.167:0/3806047654 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc32c15ac80 msgr2=0x7fc32c15c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:31.519 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.516+0000 7fc330fdb640 1 --2- 172.21.15.167:0/3806047654 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc32c15ac80 0x7fc32c15c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:31.519 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.517+0000 7fc330fdb640 1 -- 172.21.15.167:0/3806047654 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc32c0b3d50 msgr2=0x7fc32c15e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:31.519 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.517+0000 7fc330fdb640 1 --2- 172.21.15.167:0/3806047654 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc32c0b3d50 0x7fc32c15e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:31.519 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.517+0000 7fc330fdb640 1 -- 172.21.15.167:0/3806047654 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc32c140630 con 0x7fc32c1611b0 2024-05-11T08:06:31.519 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.517+0000 7fc32bfff640 1 --2- 172.21.15.167:0/3806047654 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc32c0b3d50 0x7fc32c15e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T08:06:31.520 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.517+0000 7fc323fff640 1 --2- 172.21.15.167:0/3806047654 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc32c15ac80 0x7fc32c15c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T08:06:31.520 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.517+0000 7fc330fdb640 1 --2- 172.21.15.167:0/3806047654 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc32c1611b0 0x7fc32c15c9f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fc32400b970 tx=0x7fc32400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:31.520 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.517+0000 7fc3297fa640 1 -- 172.21.15.167:0/3806047654 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc324019070 con 0x7fc32c1611b0 2024-05-11T08:06:31.520 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.517+0000 7fc3297fa640 1 -- 172.21.15.167:0/3806047654 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc32400cae0 con 0x7fc32c1611b0 2024-05-11T08:06:31.520 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.517+0000 7fc332a3e640 1 -- 172.21.15.167:0/3806047654 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc32c15ed90 con 0x7fc32c1611b0 2024-05-11T08:06:31.520 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.517+0000 7fc3297fa640 1 -- 172.21.15.167:0/3806047654 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc324014440 con 0x7fc32c1611b0 2024-05-11T08:06:31.520 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.517+0000 7fc332a3e640 1 -- 172.21.15.167:0/3806047654 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc32c140bd0 con 0x7fc32c1611b0 2024-05-11T08:06:31.520 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.518+0000 7fc332a3e640 1 -- 172.21.15.167:0/3806047654 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc32c140dd0 con 0x7fc32c1611b0 2024-05-11T08:06:31.521 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.519+0000 7fc3297fa640 1 -- 172.21.15.167:0/3806047654 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fc32401c070 con 0x7fc32c1611b0 2024-05-11T08:06:31.521 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.519+0000 7fc3297fa640 1 --2- 172.21.15.167:0/3806047654 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fc30c0762d0 0x7fc30c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:31.521 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.519+0000 7fc3297fa640 1 -- 172.21.15.167:0/3806047654 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fc324097560 con 0x7fc32c1611b0 2024-05-11T08:06:31.521 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.519+0000 7fc32bfff640 1 --2- 172.21.15.167:0/3806047654 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fc30c0762d0 0x7fc30c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:31.521 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.519+0000 7fc32bfff640 1 --2- 172.21.15.167:0/3806047654 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fc30c0762d0 0x7fc30c078790 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc31c034000 tx=0x7fc31c02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:31.523 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.523+0000 7fc3297fa640 1 -- 172.21.15.167:0/3806047654 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fc32c140dd0 con 0x7fc32c1611b0 2024-05-11T08:06:31.706 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.705+0000 7fc332a3e640 1 -- 172.21.15.167:0/3806047654 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fc32c070120 con 0x7fc32c1611b0 2024-05-11T08:06:31.706 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.706+0000 7fc3297fa640 1 -- 172.21.15.167:0/3806047654 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7fc32c15ed90 con 0x7fc32c1611b0 2024-05-11T08:06:31.706 INFO:teuthology.orchestra.run.smithi167.stdout:pg_num: 64 2024-05-11T08:06:31.708 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.708+0000 7fc332a3e640 1 -- 172.21.15.167:0/3806047654 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fc30c0762d0 msgr2=0x7fc30c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:31.708 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.708+0000 7fc332a3e640 1 --2- 172.21.15.167:0/3806047654 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fc30c0762d0 0x7fc30c078790 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc31c034000 tx=0x7fc31c02e000 comp rx=0 tx=0).stop 2024-05-11T08:06:31.708 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.708+0000 7fc332a3e640 1 -- 172.21.15.167:0/3806047654 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc32c1611b0 msgr2=0x7fc32c15c9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:31.708 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.708+0000 7fc332a3e640 1 --2- 172.21.15.167:0/3806047654 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc32c1611b0 0x7fc32c15c9f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fc32400b970 tx=0x7fc32400be30 comp rx=0 tx=0).stop 2024-05-11T08:06:31.708 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.708+0000 7fc332a3e640 1 -- 172.21.15.167:0/3806047654 shutdown_connections 2024-05-11T08:06:31.708 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.708+0000 7fc332a3e640 1 --2- 172.21.15.167:0/3806047654 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fc30c0762d0 0x7fc30c078790 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:31.709 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.708+0000 7fc332a3e640 1 --2- 172.21.15.167:0/3806047654 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc32c1611b0 0x7fc32c15c9f0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:31.709 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.708+0000 7fc332a3e640 1 --2- 172.21.15.167:0/3806047654 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc32c15ac80 0x7fc32c15c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:31.709 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.708+0000 7fc332a3e640 1 --2- 172.21.15.167:0/3806047654 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc32c0b3d50 0x7fc32c15e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:31.709 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.708+0000 7fc332a3e640 1 -- 172.21.15.167:0/3806047654 >> 172.21.15.167:0/3806047654 conn(0x7fc32c06c450 msgr2=0x7fc32c06e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:31.709 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.708+0000 7fc332a3e640 1 -- 172.21.15.167:0/3806047654 shutdown_connections 2024-05-11T08:06:31.709 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.708+0000 7fc332a3e640 1 -- 172.21.15.167:0/3806047654 wait complete. 2024-05-11T08:06:31.719 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-11T08:06:31.823 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.823+0000 7f481cb9e640 1 Processor -- start 2024-05-11T08:06:31.823 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.823+0000 7f481cb9e640 1 -- start start 2024-05-11T08:06:31.824 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.823+0000 7f481cb9e640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f48181466c0 0x7f48180700e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:31.824 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.823+0000 7f481cb9e640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f4818072ff0 0x7f481806cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:31.824 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.823+0000 7f481cb9e640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48180b6890 con 0x7f481806d220 2024-05-11T08:06:31.824 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.823+0000 7f481cb9e640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f481806d670 con 0x7f4818072ff0 2024-05-11T08:06:31.824 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.823+0000 7f481cb9e640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f4818146aa0 con 0x7f48181466c0 2024-05-11T08:06:31.824 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.824+0000 7f480ffff640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f4818072ff0 0x7f481806cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:31.825 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.824+0000 7f480ffff640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f4818072ff0 0x7f481806cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:58638/0 (socket says 172.21.15.167:58638) 2024-05-11T08:06:31.825 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.824+0000 7f480ffff640 1 -- 172.21.15.167:0/862077465 learned_addr learned my addr 172.21.15.167:0/862077465 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:31.825 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.824+0000 7f48177fe640 1 --2- 172.21.15.167:0/862077465 >> v2:172.21.15.204:3300/0 conn(0x7f48181466c0 0x7f48180700e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:31.825 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.824+0000 7f480ffff640 1 -- 172.21.15.167:0/862077465 >> v2:172.21.15.204:3300/0 conn(0x7f48181466c0 msgr2=0x7f48180700e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:31.825 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.824+0000 7f480ffff640 1 --2- 172.21.15.167:0/862077465 >> v2:172.21.15.204:3300/0 conn(0x7f48181466c0 0x7f48180700e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:31.825 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.824+0000 7f480ffff640 1 -- 172.21.15.167:0/862077465 >> v1:172.21.15.204:6789/0 conn(0x7f481806d220 legacy=0x7f4818160a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:31.825 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.824+0000 7f480ffff640 1 -- 172.21.15.167:0/862077465 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48180737d0 con 0x7f4818072ff0 2024-05-11T08:06:31.825 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.824+0000 7f48177fe640 1 --2- 172.21.15.167:0/862077465 >> v2:172.21.15.204:3300/0 conn(0x7f48181466c0 0x7f48180700e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T08:06:31.827 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.825+0000 7f480ffff640 1 --2- 172.21.15.167:0/862077465 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4818072ff0 0x7f481806cce0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f48180d2af0 tx=0x7f480002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99e370ccde79eba7 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:31.827 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.825+0000 7f4816ffd640 1 -- 172.21.15.167:0/862077465 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4800037070 con 0x7f4818072ff0 2024-05-11T08:06:31.827 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.825+0000 7f4816ffd640 1 -- 172.21.15.167:0/862077465 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4800032440 con 0x7f4818072ff0 2024-05-11T08:06:31.827 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.825+0000 7f4816ffd640 1 -- 172.21.15.167:0/862077465 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4800032810 con 0x7f4818072ff0 2024-05-11T08:06:31.827 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.825+0000 7f481cb9e640 1 -- 172.21.15.167:0/862077465 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4818072ff0 msgr2=0x7f481806cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:31.827 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.825+0000 7f481cb9e640 1 --2- 172.21.15.167:0/862077465 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4818072ff0 0x7f481806cce0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f48180d2af0 tx=0x7f480002a270 comp rx=0 tx=0).stop 2024-05-11T08:06:31.827 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.825+0000 7f481cb9e640 1 -- 172.21.15.167:0/862077465 shutdown_connections 2024-05-11T08:06:31.827 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.825+0000 7f481cb9e640 1 --2- 172.21.15.167:0/862077465 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4818072ff0 0x7f481806cce0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:31.827 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.825+0000 7f481cb9e640 1 --2- 172.21.15.167:0/862077465 >> v2:172.21.15.204:3300/0 conn(0x7f48181466c0 0x7f48180700e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:31.827 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.825+0000 7f481cb9e640 1 -- 172.21.15.167:0/862077465 >> 172.21.15.167:0/862077465 conn(0x7f48180d1c50 msgr2=0x7f48180d2030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:31.827 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.825+0000 7f481cb9e640 1 -- 172.21.15.167:0/862077465 shutdown_connections 2024-05-11T08:06:31.827 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.825+0000 7f481cb9e640 1 -- 172.21.15.167:0/862077465 wait complete. 2024-05-11T08:06:31.827 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.827+0000 7f481cb9e640 1 Processor -- start 2024-05-11T08:06:31.828 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.827+0000 7f481cb9e640 1 -- start start 2024-05-11T08:06:31.830 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.827+0000 7f481cb9e640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f481806d220 0x7f481815c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:31.830 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.827+0000 7f481cb9e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4818072ff0 0x7f481813ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:31.830 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.827+0000 7f481cb9e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f48181466c0 0x7f48180ce6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:31.830 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.827+0000 7f481cb9e640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4818074070 con 0x7f48181466c0 2024-05-11T08:06:31.830 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.827+0000 7f481cb9e640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4818073d90 con 0x7f4818072ff0 2024-05-11T08:06:31.830 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.827+0000 7f481cb9e640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f4818074350 con 0x7f481806d220 2024-05-11T08:06:31.830 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.828+0000 7f4817fff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f48181466c0 0x7f48180ce6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:31.830 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.828+0000 7f4817fff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f48181466c0 0x7f48180ce6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:58648/0 (socket says 172.21.15.167:58648) 2024-05-11T08:06:31.830 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.828+0000 7f4817fff640 1 -- 172.21.15.167:0/2453176362 learned_addr learned my addr 172.21.15.167:0/2453176362 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:31.830 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.828+0000 7f48177fe640 1 --2- 172.21.15.167:0/2453176362 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f481806d220 0x7f481815c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:31.830 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.828+0000 7f480ffff640 1 --2- 172.21.15.167:0/2453176362 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4818072ff0 0x7f481813ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:31.830 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.828+0000 7f4817fff640 1 -- 172.21.15.167:0/2453176362 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f481806d220 msgr2=0x7f481815c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:31.830 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.828+0000 7f4817fff640 1 --2- 172.21.15.167:0/2453176362 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f481806d220 0x7f481815c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:31.831 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.828+0000 7f4817fff640 1 -- 172.21.15.167:0/2453176362 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4818072ff0 msgr2=0x7f481813ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:31.831 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.828+0000 7f4817fff640 1 --2- 172.21.15.167:0/2453176362 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4818072ff0 0x7f481813ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:31.831 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.828+0000 7f4817fff640 1 -- 172.21.15.167:0/2453176362 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f481813c3f0 con 0x7f48181466c0 2024-05-11T08:06:31.831 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.828+0000 7f4817fff640 1 --2- 172.21.15.167:0/2453176362 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f48181466c0 0x7f48180ce6d0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f481000c6f0 tx=0x7f481000cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:31.831 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.828+0000 7f4814ff9640 1 -- 172.21.15.167:0/2453176362 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4810003970 con 0x7f48181466c0 2024-05-11T08:06:31.831 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.828+0000 7f4814ff9640 1 -- 172.21.15.167:0/2453176362 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4810003b10 con 0x7f48181466c0 2024-05-11T08:06:31.831 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.828+0000 7f481cb9e640 1 -- 172.21.15.167:0/2453176362 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f481815e830 con 0x7f48181466c0 2024-05-11T08:06:31.831 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.828+0000 7f4814ff9640 1 -- 172.21.15.167:0/2453176362 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f481000a850 con 0x7f48181466c0 2024-05-11T08:06:31.831 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.828+0000 7f481cb9e640 1 -- 172.21.15.167:0/2453176362 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f481815edf0 con 0x7f48181466c0 2024-05-11T08:06:31.831 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.829+0000 7f481cb9e640 1 -- 172.21.15.167:0/2453176362 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4818069bc0 con 0x7f48181466c0 2024-05-11T08:06:31.831 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.830+0000 7f4814ff9640 1 -- 172.21.15.167:0/2453176362 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f481000a9f0 con 0x7f48181466c0 2024-05-11T08:06:31.832 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.830+0000 7f4814ff9640 1 --2- 172.21.15.167:0/2453176362 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f47f80762d0 0x7f47f8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:31.832 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.831+0000 7f48177fe640 1 --2- 172.21.15.167:0/2453176362 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f47f80762d0 0x7f47f8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:31.832 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.831+0000 7f4814ff9640 1 -- 172.21.15.167:0/2453176362 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f4810097650 con 0x7f48181466c0 2024-05-11T08:06:31.832 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.831+0000 7f48177fe640 1 --2- 172.21.15.167:0/2453176362 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f47f80762d0 0x7f47f8078790 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4808006e80 tx=0x7f4808008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:31.835 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:31.835+0000 7f4814ff9640 1 -- 172.21.15.167:0/2453176362 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f4810060620 con 0x7f48181466c0 2024-05-11T08:06:32.019 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.019+0000 7f481cb9e640 1 -- 172.21.15.167:0/2453176362 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f4818145cc0 con 0x7f48181466c0 2024-05-11T08:06:32.019 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.019+0000 7f4814ff9640 1 -- 172.21.15.167:0/2453176362 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7f4810064920 con 0x7f48181466c0 2024-05-11T08:06:32.020 INFO:teuthology.orchestra.run.smithi167.stdout:pg_num: 32 2024-05-11T08:06:32.021 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.021+0000 7f481cb9e640 1 -- 172.21.15.167:0/2453176362 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f47f80762d0 msgr2=0x7f47f8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:32.022 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.021+0000 7f481cb9e640 1 --2- 172.21.15.167:0/2453176362 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f47f80762d0 0x7f47f8078790 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4808006e80 tx=0x7f4808008040 comp rx=0 tx=0).stop 2024-05-11T08:06:32.022 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.021+0000 7f481cb9e640 1 -- 172.21.15.167:0/2453176362 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f48181466c0 msgr2=0x7f48180ce6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:32.022 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.021+0000 7f481cb9e640 1 --2- 172.21.15.167:0/2453176362 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f48181466c0 0x7f48180ce6d0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f481000c6f0 tx=0x7f481000cbb0 comp rx=0 tx=0).stop 2024-05-11T08:06:32.022 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.021+0000 7f481cb9e640 1 -- 172.21.15.167:0/2453176362 shutdown_connections 2024-05-11T08:06:32.022 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.021+0000 7f481cb9e640 1 --2- 172.21.15.167:0/2453176362 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f47f80762d0 0x7f47f8078790 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:32.022 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.021+0000 7f481cb9e640 1 --2- 172.21.15.167:0/2453176362 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f48181466c0 0x7f48180ce6d0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:32.022 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.021+0000 7f481cb9e640 1 --2- 172.21.15.167:0/2453176362 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4818072ff0 0x7f481813ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:32.022 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.021+0000 7f481cb9e640 1 --2- 172.21.15.167:0/2453176362 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f481806d220 0x7f481815c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:32.022 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.021+0000 7f481cb9e640 1 -- 172.21.15.167:0/2453176362 >> 172.21.15.167:0/2453176362 conn(0x7f48180d1c50 msgr2=0x7f48180b2180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:32.022 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.021+0000 7f481cb9e640 1 -- 172.21.15.167:0/2453176362 shutdown_connections 2024-05-11T08:06:32.022 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.021+0000 7f481cb9e640 1 -- 172.21.15.167:0/2453176362 wait complete. 2024-05-11T08:06:32.033 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_backtrace.TestBacktrace.test_backtrace' 2024-05-11T08:06:32.134 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.134+0000 7f41b9ed1640 1 Processor -- start 2024-05-11T08:06:32.134 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.134+0000 7f41b9ed1640 1 -- start start 2024-05-11T08:06:32.135 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.135+0000 7f41b9ed1640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f41b40cf390 0x7f41b40d1470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:32.135 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.135+0000 7f41b9ed1640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f41b4070650 0x7f41b4165a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:32.135 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.135+0000 7f41b9ed1640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41b4145ca0 con 0x7f41b40cf770 2024-05-11T08:06:32.135 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.135+0000 7f41b9ed1640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f41b40b5960 con 0x7f41b4070650 2024-05-11T08:06:32.135 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.135+0000 7f41b9ed1640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f41b40b5c40 con 0x7f41b40cf390 2024-05-11T08:06:32.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.135+0000 7f41b37fe640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f41b40cf390 0x7f41b40d1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:32.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.135+0000 7f41b3fff640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f41b4070650 0x7f41b4165a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:32.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.135+0000 7f41b2ffd640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f41b40cf770 0x7f41b40cfbd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.167:42400/0 (socket says 172.21.15.167:42400) 2024-05-11T08:06:32.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.135+0000 7f41b2ffd640 1 -- 172.21.15.167:0/3373417826 learned_addr learned my addr 172.21.15.167:0/3373417826 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T08:06:32.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.135+0000 7f41b3fff640 1 -- 172.21.15.167:0/3373417826 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f41b40cf390 msgr2=0x7f41b40d1470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:32.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.135+0000 7f41b3fff640 1 --2- 172.21.15.167:0/3373417826 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f41b40cf390 0x7f41b40d1470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:32.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.135+0000 7f41b3fff640 1 -- 172.21.15.167:0/3373417826 >> v1:172.21.15.204:6789/0 conn(0x7f41b40cf770 legacy=0x7f41b40cfbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:32.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.135+0000 7f41b3fff640 1 -- 172.21.15.167:0/3373417826 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41b40b5f20 con 0x7f41b4070650 2024-05-11T08:06:32.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.136+0000 7f41b3fff640 1 --2- 172.21.15.167:0/3373417826 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f41b4070650 0x7f41b4165a80 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f41ac009d90 tx=0x7f41ac02c7e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc8661c9c2e6b94 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:32.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.136+0000 7f41b27fc640 1 -- 172.21.15.167:0/3373417826 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41ac007740 con 0x7f41b4070650 2024-05-11T08:06:32.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.136+0000 7f41b27fc640 1 -- 172.21.15.167:0/3373417826 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41ac0078e0 con 0x7f41b4070650 2024-05-11T08:06:32.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.136+0000 7f41b27fc640 1 -- 172.21.15.167:0/3373417826 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41ac007cb0 con 0x7f41b4070650 2024-05-11T08:06:32.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.137+0000 7f41b9ed1640 1 -- 172.21.15.167:0/3373417826 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f41b4070650 msgr2=0x7f41b4165a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:32.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.137+0000 7f41b9ed1640 1 --2- 172.21.15.167:0/3373417826 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f41b4070650 0x7f41b4165a80 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f41ac009d90 tx=0x7f41ac02c7e0 comp rx=0 tx=0).stop 2024-05-11T08:06:32.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.137+0000 7f41b9ed1640 1 -- 172.21.15.167:0/3373417826 shutdown_connections 2024-05-11T08:06:32.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.137+0000 7f41b9ed1640 1 --2- 172.21.15.167:0/3373417826 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f41b4070650 0x7f41b4165a80 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:32.138 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.137+0000 7f41b9ed1640 1 --2- 172.21.15.167:0/3373417826 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f41b40cf390 0x7f41b40d1470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:32.138 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.137+0000 7f41b9ed1640 1 -- 172.21.15.167:0/3373417826 >> 172.21.15.167:0/3373417826 conn(0x7f41b40d1e40 msgr2=0x7f41b40d2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:32.138 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.137+0000 7f41b9ed1640 1 -- 172.21.15.167:0/3373417826 shutdown_connections 2024-05-11T08:06:32.138 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.137+0000 7f41b9ed1640 1 -- 172.21.15.167:0/3373417826 wait complete. 2024-05-11T08:06:32.138 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.138+0000 7f41b9ed1640 1 Processor -- start 2024-05-11T08:06:32.138 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.138+0000 7f41b9ed1640 1 -- start start 2024-05-11T08:06:32.140 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.139+0000 7f41b9ed1640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f41b4070650 0x7f41b413a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:32.140 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.139+0000 7f41b9ed1640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f41b40cf390 0x7f41b40b2c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:32.141 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.139+0000 7f41b9ed1640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f41b40cf770 0x7f41b40b31c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:32.141 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.139+0000 7f41b9ed1640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f41b40d19b0 con 0x7f41b40cf770 2024-05-11T08:06:32.141 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.139+0000 7f41b9ed1640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f41b40b64e0 con 0x7f41b40cf390 2024-05-11T08:06:32.141 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.139+0000 7f41b9ed1640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f41b40d1b30 con 0x7f41b4070650 2024-05-11T08:06:32.141 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.139+0000 7f41b3fff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f41b40cf770 0x7f41b40b31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:32.141 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.139+0000 7f41b3fff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f41b40cf770 0x7f41b40b31c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:58654/0 (socket says 172.21.15.167:58654) 2024-05-11T08:06:32.141 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.139+0000 7f41b3fff640 1 -- 172.21.15.167:0/1722255895 learned_addr learned my addr 172.21.15.167:0/1722255895 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:32.141 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.139+0000 7f41b37fe640 1 --2- 172.21.15.167:0/1722255895 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f41b4070650 0x7f41b413a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:32.141 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.139+0000 7f41b2ffd640 1 --2- 172.21.15.167:0/1722255895 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f41b40cf390 0x7f41b40b2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:32.141 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.139+0000 7f41b3fff640 1 -- 172.21.15.167:0/1722255895 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f41b4070650 msgr2=0x7f41b413a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:32.141 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.139+0000 7f41b3fff640 1 --2- 172.21.15.167:0/1722255895 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f41b4070650 0x7f41b413a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:32.141 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.139+0000 7f41b3fff640 1 -- 172.21.15.167:0/1722255895 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f41b40cf390 msgr2=0x7f41b40b2c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:32.141 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.139+0000 7f41b3fff640 1 --2- 172.21.15.167:0/1722255895 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f41b40cf390 0x7f41b40b2c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:32.141 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.139+0000 7f41b3fff640 1 -- 172.21.15.167:0/1722255895 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41b4150dd0 con 0x7f41b40cf770 2024-05-11T08:06:32.141 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.140+0000 7f41b3fff640 1 --2- 172.21.15.167:0/1722255895 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f41b40cf770 0x7f41b40b31c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f41ac009eb0 tx=0x7f41ac004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:32.142 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.140+0000 7f41abfff640 1 -- 172.21.15.167:0/1722255895 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41ac042070 con 0x7f41b40cf770 2024-05-11T08:06:32.142 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.140+0000 7f41abfff640 1 -- 172.21.15.167:0/1722255895 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41ac033bc0 con 0x7f41b40cf770 2024-05-11T08:06:32.142 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.140+0000 7f41b9ed1640 1 -- 172.21.15.167:0/1722255895 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41b41510b0 con 0x7f41b40cf770 2024-05-11T08:06:32.142 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.140+0000 7f41abfff640 1 -- 172.21.15.167:0/1722255895 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41ac009450 con 0x7f41b40cf770 2024-05-11T08:06:32.142 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.140+0000 7f41b9ed1640 1 -- 172.21.15.167:0/1722255895 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41b4151670 con 0x7f41b40cf770 2024-05-11T08:06:32.142 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.141+0000 7f41b9ed1640 1 -- 172.21.15.167:0/1722255895 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41b4069bc0 con 0x7f41b40cf770 2024-05-11T08:06:32.143 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.141+0000 7f41abfff640 1 -- 172.21.15.167:0/1722255895 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f41ac03b040 con 0x7f41b40cf770 2024-05-11T08:06:32.143 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.142+0000 7f41abfff640 1 --2- 172.21.15.167:0/1722255895 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f41940762d0 0x7f4194078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:32.143 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.142+0000 7f41abfff640 1 -- 172.21.15.167:0/1722255895 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f41ac0bb040 con 0x7f41b40cf770 2024-05-11T08:06:32.143 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.142+0000 7f41b37fe640 1 --2- 172.21.15.167:0/1722255895 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f41940762d0 0x7f4194078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:32.143 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.142+0000 7f41b37fe640 1 --2- 172.21.15.167:0/1722255895 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f41940762d0 0x7f4194078790 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f41a4009b40 tx=0x7f41a4009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:32.146 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.146+0000 7f41abfff640 1 -- 172.21.15.167:0/1722255895 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f41ac084010 con 0x7f41b40cf770 2024-05-11T08:06:32.399 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.399+0000 7f41b9ed1640 1 -- 172.21.15.167:0/1722255895 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_backtrace.TestBacktrace.test_backtrace"]} v 0) -- 0x7f41b4075bb0 con 0x7f41b40cf770 2024-05-11T08:06:32.478 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.478+0000 7f41abfff640 1 -- 172.21.15.167:0/1722255895 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_backtrace.TestBacktrace.test_backtrace"]}]=0 v54) ==== 138+0+0 (secure 0 0 0) 0x7f41ac088310 con 0x7f41b40cf770 2024-05-11T08:06:32.480 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.480+0000 7f41b9ed1640 1 -- 172.21.15.167:0/1722255895 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f41940762d0 msgr2=0x7f4194078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:32.481 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.480+0000 7f41b9ed1640 1 --2- 172.21.15.167:0/1722255895 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f41940762d0 0x7f4194078790 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f41a4009b40 tx=0x7f41a4009190 comp rx=0 tx=0).stop 2024-05-11T08:06:32.481 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.480+0000 7f41b9ed1640 1 -- 172.21.15.167:0/1722255895 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f41b40cf770 msgr2=0x7f41b40b31c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:32.481 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.480+0000 7f41b9ed1640 1 --2- 172.21.15.167:0/1722255895 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f41b40cf770 0x7f41b40b31c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f41ac009eb0 tx=0x7f41ac004a40 comp rx=0 tx=0).stop 2024-05-11T08:06:32.481 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.480+0000 7f41b9ed1640 1 -- 172.21.15.167:0/1722255895 shutdown_connections 2024-05-11T08:06:32.481 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.480+0000 7f41b9ed1640 1 --2- 172.21.15.167:0/1722255895 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f41940762d0 0x7f4194078790 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:32.481 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.480+0000 7f41b9ed1640 1 --2- 172.21.15.167:0/1722255895 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f41b40cf770 0x7f41b40b31c0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:32.481 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.480+0000 7f41b9ed1640 1 --2- 172.21.15.167:0/1722255895 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f41b40cf390 0x7f41b40b2c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:32.481 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.480+0000 7f41b9ed1640 1 --2- 172.21.15.167:0/1722255895 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f41b4070650 0x7f41b413a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:32.481 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.480+0000 7f41b9ed1640 1 -- 172.21.15.167:0/1722255895 >> 172.21.15.167:0/1722255895 conn(0x7f41b40d1e40 msgr2=0x7f41b40732c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:32.481 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.480+0000 7f41b9ed1640 1 -- 172.21.15.167:0/1722255895 shutdown_connections 2024-05-11T08:06:32.481 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.480+0000 7f41b9ed1640 1 -- 172.21.15.167:0/1722255895 wait complete. 2024-05-11T08:06:32.492 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-11T08:06:32.595 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.595+0000 7f1dcbeda640 1 Processor -- start 2024-05-11T08:06:32.595 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.595+0000 7f1dcbeda640 1 -- start start 2024-05-11T08:06:32.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.596+0000 7f1dcbeda640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f1dc406d200 0x7f1dc406d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:32.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.596+0000 7f1dcbeda640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f1dc40d2db0 0x7f1dc40d3240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:32.597 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.596+0000 7f1dcbeda640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1dc4159d50 con 0x7f1dc4159970 2024-05-11T08:06:32.597 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.596+0000 7f1dcbeda640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f1dc40d04b0 con 0x7f1dc40d2db0 2024-05-11T08:06:32.597 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.596+0000 7f1dcbeda640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f1dc40d0790 con 0x7f1dc406d200 2024-05-11T08:06:32.597 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.596+0000 7f1dca477640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f1dc40d2db0 0x7f1dc40d3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:32.597 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.596+0000 7f1dca477640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f1dc40d2db0 0x7f1dc40d3240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:58664/0 (socket says 172.21.15.167:58664) 2024-05-11T08:06:32.597 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.596+0000 7f1dca477640 1 -- 172.21.15.167:0/1840940025 learned_addr learned my addr 172.21.15.167:0/1840940025 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:32.597 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.597+0000 7f1dc9475640 1 --2- 172.21.15.167:0/1840940025 >> v2:172.21.15.204:3300/0 conn(0x7f1dc406d200 0x7f1dc406d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:32.597 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.597+0000 7f1dca477640 1 -- 172.21.15.167:0/1840940025 >> v2:172.21.15.204:3300/0 conn(0x7f1dc406d200 msgr2=0x7f1dc406d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:32.597 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.597+0000 7f1dca477640 1 --2- 172.21.15.167:0/1840940025 >> v2:172.21.15.204:3300/0 conn(0x7f1dc406d200 0x7f1dc406d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:32.597 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.597+0000 7f1dca477640 1 -- 172.21.15.167:0/1840940025 >> v1:172.21.15.204:6789/0 conn(0x7f1dc4159970 legacy=0x7f1dc40d1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:32.597 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.597+0000 7f1dca477640 1 -- 172.21.15.167:0/1840940025 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dc40d0a70 con 0x7f1dc40d2db0 2024-05-11T08:06:32.598 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.597+0000 7f1dca477640 1 --2- 172.21.15.167:0/1840940025 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1dc40d2db0 0x7f1dc40d3240 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f1dc40cfe20 tx=0x7f1dbc02a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=366207dc625cd652 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:32.599 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.597+0000 7f1dc8c74640 1 -- 172.21.15.167:0/1840940025 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dbc036070 con 0x7f1dc40d2db0 2024-05-11T08:06:32.599 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.597+0000 7f1dc8c74640 1 -- 172.21.15.167:0/1840940025 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1dbc031600 con 0x7f1dc40d2db0 2024-05-11T08:06:32.599 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.597+0000 7f1dc8c74640 1 -- 172.21.15.167:0/1840940025 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dbc0319d0 con 0x7f1dc40d2db0 2024-05-11T08:06:32.599 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.598+0000 7f1dcbeda640 1 -- 172.21.15.167:0/1840940025 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1dc40d2db0 msgr2=0x7f1dc40d3240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:32.599 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.598+0000 7f1dcbeda640 1 --2- 172.21.15.167:0/1840940025 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1dc40d2db0 0x7f1dc40d3240 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f1dc40cfe20 tx=0x7f1dbc02a730 comp rx=0 tx=0).stop 2024-05-11T08:06:32.599 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.598+0000 7f1dcbeda640 1 -- 172.21.15.167:0/1840940025 shutdown_connections 2024-05-11T08:06:32.599 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.598+0000 7f1dcbeda640 1 --2- 172.21.15.167:0/1840940025 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1dc40d2db0 0x7f1dc40d3240 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:32.599 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.598+0000 7f1dcbeda640 1 --2- 172.21.15.167:0/1840940025 >> v2:172.21.15.204:3300/0 conn(0x7f1dc406d200 0x7f1dc406d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:32.599 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.598+0000 7f1dcbeda640 1 -- 172.21.15.167:0/1840940025 >> 172.21.15.167:0/1840940025 conn(0x7f1dc40d51e0 msgr2=0x7f1dc40d55f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:32.599 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.598+0000 7f1dcbeda640 1 -- 172.21.15.167:0/1840940025 shutdown_connections 2024-05-11T08:06:32.599 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.598+0000 7f1dcbeda640 1 -- 172.21.15.167:0/1840940025 wait complete. 2024-05-11T08:06:32.601 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.599+0000 7f1dcbeda640 1 Processor -- start 2024-05-11T08:06:32.601 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.599+0000 7f1dcbeda640 1 -- start start 2024-05-11T08:06:32.601 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.600+0000 7f1dcbeda640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1dc406d200 0x7f1dc41f4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:32.601 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.600+0000 7f1dcbeda640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1dc40d2db0 0x7f1dc40d3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:32.601 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.600+0000 7f1dcbeda640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1dc4159970 0x7f1dc40d4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:32.601 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.600+0000 7f1dcbeda640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1dc40d1980 con 0x7f1dc4159970 2024-05-11T08:06:32.601 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.600+0000 7f1dcbeda640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1dc4163d00 con 0x7f1dc40d2db0 2024-05-11T08:06:32.602 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.600+0000 7f1dcbeda640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f1dc4162f10 con 0x7f1dc406d200 2024-05-11T08:06:32.602 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.600+0000 7f1dca477640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1dc4159970 0x7f1dc40d4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:32.602 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.600+0000 7f1dca477640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1dc4159970 0x7f1dc40d4010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:58676/0 (socket says 172.21.15.167:58676) 2024-05-11T08:06:32.602 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.600+0000 7f1dca477640 1 -- 172.21.15.167:0/1074386347 learned_addr learned my addr 172.21.15.167:0/1074386347 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:32.602 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.600+0000 7f1dc9c76640 1 --2- 172.21.15.167:0/1074386347 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1dc406d200 0x7f1dc41f4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:32.602 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.600+0000 7f1dc9475640 1 --2- 172.21.15.167:0/1074386347 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1dc40d2db0 0x7f1dc40d3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:32.602 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.600+0000 7f1dca477640 1 -- 172.21.15.167:0/1074386347 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1dc406d200 msgr2=0x7f1dc41f4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:32.602 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.600+0000 7f1dca477640 1 --2- 172.21.15.167:0/1074386347 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1dc406d200 0x7f1dc41f4d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:32.602 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.600+0000 7f1dca477640 1 -- 172.21.15.167:0/1074386347 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1dc40d2db0 msgr2=0x7f1dc40d3ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:32.602 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.600+0000 7f1dca477640 1 --2- 172.21.15.167:0/1074386347 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1dc40d2db0 0x7f1dc40d3ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:32.602 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.600+0000 7f1dca477640 1 -- 172.21.15.167:0/1074386347 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dc40c8e50 con 0x7f1dc4159970 2024-05-11T08:06:32.602 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.600+0000 7f1dc9475640 1 --2- 172.21.15.167:0/1074386347 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1dc40d2db0 0x7f1dc40d3ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T08:06:32.602 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.601+0000 7f1dca477640 1 --2- 172.21.15.167:0/1074386347 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1dc4159970 0x7f1dc40d4010 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f1dbc009f40 tx=0x7f1dbc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:32.602 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.601+0000 7f1dba7fc640 1 -- 172.21.15.167:0/1074386347 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dbc03f020 con 0x7f1dc4159970 2024-05-11T08:06:32.603 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.601+0000 7f1dba7fc640 1 -- 172.21.15.167:0/1074386347 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1dbc037040 con 0x7f1dc4159970 2024-05-11T08:06:32.603 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.601+0000 7f1dcbeda640 1 -- 172.21.15.167:0/1074386347 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1dc40c90d0 con 0x7f1dc4159970 2024-05-11T08:06:32.603 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.601+0000 7f1dba7fc640 1 -- 172.21.15.167:0/1074386347 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dbc036070 con 0x7f1dc4159970 2024-05-11T08:06:32.603 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.601+0000 7f1dcbeda640 1 -- 172.21.15.167:0/1074386347 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1dc40cdc40 con 0x7f1dc4159970 2024-05-11T08:06:32.604 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.602+0000 7f1dcbeda640 1 -- 172.21.15.167:0/1074386347 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1dc4069520 con 0x7f1dc4159970 2024-05-11T08:06:32.604 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.602+0000 7f1dba7fc640 1 -- 172.21.15.167:0/1074386347 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f1dbc042020 con 0x7f1dc4159970 2024-05-11T08:06:32.604 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.603+0000 7f1dba7fc640 1 --2- 172.21.15.167:0/1074386347 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f1da80762d0 0x7f1da8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:32.604 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.603+0000 7f1dba7fc640 1 -- 172.21.15.167:0/1074386347 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f1dbc0b7df0 con 0x7f1dc4159970 2024-05-11T08:06:32.604 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.603+0000 7f1dc9c76640 1 --2- 172.21.15.167:0/1074386347 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f1da80762d0 0x7f1da8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:32.604 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.603+0000 7f1dc9c76640 1 --2- 172.21.15.167:0/1074386347 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f1da80762d0 0x7f1da8078790 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f1dc00015e0 tx=0x7f1dc0009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:32.607 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.607+0000 7f1dba7fc640 1 -- 172.21.15.167:0/1074386347 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f1dbc049030 con 0x7f1dc4159970 2024-05-11T08:06:32.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.793+0000 7f1dcbeda640 1 -- 172.21.15.167:0/1074386347 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f1dc40d00f0 con 0x7f1dc4159970 2024-05-11T08:06:32.796 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.796+0000 7f1dba7fc640 1 -- 172.21.15.167:0/1074386347 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) ==== 122+0+0 (secure 0 0 0) 0x7f1dc40c8e50 con 0x7f1dc4159970 2024-05-11T08:06:32.799 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.799+0000 7f1dcbeda640 1 -- 172.21.15.167:0/1074386347 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f1da80762d0 msgr2=0x7f1da8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:32.799 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.799+0000 7f1dcbeda640 1 --2- 172.21.15.167:0/1074386347 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f1da80762d0 0x7f1da8078790 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f1dc00015e0 tx=0x7f1dc0009040 comp rx=0 tx=0).stop 2024-05-11T08:06:32.799 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.799+0000 7f1dcbeda640 1 -- 172.21.15.167:0/1074386347 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1dc4159970 msgr2=0x7f1dc40d4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:32.799 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.799+0000 7f1dcbeda640 1 --2- 172.21.15.167:0/1074386347 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1dc4159970 0x7f1dc40d4010 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f1dbc009f40 tx=0x7f1dbc004a40 comp rx=0 tx=0).stop 2024-05-11T08:06:32.799 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.799+0000 7f1dcbeda640 1 -- 172.21.15.167:0/1074386347 shutdown_connections 2024-05-11T08:06:32.799 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.799+0000 7f1dcbeda640 1 --2- 172.21.15.167:0/1074386347 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f1da80762d0 0x7f1da8078790 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:32.799 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.799+0000 7f1dcbeda640 1 --2- 172.21.15.167:0/1074386347 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1dc4159970 0x7f1dc40d4010 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:32.799 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.799+0000 7f1dcbeda640 1 --2- 172.21.15.167:0/1074386347 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1dc40d2db0 0x7f1dc40d3ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:32.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.799+0000 7f1dcbeda640 1 --2- 172.21.15.167:0/1074386347 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1dc406d200 0x7f1dc41f4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:32.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.799+0000 7f1dcbeda640 1 -- 172.21.15.167:0/1074386347 >> 172.21.15.167:0/1074386347 conn(0x7f1dc40d51e0 msgr2=0x7f1dc40d55c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:32.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.799+0000 7f1dcbeda640 1 -- 172.21.15.167:0/1074386347 shutdown_connections 2024-05-11T08:06:32.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:32.799+0000 7f1dcbeda640 1 -- 172.21.15.167:0/1074386347 wait complete. 2024-05-11T08:06:32.810 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-11T08:06:32.810 DEBUG:teuthology.orchestra.run.smithi167:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-11T08:06:32.847 DEBUG:teuthology.orchestra.run.smithi204:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-11T08:06:32.910 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-11T08:06:32.910 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-05-11T08:06:32.910 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T08:06:32.926 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-05-11T08:06:32.926 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T08:06:32.982 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-05-11T08:06:32.982 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T08:06:33.039 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-11T08:06:33.181 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.181+0000 7fafddc75640 1 Processor -- start 2024-05-11T08:06:33.181 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.181+0000 7fafddc75640 1 -- start start 2024-05-11T08:06:33.184 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.182+0000 7fafddc75640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fafd81488a0 0x7fafd815f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:33.184 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.182+0000 7fafddc75640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fafd815fe50 0x7fafd80b5a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:33.184 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.182+0000 7fafddc75640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fafd80b5f50 0x7fafd80710f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:33.184 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.182+0000 7fafddc75640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7fafd80b5580 con 0x7fafd80b5f50 2024-05-11T08:06:33.184 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.182+0000 7fafddc75640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fafd8163e60 con 0x7fafd81488a0 2024-05-11T08:06:33.184 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.182+0000 7fafddc75640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fafd80b4f70 con 0x7fafd815fe50 2024-05-11T08:06:33.185 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.182+0000 7fafd7fff640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fafd80b5f50 0x7fafd80710f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:33.185 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.182+0000 7fafd7fff640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fafd80b5f50 0x7fafd80710f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:58678/0 (socket says 172.21.15.167:58678) 2024-05-11T08:06:33.185 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.182+0000 7fafd7fff640 1 -- 172.21.15.167:0/1862999544 learned_addr learned my addr 172.21.15.167:0/1862999544 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:33.185 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.182+0000 7fafd77fe640 1 --2- 172.21.15.167:0/1862999544 >> v2:172.21.15.204:3300/0 conn(0x7fafd81488a0 0x7fafd815f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:33.185 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.182+0000 7fafd6ffd640 1 --2- 172.21.15.167:0/1862999544 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fafd815fe50 0x7fafd80b5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:33.185 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.182+0000 7fafd7fff640 1 -- 172.21.15.167:0/1862999544 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fafd815fe50 msgr2=0x7fafd80b5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:33.185 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.182+0000 7fafd7fff640 1 --2- 172.21.15.167:0/1862999544 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fafd815fe50 0x7fafd80b5a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:33.185 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.182+0000 7fafd7fff640 1 -- 172.21.15.167:0/1862999544 >> v2:172.21.15.204:3300/0 conn(0x7fafd81488a0 msgr2=0x7fafd815f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:33.185 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.182+0000 7fafd7fff640 1 --2- 172.21.15.167:0/1862999544 >> v2:172.21.15.204:3300/0 conn(0x7fafd81488a0 0x7fafd815f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:33.185 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.182+0000 7fafd7fff640 1 -- 172.21.15.167:0/1862999544 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafd8163b80 con 0x7fafd80b5f50 2024-05-11T08:06:33.185 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.182+0000 7fafd77fe640 1 --2- 172.21.15.167:0/1862999544 >> v2:172.21.15.204:3300/0 conn(0x7fafd81488a0 0x7fafd815f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T08:06:33.185 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.182+0000 7fafd6ffd640 1 --2- 172.21.15.167:0/1862999544 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fafd815fe50 0x7fafd80b5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T08:06:33.185 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.183+0000 7fafd7fff640 1 --2- 172.21.15.167:0/1862999544 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fafd80b5f50 0x7fafd80710f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fafd8071d00 tx=0x7fafd002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc7b508b15df795a server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:33.185 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.183+0000 7fafd67fc640 1 -- 172.21.15.167:0/1862999544 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafd0037070 con 0x7fafd80b5f50 2024-05-11T08:06:33.186 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.183+0000 7fafd67fc640 1 -- 172.21.15.167:0/1862999544 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafd0032440 con 0x7fafd80b5f50 2024-05-11T08:06:33.186 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.183+0000 7fafd67fc640 1 -- 172.21.15.167:0/1862999544 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafd0032810 con 0x7fafd80b5f50 2024-05-11T08:06:33.186 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.184+0000 7fafddc75640 1 -- 172.21.15.167:0/1862999544 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fafd80b5f50 msgr2=0x7fafd80710f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:33.186 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.184+0000 7fafddc75640 1 --2- 172.21.15.167:0/1862999544 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fafd80b5f50 0x7fafd80710f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fafd8071d00 tx=0x7fafd002a140 comp rx=0 tx=0).stop 2024-05-11T08:06:33.186 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.184+0000 7fafddc75640 1 -- 172.21.15.167:0/1862999544 shutdown_connections 2024-05-11T08:06:33.186 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.184+0000 7fafddc75640 1 --2- 172.21.15.167:0/1862999544 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fafd80b5f50 0x7fafd80710f0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:33.186 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.184+0000 7fafddc75640 1 --2- 172.21.15.167:0/1862999544 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fafd815fe50 0x7fafd80b5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:33.186 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.184+0000 7fafddc75640 1 --2- 172.21.15.167:0/1862999544 >> v2:172.21.15.204:3300/0 conn(0x7fafd81488a0 0x7fafd815f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:33.186 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.184+0000 7fafddc75640 1 -- 172.21.15.167:0/1862999544 >> 172.21.15.167:0/1862999544 conn(0x7fafd80d2550 msgr2=0x7fafd80ce890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:33.186 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.184+0000 7fafddc75640 1 -- 172.21.15.167:0/1862999544 shutdown_connections 2024-05-11T08:06:33.186 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.184+0000 7fafddc75640 1 -- 172.21.15.167:0/1862999544 wait complete. 2024-05-11T08:06:33.189 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.185+0000 7fafddc75640 1 Processor -- start 2024-05-11T08:06:33.189 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.185+0000 7fafddc75640 1 -- start start 2024-05-11T08:06:33.189 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.186+0000 7fafddc75640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fafd80b5f50 0x7fafd815eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:33.189 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.186+0000 7fafddc75640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fafd81488a0 0x7fafd80ac230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:33.189 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.186+0000 7fafddc75640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fafd815fe50 0x7fafd80ad400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:33.189 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.186+0000 7fafddc75640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fafd8148e00 con 0x7fafd815fe50 2024-05-11T08:06:33.189 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.186+0000 7fafddc75640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fafd8148c80 con 0x7fafd81488a0 2024-05-11T08:06:33.189 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.186+0000 7fafddc75640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fafd8071630 con 0x7fafd80b5f50 2024-05-11T08:06:33.189 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.186+0000 7fafd7fff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fafd815fe50 0x7fafd80ad400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:33.189 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.186+0000 7fafd7fff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fafd815fe50 0x7fafd80ad400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:58680/0 (socket says 172.21.15.167:58680) 2024-05-11T08:06:33.190 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.186+0000 7fafd7fff640 1 -- 172.21.15.167:0/2442261783 learned_addr learned my addr 172.21.15.167:0/2442261783 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:33.190 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.186+0000 7fafd77fe640 1 --2- 172.21.15.167:0/2442261783 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fafd80b5f50 0x7fafd815eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:33.190 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.186+0000 7fafd6ffd640 1 --2- 172.21.15.167:0/2442261783 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fafd81488a0 0x7fafd80ac230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:33.190 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.187+0000 7fafd7fff640 1 -- 172.21.15.167:0/2442261783 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fafd80b5f50 msgr2=0x7fafd815eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:33.190 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.187+0000 7fafd7fff640 1 --2- 172.21.15.167:0/2442261783 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fafd80b5f50 0x7fafd815eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:33.190 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.187+0000 7fafd7fff640 1 -- 172.21.15.167:0/2442261783 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fafd81488a0 msgr2=0x7fafd80ac230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:33.190 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.187+0000 7fafd7fff640 1 --2- 172.21.15.167:0/2442261783 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fafd81488a0 0x7fafd80ac230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:33.190 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.187+0000 7fafd7fff640 1 -- 172.21.15.167:0/2442261783 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafd8133620 con 0x7fafd815fe50 2024-05-11T08:06:33.190 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.187+0000 7fafd7fff640 1 --2- 172.21.15.167:0/2442261783 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fafd815fe50 0x7fafd80ad400 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fafd002a180 tx=0x7fafd00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:33.190 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.187+0000 7fafbffff640 1 -- 172.21.15.167:0/2442261783 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafd0037040 con 0x7fafd815fe50 2024-05-11T08:06:33.190 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.187+0000 7fafbffff640 1 -- 172.21.15.167:0/2442261783 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafd0042070 con 0x7fafd815fe50 2024-05-11T08:06:33.190 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.187+0000 7fafddc75640 1 -- 172.21.15.167:0/2442261783 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafd8133900 con 0x7fafd815fe50 2024-05-11T08:06:33.190 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.187+0000 7fafbffff640 1 -- 172.21.15.167:0/2442261783 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafd0030bd0 con 0x7fafd815fe50 2024-05-11T08:06:33.190 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.187+0000 7fafddc75640 1 -- 172.21.15.167:0/2442261783 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafd8133ec0 con 0x7fafd815fe50 2024-05-11T08:06:33.191 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.188+0000 7fafddc75640 1 -- 172.21.15.167:0/2442261783 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafd8069590 con 0x7fafd815fe50 2024-05-11T08:06:33.191 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.188+0000 7fafbffff640 1 -- 172.21.15.167:0/2442261783 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fafd0047460 con 0x7fafd815fe50 2024-05-11T08:06:33.191 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.189+0000 7fafbffff640 1 --2- 172.21.15.167:0/2442261783 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fafc40762d0 0x7fafc4078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:33.191 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.189+0000 7fafd77fe640 1 --2- 172.21.15.167:0/2442261783 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fafc40762d0 0x7fafc4078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:33.191 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.189+0000 7fafbffff640 1 -- 172.21.15.167:0/2442261783 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fafd00b8500 con 0x7fafd815fe50 2024-05-11T08:06:33.191 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.189+0000 7fafd77fe640 1 --2- 172.21.15.167:0/2442261783 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fafc40762d0 0x7fafc4078790 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fafc80075b0 tx=0x7fafc8007540 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:33.193 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.193+0000 7fafbffff640 1 -- 172.21.15.167:0/2442261783 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fafd0081b20 con 0x7fafd815fe50 2024-05-11T08:06:33.389 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.389+0000 7fafddc75640 1 -- 172.21.15.167:0/2442261783 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fafd80d1220 con 0x7fafd815fe50 2024-05-11T08:06:33.389 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.389+0000 7fafbffff640 1 -- 172.21.15.167:0/2442261783 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) ==== 80+0+0 (secure 0 0 0) 0x7fafd0081420 con 0x7fafd815fe50 2024-05-11T08:06:33.389 INFO:teuthology.orchestra.run.smithi167.stderr:listed 0 entries 2024-05-11T08:06:33.391 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.391+0000 7fafddc75640 1 -- 172.21.15.167:0/2442261783 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fafc40762d0 msgr2=0x7fafc4078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:33.392 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.391+0000 7fafddc75640 1 --2- 172.21.15.167:0/2442261783 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fafc40762d0 0x7fafc4078790 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fafc80075b0 tx=0x7fafc8007540 comp rx=0 tx=0).stop 2024-05-11T08:06:33.392 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.391+0000 7fafddc75640 1 -- 172.21.15.167:0/2442261783 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fafd815fe50 msgr2=0x7fafd80ad400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:33.392 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.391+0000 7fafddc75640 1 --2- 172.21.15.167:0/2442261783 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fafd815fe50 0x7fafd80ad400 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fafd002a180 tx=0x7fafd00046c0 comp rx=0 tx=0).stop 2024-05-11T08:06:33.392 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.391+0000 7fafddc75640 1 -- 172.21.15.167:0/2442261783 shutdown_connections 2024-05-11T08:06:33.392 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.391+0000 7fafddc75640 1 --2- 172.21.15.167:0/2442261783 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fafc40762d0 0x7fafc4078790 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:33.392 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.391+0000 7fafddc75640 1 --2- 172.21.15.167:0/2442261783 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fafd815fe50 0x7fafd80ad400 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:33.392 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.391+0000 7fafddc75640 1 --2- 172.21.15.167:0/2442261783 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fafd81488a0 0x7fafd80ac230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:33.392 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.391+0000 7fafddc75640 1 --2- 172.21.15.167:0/2442261783 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fafd80b5f50 0x7fafd815eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:33.392 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.391+0000 7fafddc75640 1 -- 172.21.15.167:0/2442261783 >> 172.21.15.167:0/2442261783 conn(0x7fafd80d2550 msgr2=0x7fafd80cfd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:33.392 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.391+0000 7fafddc75640 1 -- 172.21.15.167:0/2442261783 shutdown_connections 2024-05-11T08:06:33.392 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.391+0000 7fafddc75640 1 -- 172.21.15.167:0/2442261783 wait complete. 2024-05-11T08:06:33.402 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-11T08:06:33.403 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T08:06:33.403 DEBUG:teuthology.orchestra.run.smithi167:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-05-11T08:06:33.429 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi167.front.sepia.ceph.com 2024-05-11T08:06:33.429 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T08:06:33.429 DEBUG:teuthology.orchestra.run.smithi167:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-11T08:06:33.485 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-11T08:06:33.485 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T08:06:33.504 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-11T08:06:33.504 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T08:06:33.564 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-11T08:06:33.564 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T08:06:33.623 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-11T08:06:33.724 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.724+0000 7fe13950a640 1 Processor -- start 2024-05-11T08:06:33.724 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.724+0000 7fe13950a640 1 -- start start 2024-05-11T08:06:33.727 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.725+0000 7fe13950a640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fe1341598d0 0x7fe1340d2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:33.727 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.725+0000 7fe13950a640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe1340d0640 0x7fe1340d0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:33.727 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.725+0000 7fe13950a640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1340d1190 con 0x7fe1340cafb0 2024-05-11T08:06:33.727 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.725+0000 7fe13950a640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7fe13406d500 con 0x7fe1341598d0 2024-05-11T08:06:33.727 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.725+0000 7fe13950a640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fe13406d7e0 con 0x7fe1340d0640 2024-05-11T08:06:33.727 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.725+0000 7fe132ffd640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fe1341598d0 0x7fe1340d2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:33.727 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.725+0000 7fe132ffd640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fe1341598d0 0x7fe1340d2cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:58694/0 (socket says 172.21.15.167:58694) 2024-05-11T08:06:33.727 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.725+0000 7fe132ffd640 1 -- 172.21.15.167:0/874168284 learned_addr learned my addr 172.21.15.167:0/874168284 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:33.728 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.725+0000 7fe12bfff640 1 --2- 172.21.15.167:0/874168284 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe1340d0640 0x7fe1340d0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:33.728 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.725+0000 7fe132ffd640 1 -- 172.21.15.167:0/874168284 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe1340d0640 msgr2=0x7fe1340d0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:33.728 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.725+0000 7fe132ffd640 1 --2- 172.21.15.167:0/874168284 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe1340d0640 0x7fe1340d0aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:33.728 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.725+0000 7fe132ffd640 1 -- 172.21.15.167:0/874168284 >> v1:172.21.15.204:6789/0 conn(0x7fe1340cafb0 legacy=0x7fe13406dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:33.728 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.725+0000 7fe132ffd640 1 -- 172.21.15.167:0/874168284 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe13406dac0 con 0x7fe1341598d0 2024-05-11T08:06:33.728 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.725+0000 7fe12bfff640 1 --2- 172.21.15.167:0/874168284 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe1340d0640 0x7fe1340d0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T08:06:33.728 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.726+0000 7fe132ffd640 1 --2- 172.21.15.167:0/874168284 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe1341598d0 0x7fe1340d2cf0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fe124009870 tx=0x7fe12402a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec0e94b0b25110ca server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:33.728 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.726+0000 7fe1327fc640 1 -- 172.21.15.167:0/874168284 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe124037070 con 0x7fe1341598d0 2024-05-11T08:06:33.728 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.726+0000 7fe1327fc640 1 -- 172.21.15.167:0/874168284 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe124032440 con 0x7fe1341598d0 2024-05-11T08:06:33.728 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.726+0000 7fe1327fc640 1 -- 172.21.15.167:0/874168284 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe124032810 con 0x7fe1341598d0 2024-05-11T08:06:33.728 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.727+0000 7fe13950a640 1 -- 172.21.15.167:0/874168284 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe1341598d0 msgr2=0x7fe1340d2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:33.729 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.727+0000 7fe13950a640 1 --2- 172.21.15.167:0/874168284 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe1341598d0 0x7fe1340d2cf0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fe124009870 tx=0x7fe12402a1c0 comp rx=0 tx=0).stop 2024-05-11T08:06:33.729 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.727+0000 7fe13950a640 1 -- 172.21.15.167:0/874168284 shutdown_connections 2024-05-11T08:06:33.729 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.727+0000 7fe13950a640 1 --2- 172.21.15.167:0/874168284 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe1340d0640 0x7fe1340d0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:33.729 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.727+0000 7fe13950a640 1 --2- 172.21.15.167:0/874168284 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe1341598d0 0x7fe1340d2cf0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:33.729 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.727+0000 7fe13950a640 1 -- 172.21.15.167:0/874168284 >> 172.21.15.167:0/874168284 conn(0x7fe1340d1ba0 msgr2=0x7fe1340d1fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:33.729 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.727+0000 7fe13950a640 1 -- 172.21.15.167:0/874168284 shutdown_connections 2024-05-11T08:06:33.729 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.727+0000 7fe13950a640 1 -- 172.21.15.167:0/874168284 wait complete. 2024-05-11T08:06:33.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.728+0000 7fe13950a640 1 Processor -- start 2024-05-11T08:06:33.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.728+0000 7fe13950a640 1 -- start start 2024-05-11T08:06:33.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.729+0000 7fe13950a640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe1340cafb0 0x7fe1340b7e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:33.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.729+0000 7fe13950a640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe1340d0640 0x7fe134146810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:33.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.729+0000 7fe13950a640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe1341598d0 0x7fe1341434b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:33.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.729+0000 7fe13950a640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe13406d1b0 con 0x7fe1340d0640 2024-05-11T08:06:33.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.729+0000 7fe13950a640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe134159cb0 con 0x7fe1340cafb0 2024-05-11T08:06:33.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.729+0000 7fe13950a640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fe1340cb400 con 0x7fe1341598d0 2024-05-11T08:06:33.733 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.729+0000 7fe12bfff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe1340d0640 0x7fe134146810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:33.733 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.729+0000 7fe12bfff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe1340d0640 0x7fe134146810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:58710/0 (socket says 172.21.15.167:58710) 2024-05-11T08:06:33.733 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.729+0000 7fe12bfff640 1 -- 172.21.15.167:0/3329500488 learned_addr learned my addr 172.21.15.167:0/3329500488 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:33.733 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.729+0000 7fe132ffd640 1 --2- 172.21.15.167:0/3329500488 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe1340cafb0 0x7fe1340b7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:33.733 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.729+0000 7fe12bfff640 1 -- 172.21.15.167:0/3329500488 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe1341598d0 msgr2=0x7fe1341434b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:33.733 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.729+0000 7fe12bfff640 1 --2- 172.21.15.167:0/3329500488 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe1341598d0 0x7fe1341434b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:33.733 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.729+0000 7fe12bfff640 1 -- 172.21.15.167:0/3329500488 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe1340cafb0 msgr2=0x7fe1340b7e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:33.733 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.729+0000 7fe12bfff640 1 --2- 172.21.15.167:0/3329500488 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe1340cafb0 0x7fe1340b7e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:33.733 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.729+0000 7fe12bfff640 1 -- 172.21.15.167:0/3329500488 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1341439f0 con 0x7fe1340d0640 2024-05-11T08:06:33.733 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.729+0000 7fe132ffd640 1 --2- 172.21.15.167:0/3329500488 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe1340cafb0 0x7fe1340b7e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T08:06:33.733 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.729+0000 7fe12bfff640 1 --2- 172.21.15.167:0/3329500488 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe1340d0640 0x7fe134146810 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fe11c00c820 tx=0x7fe11c00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:33.733 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.729+0000 7fe12b7fe640 1 -- 172.21.15.167:0/3329500488 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe11c015070 con 0x7fe1340d0640 2024-05-11T08:06:33.733 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.729+0000 7fe12b7fe640 1 -- 172.21.15.167:0/3329500488 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe11c007a30 con 0x7fe1340d0640 2024-05-11T08:06:33.733 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.730+0000 7fe13950a640 1 -- 172.21.15.167:0/3329500488 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1340b9bd0 con 0x7fe1340d0640 2024-05-11T08:06:33.734 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.730+0000 7fe12b7fe640 1 -- 172.21.15.167:0/3329500488 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe11c007e00 con 0x7fe1340d0640 2024-05-11T08:06:33.734 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.730+0000 7fe13950a640 1 -- 172.21.15.167:0/3329500488 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1340ba190 con 0x7fe1340d0640 2024-05-11T08:06:33.734 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.730+0000 7fe13950a640 1 -- 172.21.15.167:0/3329500488 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe134069590 con 0x7fe1340d0640 2024-05-11T08:06:33.734 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.731+0000 7fe12b7fe640 1 -- 172.21.15.167:0/3329500488 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fe11c018070 con 0x7fe1340d0640 2024-05-11T08:06:33.734 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.732+0000 7fe12b7fe640 1 --2- 172.21.15.167:0/3329500488 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fe1140762d0 0x7fe114078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:33.734 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.732+0000 7fe132ffd640 1 --2- 172.21.15.167:0/3329500488 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fe1140762d0 0x7fe114078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:33.734 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.732+0000 7fe12b7fe640 1 -- 172.21.15.167:0/3329500488 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fe11c0978c0 con 0x7fe1340d0640 2024-05-11T08:06:33.734 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.732+0000 7fe132ffd640 1 --2- 172.21.15.167:0/3329500488 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fe1140762d0 0x7fe114078790 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe124034000 tx=0x7fe12402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:33.736 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.736+0000 7fe12b7fe640 1 -- 172.21.15.167:0/3329500488 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fe11c060f70 con 0x7fe1340d0640 2024-05-11T08:06:33.930 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.930+0000 7fe13950a640 1 -- 172.21.15.167:0/3329500488 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fe1340d0eb0 con 0x7fe1340d0640 2024-05-11T08:06:33.930 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.930+0000 7fe12b7fe640 1 -- 172.21.15.167:0/3329500488 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) ==== 80+0+0 (secure 0 0 0) 0x7fe11c060870 con 0x7fe1340d0640 2024-05-11T08:06:33.930 INFO:teuthology.orchestra.run.smithi167.stderr:listed 0 entries 2024-05-11T08:06:33.932 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.932+0000 7fe13950a640 1 -- 172.21.15.167:0/3329500488 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fe1140762d0 msgr2=0x7fe114078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:33.933 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.932+0000 7fe13950a640 1 --2- 172.21.15.167:0/3329500488 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fe1140762d0 0x7fe114078790 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe124034000 tx=0x7fe12402e000 comp rx=0 tx=0).stop 2024-05-11T08:06:33.933 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.932+0000 7fe13950a640 1 -- 172.21.15.167:0/3329500488 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe1340d0640 msgr2=0x7fe134146810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:33.933 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.932+0000 7fe13950a640 1 --2- 172.21.15.167:0/3329500488 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe1340d0640 0x7fe134146810 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fe11c00c820 tx=0x7fe11c00cc20 comp rx=0 tx=0).stop 2024-05-11T08:06:33.933 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.932+0000 7fe13950a640 1 -- 172.21.15.167:0/3329500488 shutdown_connections 2024-05-11T08:06:33.933 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.932+0000 7fe13950a640 1 --2- 172.21.15.167:0/3329500488 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fe1140762d0 0x7fe114078790 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:33.933 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.932+0000 7fe13950a640 1 --2- 172.21.15.167:0/3329500488 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe1341598d0 0x7fe1341434b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:33.933 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.932+0000 7fe13950a640 1 --2- 172.21.15.167:0/3329500488 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe1340d0640 0x7fe134146810 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:33.933 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.932+0000 7fe13950a640 1 --2- 172.21.15.167:0/3329500488 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe1340cafb0 0x7fe1340b7e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:33.933 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.932+0000 7fe13950a640 1 -- 172.21.15.167:0/3329500488 >> 172.21.15.167:0/3329500488 conn(0x7fe1340d1ba0 msgr2=0x7fe134162350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:33.933 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.932+0000 7fe13950a640 1 -- 172.21.15.167:0/3329500488 shutdown_connections 2024-05-11T08:06:33.933 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:33.932+0000 7fe13950a640 1 -- 172.21.15.167:0/3329500488 wait complete. 2024-05-11T08:06:33.944 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-05-11T08:06:33.944 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T08:06:33.944 DEBUG:teuthology.orchestra.run.smithi204:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2024-05-11T08:06:33.974 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi204.front.sepia.ceph.com 2024-05-11T08:06:33.974 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T08:06:33.975 DEBUG:teuthology.orchestra.run.smithi204:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-05-11T08:06:34.035 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-05-11T08:06:34.036 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T08:06:34.052 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-05-11T08:06:34.052 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T08:06:34.108 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-05-11T08:06:34.108 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T08:06:34.164 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-11T08:06:34.305 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.304+0000 7f28c7a08640 1 Processor -- start 2024-05-11T08:06:34.305 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.305+0000 7f28c7a08640 1 -- start start 2024-05-11T08:06:34.308 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.305+0000 7f28c7a08640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f28c013eeb0 0x7f28c006f480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:34.308 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.305+0000 7f28c7a08640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f28c013f290 0x7f28c006f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:34.308 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.305+0000 7f28c7a08640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28c015fb90 con 0x7f28c00b50a0 2024-05-11T08:06:34.309 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.305+0000 7f28c7a08640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f28c015d5b0 con 0x7f28c013eeb0 2024-05-11T08:06:34.309 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.306+0000 7f28c7a08640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f28c015fe70 con 0x7f28c013f290 2024-05-11T08:06:34.309 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.306+0000 7f28c5fa5640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f28c00b50a0 0x7f28c00735c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:47324/0 (socket says 172.21.15.167:47324) 2024-05-11T08:06:34.309 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.306+0000 7f28c5fa5640 1 -- 172.21.15.167:0/2790269794 learned_addr learned my addr 172.21.15.167:0/2790269794 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T08:06:34.309 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.306+0000 7f28c4fa3640 1 --2- 172.21.15.167:0/2790269794 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f28c013f290 0x7f28c006f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:34.309 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.306+0000 7f28c57a4640 1 --2- 172.21.15.167:0/2790269794 >> v2:172.21.15.204:3300/0 conn(0x7f28c013eeb0 0x7f28c006f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:34.309 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.306+0000 7f28bffff640 1 -- 172.21.15.167:0/2790269794 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1004590342 0 0) 0x7f28c015fb90 con 0x7f28c00b50a0 2024-05-11T08:06:34.309 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.306+0000 7f28bffff640 1 -- 172.21.15.167:0/2790269794 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f28a8003520 con 0x7f28c00b50a0 2024-05-11T08:06:34.309 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.306+0000 7f28bffff640 1 -- 172.21.15.167:0/2790269794 <== mon.0 v1:172.21.15.167:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2398841855 0 0) 0x7f28a8003520 con 0x7f28c00b50a0 2024-05-11T08:06:34.309 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.306+0000 7f28bffff640 1 -- 172.21.15.167:0/2790269794 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f28c013f290 msgr2=0x7f28c006f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:34.309 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.306+0000 7f28bffff640 1 --2- 172.21.15.167:0/2790269794 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f28c013f290 0x7f28c006f9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:34.309 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.306+0000 7f28bffff640 1 -- 172.21.15.167:0/2790269794 >> v2:172.21.15.204:3300/0 conn(0x7f28c013eeb0 msgr2=0x7f28c006f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:34.309 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.306+0000 7f28bffff640 1 --2- 172.21.15.167:0/2790269794 >> v2:172.21.15.204:3300/0 conn(0x7f28c013eeb0 0x7f28c006f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:34.310 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.306+0000 7f28bffff640 1 -- 172.21.15.167:0/2790269794 --> v1:172.21.15.167:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28c0160150 con 0x7f28c00b50a0 2024-05-11T08:06:34.310 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.306+0000 7f28bffff640 1 -- 172.21.15.167:0/2790269794 <== mon.0 v1:172.21.15.167:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3602277846 0 0) 0x7f28b0001a70 con 0x7f28c00b50a0 2024-05-11T08:06:34.310 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.306+0000 7f28bffff640 1 -- 172.21.15.167:0/2790269794 >> v1:172.21.15.167:6789/0 conn(0x7f28c00b50a0 legacy=0x7f28c00735c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:34.310 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.306+0000 7f28bffff640 1 --2- 172.21.15.167:0/2790269794 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f28c013f290 0x7f28a8003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:34.310 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.306+0000 7f28bffff640 1 --2- 172.21.15.167:0/2790269794 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f28a8003fd0 0x7f28a8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:34.310 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.306+0000 7f28bffff640 1 --2- 172.21.15.167:0/2790269794 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f28a8004d40 0x7f28a80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:34.310 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.306+0000 7f28bffff640 1 -- 172.21.15.167:0/2790269794 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f28c015fe70 con 0x7f28a8004d40 2024-05-11T08:06:34.310 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.306+0000 7f28bffff640 1 -- 172.21.15.167:0/2790269794 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f28a800ba20 con 0x7f28a8003fd0 2024-05-11T08:06:34.310 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.306+0000 7f28bffff640 1 -- 172.21.15.167:0/2790269794 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f28a800bc20 con 0x7f28c013f290 2024-05-11T08:06:34.310 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.306+0000 7f28c5fa5640 1 --2- 172.21.15.167:0/2790269794 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f28a8004d40 0x7f28a80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:34.310 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.306+0000 7f28c4fa3640 1 --2- 172.21.15.167:0/2790269794 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f28a8003fd0 0x7f28a8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:34.310 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.306+0000 7f28c57a4640 1 --2- 172.21.15.167:0/2790269794 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f28c013f290 0x7f28a8003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:34.310 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.307+0000 7f28c5fa5640 1 -- 172.21.15.167:0/2790269794 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f28c013f290 msgr2=0x7f28a8003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:34.310 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.307+0000 7f28c5fa5640 1 --2- 172.21.15.167:0/2790269794 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f28c013f290 0x7f28a8003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:34.310 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.307+0000 7f28c5fa5640 1 -- 172.21.15.167:0/2790269794 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f28a8003fd0 msgr2=0x7f28a8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:34.311 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.307+0000 7f28c5fa5640 1 --2- 172.21.15.167:0/2790269794 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f28a8003fd0 0x7f28a8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:34.311 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.307+0000 7f28c5fa5640 1 -- 172.21.15.167:0/2790269794 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28a8003520 con 0x7f28a8004d40 2024-05-11T08:06:34.311 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.307+0000 7f28c5fa5640 1 --2- 172.21.15.167:0/2790269794 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f28a8004d40 0x7f28a80051c0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f28b000a920 tx=0x7f28a8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4d9a0870410324f server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:34.311 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.307+0000 7f28bffff640 1 -- 172.21.15.167:0/2790269794 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28b000a4c0 con 0x7f28a8004d40 2024-05-11T08:06:34.311 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.307+0000 7f28bffff640 1 -- 172.21.15.167:0/2790269794 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28b002f070 con 0x7f28a8004d40 2024-05-11T08:06:34.311 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.308+0000 7f28bffff640 1 -- 172.21.15.167:0/2790269794 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28b002bc50 con 0x7f28a8004d40 2024-05-11T08:06:34.311 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.308+0000 7f28c7a08640 1 -- 172.21.15.167:0/2790269794 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f28a8004d40 msgr2=0x7f28a80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:34.311 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.308+0000 7f28c7a08640 1 --2- 172.21.15.167:0/2790269794 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f28a8004d40 0x7f28a80051c0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f28b000a920 tx=0x7f28a8005830 comp rx=0 tx=0).stop 2024-05-11T08:06:34.311 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.308+0000 7f28c5fa5640 1 -- 172.21.15.167:0/2790269794 reap_dead start 2024-05-11T08:06:34.311 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.308+0000 7f28c7a08640 1 -- 172.21.15.167:0/2790269794 shutdown_connections 2024-05-11T08:06:34.311 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.308+0000 7f28c7a08640 1 -- 172.21.15.167:0/2790269794 >> 172.21.15.167:0/2790269794 conn(0x7f28c00cd8f0 msgr2=0x7f28c00cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:34.311 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.308+0000 7f28c7a08640 1 -- 172.21.15.167:0/2790269794 shutdown_connections 2024-05-11T08:06:34.311 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.308+0000 7f28c7a08640 1 -- 172.21.15.167:0/2790269794 wait complete. 2024-05-11T08:06:34.314 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.309+0000 7f28c7a08640 1 Processor -- start 2024-05-11T08:06:34.314 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.309+0000 7f28c7a08640 1 -- start start 2024-05-11T08:06:34.314 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.310+0000 7f28c7a08640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f28a8004d40 0x7f28c013a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:34.314 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.310+0000 7f28c7a08640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f28c0143e50 0x7f28c013d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:34.314 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.310+0000 7f28c7a08640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f28c01442b0 0x7f28c015e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:34.314 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.310+0000 7f28c7a08640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f28c0073cd0 con 0x7f28c0143e50 2024-05-11T08:06:34.314 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.310+0000 7f28c7a08640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f28c0160710 con 0x7f28c01442b0 2024-05-11T08:06:34.314 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.310+0000 7f28c7a08640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f28c0073e50 con 0x7f28a8004d40 2024-05-11T08:06:34.314 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.310+0000 7f28c4fa3640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f28c0143e50 0x7f28c013d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:34.315 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.310+0000 7f28c4fa3640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f28c0143e50 0x7f28c013d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:58730/0 (socket says 172.21.15.167:58730) 2024-05-11T08:06:34.315 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.310+0000 7f28c4fa3640 1 -- 172.21.15.167:0/371100339 learned_addr learned my addr 172.21.15.167:0/371100339 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:34.315 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.310+0000 7f28c4fa3640 1 -- 172.21.15.167:0/371100339 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f28a8004d40 msgr2=0x7f28c013a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:34.315 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.310+0000 7f28c4fa3640 1 --2- 172.21.15.167:0/371100339 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f28a8004d40 0x7f28c013a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:34.315 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.310+0000 7f28c4fa3640 1 -- 172.21.15.167:0/371100339 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f28c01442b0 msgr2=0x7f28c015e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:34.315 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.310+0000 7f28c4fa3640 1 --2- 172.21.15.167:0/371100339 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f28c01442b0 0x7f28c015e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:34.315 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.310+0000 7f28c4fa3640 1 -- 172.21.15.167:0/371100339 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28c01551f0 con 0x7f28c0143e50 2024-05-11T08:06:34.315 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.311+0000 7f28c4fa3640 1 --2- 172.21.15.167:0/371100339 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f28c0143e50 0x7f28c013d990 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f28b40119a0 tx=0x7f28b4011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:34.315 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.311+0000 7f28bdffb640 1 -- 172.21.15.167:0/371100339 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28b400fa70 con 0x7f28c0143e50 2024-05-11T08:06:34.315 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.311+0000 7f28bdffb640 1 -- 172.21.15.167:0/371100339 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28b400fc10 con 0x7f28c0143e50 2024-05-11T08:06:34.315 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.311+0000 7f28c7a08640 1 -- 172.21.15.167:0/371100339 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28c01554d0 con 0x7f28c0143e50 2024-05-11T08:06:34.315 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.311+0000 7f28bdffb640 1 -- 172.21.15.167:0/371100339 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28b4014440 con 0x7f28c0143e50 2024-05-11T08:06:34.315 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.311+0000 7f28c7a08640 1 -- 172.21.15.167:0/371100339 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28c0155a90 con 0x7f28c0143e50 2024-05-11T08:06:34.315 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.312+0000 7f28c7a08640 1 -- 172.21.15.167:0/371100339 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28c0069590 con 0x7f28c0143e50 2024-05-11T08:06:34.315 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.313+0000 7f28bdffb640 1 -- 172.21.15.167:0/371100339 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f28b4012040 con 0x7f28c0143e50 2024-05-11T08:06:34.315 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.313+0000 7f28bdffb640 1 --2- 172.21.15.167:0/371100339 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f28ac076300 0x7f28ac0787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:34.315 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.313+0000 7f28bdffb640 1 -- 172.21.15.167:0/371100339 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f28b4098040 con 0x7f28c0143e50 2024-05-11T08:06:34.316 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.313+0000 7f28c57a4640 1 --2- 172.21.15.167:0/371100339 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f28ac076300 0x7f28ac0787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:34.316 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.314+0000 7f28c57a4640 1 --2- 172.21.15.167:0/371100339 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f28ac076300 0x7f28ac0787c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f28b8006e80 tx=0x7f28b8006df0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:34.317 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.318+0000 7f28bdffb640 1 -- 172.21.15.167:0/371100339 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f28b4019030 con 0x7f28c0143e50 2024-05-11T08:06:34.512 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.512+0000 7f28c7a08640 1 -- 172.21.15.167:0/371100339 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f28c00b49c0 con 0x7f28c0143e50 2024-05-11T08:06:34.512 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.512+0000 7f28bdffb640 1 -- 172.21.15.167:0/371100339 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) ==== 80+0+0 (secure 0 0 0) 0x7f28b40615d0 con 0x7f28c0143e50 2024-05-11T08:06:34.512 INFO:teuthology.orchestra.run.smithi167.stderr:listed 0 entries 2024-05-11T08:06:34.514 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.514+0000 7f28c7a08640 1 -- 172.21.15.167:0/371100339 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f28ac076300 msgr2=0x7f28ac0787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:34.514 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.514+0000 7f28c7a08640 1 --2- 172.21.15.167:0/371100339 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f28ac076300 0x7f28ac0787c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f28b8006e80 tx=0x7f28b8006df0 comp rx=0 tx=0).stop 2024-05-11T08:06:34.515 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.514+0000 7f28c7a08640 1 -- 172.21.15.167:0/371100339 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f28c0143e50 msgr2=0x7f28c013d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:34.515 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.514+0000 7f28c7a08640 1 --2- 172.21.15.167:0/371100339 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f28c0143e50 0x7f28c013d990 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f28b40119a0 tx=0x7f28b4011e60 comp rx=0 tx=0).stop 2024-05-11T08:06:34.515 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.514+0000 7f28c7a08640 1 -- 172.21.15.167:0/371100339 shutdown_connections 2024-05-11T08:06:34.515 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.514+0000 7f28c7a08640 1 --2- 172.21.15.167:0/371100339 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f28ac076300 0x7f28ac0787c0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:34.515 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.514+0000 7f28c7a08640 1 --2- 172.21.15.167:0/371100339 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f28c01442b0 0x7f28c015e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:34.515 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.514+0000 7f28c7a08640 1 --2- 172.21.15.167:0/371100339 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f28c0143e50 0x7f28c013d990 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:34.515 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.514+0000 7f28c7a08640 1 --2- 172.21.15.167:0/371100339 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f28a8004d40 0x7f28c013a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:34.515 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.514+0000 7f28c7a08640 1 -- 172.21.15.167:0/371100339 >> 172.21.15.167:0/371100339 conn(0x7f28c00cd8f0 msgr2=0x7f28c015b430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:34.515 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.514+0000 7f28c7a08640 1 -- 172.21.15.167:0/371100339 shutdown_connections 2024-05-11T08:06:34.515 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.514+0000 7f28c7a08640 1 -- 172.21.15.167:0/371100339 wait complete. 2024-05-11T08:06:34.525 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2024-05-11T08:06:34.525 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T08:06:34.526 DEBUG:teuthology.orchestra.run.smithi167:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2024-05-11T08:06:34.551 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi167.front.sepia.ceph.com 2024-05-11T08:06:34.552 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T08:06:34.552 DEBUG:teuthology.orchestra.run.smithi167:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-05-11T08:06:34.607 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-11T08:06:34.607 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T08:06:34.627 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-11T08:06:34.627 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T08:06:34.686 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-11T08:06:34.687 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T08:06:34.744 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-11T08:06:34.846 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.845+0000 7f7c6b209640 1 Processor -- start 2024-05-11T08:06:34.846 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.845+0000 7f7c6b209640 1 -- start start 2024-05-11T08:06:34.847 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.846+0000 7f7c6b209640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f7c64146b40 0x7f7c6406e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:34.847 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.846+0000 7f7c6b209640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c6415e520 con 0x7f7c64146760 2024-05-11T08:06:34.847 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.846+0000 7f7c6b209640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c640709d0 con 0x7f7c6406eb50 2024-05-11T08:06:34.847 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.846+0000 7f7c6b209640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f7c640707a0 con 0x7f7c64146b40 2024-05-11T08:06:34.847 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.847+0000 7f7c68fa5640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f7c64146760 0x7f7c640d13a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:49554/0 (socket says 172.21.15.167:49554) 2024-05-11T08:06:34.847 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.847+0000 7f7c68fa5640 1 -- 172.21.15.167:0/4052343556 learned_addr learned my addr 172.21.15.167:0/4052343556 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T08:06:34.848 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.847+0000 7f7c63fff640 1 --2- 172.21.15.167:0/4052343556 >> v2:172.21.15.204:3300/0 conn(0x7f7c64146b40 0x7f7c6406e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:34.848 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.847+0000 7f7c637fe640 1 -- 172.21.15.167:0/4052343556 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 488955440 0 0) 0x7f7c6415e520 con 0x7f7c64146760 2024-05-11T08:06:34.848 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.847+0000 7f7c637fe640 1 -- 172.21.15.167:0/4052343556 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7c50003520 con 0x7f7c64146760 2024-05-11T08:06:34.848 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.847+0000 7f7c63fff640 1 -- 172.21.15.167:0/4052343556 >> v1:172.21.15.204:6789/0 conn(0x7f7c6406eb50 legacy=0x7f7c6406cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:34.848 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.847+0000 7f7c63fff640 1 -- 172.21.15.167:0/4052343556 >> v1:172.21.15.167:6789/0 conn(0x7f7c64146760 legacy=0x7f7c640d13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:34.848 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.847+0000 7f7c637fe640 1 -- 172.21.15.167:0/4052343556 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3551238563 0 0) 0x7f7c640709d0 con 0x7f7c6406eb50 2024-05-11T08:06:34.848 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.847+0000 7f7c63fff640 1 -- 172.21.15.167:0/4052343556 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c64070cb0 con 0x7f7c64146b40 2024-05-11T08:06:34.849 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.848+0000 7f7c63fff640 1 --2- 172.21.15.167:0/4052343556 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7c64146b40 0x7f7c6406e610 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7c4c0097a0 tx=0x7f7c4c02a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=64e4ef7450b74344 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:34.849 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.848+0000 7f7c637fe640 1 -- 172.21.15.167:0/4052343556 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c4c037070 con 0x7f7c64146b40 2024-05-11T08:06:34.850 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.848+0000 7f7c637fe640 1 -- 172.21.15.167:0/4052343556 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c4c02ac30 con 0x7f7c64146b40 2024-05-11T08:06:34.850 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.848+0000 7f7c637fe640 1 -- 172.21.15.167:0/4052343556 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c54001a70 con 0x7f7c64146b40 2024-05-11T08:06:34.850 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.848+0000 7f7c6b209640 1 -- 172.21.15.167:0/4052343556 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7c64146b40 msgr2=0x7f7c6406e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:34.850 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.848+0000 7f7c6b209640 1 --2- 172.21.15.167:0/4052343556 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7c64146b40 0x7f7c6406e610 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7c4c0097a0 tx=0x7f7c4c02a1a0 comp rx=0 tx=0).stop 2024-05-11T08:06:34.850 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.848+0000 7f7c6b209640 1 -- 172.21.15.167:0/4052343556 shutdown_connections 2024-05-11T08:06:34.850 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.848+0000 7f7c6b209640 1 --2- 172.21.15.167:0/4052343556 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7c64146b40 0x7f7c6406e610 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:34.850 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.848+0000 7f7c6b209640 1 -- 172.21.15.167:0/4052343556 >> 172.21.15.167:0/4052343556 conn(0x7f7c640d1e40 msgr2=0x7f7c640d2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:34.850 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.849+0000 7f7c6b209640 1 -- 172.21.15.167:0/4052343556 shutdown_connections 2024-05-11T08:06:34.850 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.849+0000 7f7c6b209640 1 -- 172.21.15.167:0/4052343556 wait complete. 2024-05-11T08:06:34.850 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.850+0000 7f7c6b209640 1 Processor -- start 2024-05-11T08:06:34.850 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.850+0000 7f7c6b209640 1 -- start start 2024-05-11T08:06:34.852 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.851+0000 7f7c6b209640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7c6406eb50 0x7f7c640ae820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:34.852 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.851+0000 7f7c6b209640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7c64146760 0x7f7c640aed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:34.852 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.851+0000 7f7c6b209640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7c64146b40 0x7f7c64133390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:34.852 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.851+0000 7f7c6b209640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7c64071270 con 0x7f7c64146b40 2024-05-11T08:06:34.853 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.851+0000 7f7c6b209640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7c6406ef80 con 0x7f7c6406eb50 2024-05-11T08:06:34.853 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.851+0000 7f7c6b209640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f7c64071550 con 0x7f7c64146760 2024-05-11T08:06:34.853 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.851+0000 7f7c697a6640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7c64146b40 0x7f7c64133390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:34.853 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.851+0000 7f7c697a6640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7c64146b40 0x7f7c64133390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:34744/0 (socket says 172.21.15.167:34744) 2024-05-11T08:06:34.853 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.851+0000 7f7c697a6640 1 -- 172.21.15.167:0/4160134622 learned_addr learned my addr 172.21.15.167:0/4160134622 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:34.853 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.851+0000 7f7c68fa5640 1 --2- 172.21.15.167:0/4160134622 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7c6406eb50 0x7f7c640ae820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:34.853 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.851+0000 7f7c63fff640 1 --2- 172.21.15.167:0/4160134622 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7c64146760 0x7f7c640aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:34.853 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.851+0000 7f7c697a6640 1 -- 172.21.15.167:0/4160134622 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7c64146760 msgr2=0x7f7c640aed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:34.853 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.851+0000 7f7c697a6640 1 --2- 172.21.15.167:0/4160134622 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7c64146760 0x7f7c640aed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:34.853 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.851+0000 7f7c697a6640 1 -- 172.21.15.167:0/4160134622 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7c6406eb50 msgr2=0x7f7c640ae820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:34.853 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.851+0000 7f7c697a6640 1 --2- 172.21.15.167:0/4160134622 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7c6406eb50 0x7f7c640ae820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:34.853 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.851+0000 7f7c697a6640 1 -- 172.21.15.167:0/4160134622 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c641314b0 con 0x7f7c64146b40 2024-05-11T08:06:34.853 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.851+0000 7f7c697a6640 1 --2- 172.21.15.167:0/4160134622 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7c64146b40 0x7f7c64133390 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f7c5c00c3a0 tx=0x7f7c5c00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:34.854 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.851+0000 7f7c617fa640 1 -- 172.21.15.167:0/4160134622 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c5c017070 con 0x7f7c64146b40 2024-05-11T08:06:34.854 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.851+0000 7f7c617fa640 1 -- 172.21.15.167:0/4160134622 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c5c0035f0 con 0x7f7c64146b40 2024-05-11T08:06:34.854 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.851+0000 7f7c6b209640 1 -- 172.21.15.167:0/4160134622 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c640b66f0 con 0x7f7c64146b40 2024-05-11T08:06:34.854 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.851+0000 7f7c617fa640 1 -- 172.21.15.167:0/4160134622 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c5c0039e0 con 0x7f7c64146b40 2024-05-11T08:06:34.854 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.851+0000 7f7c6b209640 1 -- 172.21.15.167:0/4160134622 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c64131a50 con 0x7f7c64146b40 2024-05-11T08:06:34.854 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.852+0000 7f7c6b209640 1 -- 172.21.15.167:0/4160134622 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c64069bc0 con 0x7f7c64146b40 2024-05-11T08:06:34.855 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.853+0000 7f7c617fa640 1 -- 172.21.15.167:0/4160134622 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f7c5c01a070 con 0x7f7c64146b40 2024-05-11T08:06:34.855 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.853+0000 7f7c617fa640 1 --2- 172.21.15.167:0/4160134622 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f7c440762d0 0x7f7c44078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:34.855 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.853+0000 7f7c617fa640 1 -- 172.21.15.167:0/4160134622 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f7c5c096c00 con 0x7f7c64146b40 2024-05-11T08:06:34.855 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.853+0000 7f7c68fa5640 1 --2- 172.21.15.167:0/4160134622 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f7c440762d0 0x7f7c44078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:34.855 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.854+0000 7f7c68fa5640 1 --2- 172.21.15.167:0/4160134622 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f7c440762d0 0x7f7c44078790 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f7c540019a0 tx=0x7f7c540018f0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:34.858 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:34.858+0000 7f7c617fa640 1 -- 172.21.15.167:0/4160134622 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f7c5c09d080 con 0x7f7c64146b40 2024-05-11T08:06:35.050 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.050+0000 7f7c6b209640 1 -- 172.21.15.167:0/4160134622 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f7c641607d0 con 0x7f7c64146b40 2024-05-11T08:06:35.051 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.051+0000 7f7c617fa640 1 -- 172.21.15.167:0/4160134622 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) ==== 80+0+0 (secure 0 0 0) 0x7f7c5c05fbd0 con 0x7f7c64146b40 2024-05-11T08:06:35.051 INFO:teuthology.orchestra.run.smithi167.stderr:listed 0 entries 2024-05-11T08:06:35.053 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.053+0000 7f7c6b209640 1 -- 172.21.15.167:0/4160134622 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f7c440762d0 msgr2=0x7f7c44078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:35.053 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.053+0000 7f7c6b209640 1 --2- 172.21.15.167:0/4160134622 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f7c440762d0 0x7f7c44078790 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f7c540019a0 tx=0x7f7c540018f0 comp rx=0 tx=0).stop 2024-05-11T08:06:35.053 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.053+0000 7f7c6b209640 1 -- 172.21.15.167:0/4160134622 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7c64146b40 msgr2=0x7f7c64133390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:35.053 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.053+0000 7f7c6b209640 1 --2- 172.21.15.167:0/4160134622 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7c64146b40 0x7f7c64133390 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f7c5c00c3a0 tx=0x7f7c5c00c860 comp rx=0 tx=0).stop 2024-05-11T08:06:35.053 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.053+0000 7f7c6b209640 1 -- 172.21.15.167:0/4160134622 shutdown_connections 2024-05-11T08:06:35.053 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.053+0000 7f7c6b209640 1 --2- 172.21.15.167:0/4160134622 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f7c440762d0 0x7f7c44078790 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:35.053 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.053+0000 7f7c6b209640 1 --2- 172.21.15.167:0/4160134622 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7c64146b40 0x7f7c64133390 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:35.053 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.053+0000 7f7c6b209640 1 --2- 172.21.15.167:0/4160134622 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7c64146760 0x7f7c640aed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:35.053 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.053+0000 7f7c6b209640 1 --2- 172.21.15.167:0/4160134622 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7c6406eb50 0x7f7c640ae820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:35.054 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.053+0000 7f7c6b209640 1 -- 172.21.15.167:0/4160134622 >> 172.21.15.167:0/4160134622 conn(0x7f7c640d1e40 msgr2=0x7f7c64161280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:35.054 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.053+0000 7f7c6b209640 1 -- 172.21.15.167:0/4160134622 shutdown_connections 2024-05-11T08:06:35.054 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.053+0000 7f7c6b209640 1 -- 172.21.15.167:0/4160134622 wait complete. 2024-05-11T08:06:35.064 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2024-05-11T08:06:35.064 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T08:06:35.064 DEBUG:teuthology.orchestra.run.smithi204:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2024-05-11T08:06:35.096 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi204.front.sepia.ceph.com 2024-05-11T08:06:35.097 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T08:06:35.097 DEBUG:teuthology.orchestra.run.smithi204:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-05-11T08:06:35.155 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-05-11T08:06:35.155 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-11T08:06:35.256 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.256+0000 7f570c190640 1 Processor -- start 2024-05-11T08:06:35.256 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.256+0000 7f570c190640 1 -- start start 2024-05-11T08:06:35.259 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.257+0000 7f570c190640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f570415f360 0x7f57040cfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:35.259 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.257+0000 7f570c190640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f57040b5e90 0x7f570415f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:35.259 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.257+0000 7f570c190640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57040d2410 con 0x7f570415fc80 2024-05-11T08:06:35.260 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.257+0000 7f570c190640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f57040d26f0 con 0x7f570415f360 2024-05-11T08:06:35.260 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.257+0000 7f570c190640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f57040d29d0 con 0x7f57040b5e90 2024-05-11T08:06:35.260 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.257+0000 7f5709f2c640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f570415f360 0x7f57040cfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:35.260 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.257+0000 7f570a72d640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f570415fc80 0x7f57040cd610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:49568/0 (socket says 172.21.15.167:49568) 2024-05-11T08:06:35.260 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.257+0000 7f570a72d640 1 -- 172.21.15.167:0/2825181197 learned_addr learned my addr 172.21.15.167:0/2825181197 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T08:06:35.260 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.257+0000 7f570972b640 1 --2- 172.21.15.167:0/2825181197 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f57040b5e90 0x7f570415f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:35.260 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.257+0000 7f5709f2c640 1 -- 172.21.15.167:0/2825181197 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f57040b5e90 msgr2=0x7f570415f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:35.260 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.257+0000 7f5709f2c640 1 --2- 172.21.15.167:0/2825181197 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f57040b5e90 0x7f570415f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:35.260 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.257+0000 7f5709f2c640 1 -- 172.21.15.167:0/2825181197 >> v1:172.21.15.167:6789/0 conn(0x7f570415fc80 legacy=0x7f57040cd610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:35.260 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.257+0000 7f5708f2a640 1 -- 172.21.15.167:0/2825181197 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1107476066 0 0) 0x7f57040d2410 con 0x7f570415fc80 2024-05-11T08:06:35.260 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.257+0000 7f5709f2c640 1 -- 172.21.15.167:0/2825181197 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57040b7e20 con 0x7f570415f360 2024-05-11T08:06:35.260 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.257+0000 7f570972b640 1 --2- 172.21.15.167:0/2825181197 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f57040b5e90 0x7f570415f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T08:06:35.261 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.258+0000 7f5709f2c640 1 --2- 172.21.15.167:0/2825181197 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f570415f360 0x7f57040cfdc0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f57000097a0 tx=0x7f570002a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80be14477b54811f server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:35.261 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.258+0000 7f5708f2a640 1 -- 172.21.15.167:0/2825181197 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5700037070 con 0x7f570415f360 2024-05-11T08:06:35.261 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.258+0000 7f5708f2a640 1 -- 172.21.15.167:0/2825181197 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f570002ae50 con 0x7f570415f360 2024-05-11T08:06:35.261 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.258+0000 7f5708f2a640 1 -- 172.21.15.167:0/2825181197 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5700032660 con 0x7f570415f360 2024-05-11T08:06:35.261 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.259+0000 7f570c190640 1 -- 172.21.15.167:0/2825181197 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f570415f360 msgr2=0x7f57040cfdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:35.261 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.259+0000 7f570c190640 1 --2- 172.21.15.167:0/2825181197 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f570415f360 0x7f57040cfdc0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f57000097a0 tx=0x7f570002a100 comp rx=0 tx=0).stop 2024-05-11T08:06:35.261 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.259+0000 7f570c190640 1 -- 172.21.15.167:0/2825181197 shutdown_connections 2024-05-11T08:06:35.261 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.259+0000 7f570c190640 1 --2- 172.21.15.167:0/2825181197 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f57040b5e90 0x7f570415f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:35.261 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.259+0000 7f570c190640 1 --2- 172.21.15.167:0/2825181197 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f570415f360 0x7f57040cfdc0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:35.261 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.259+0000 7f570c190640 1 -- 172.21.15.167:0/2825181197 >> 172.21.15.167:0/2825181197 conn(0x7f5704072730 msgr2=0x7f57040d2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:35.261 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.259+0000 7f570c190640 1 -- 172.21.15.167:0/2825181197 shutdown_connections 2024-05-11T08:06:35.261 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.259+0000 7f570c190640 1 -- 172.21.15.167:0/2825181197 wait complete. 2024-05-11T08:06:35.264 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.260+0000 7f570c190640 1 Processor -- start 2024-05-11T08:06:35.264 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.260+0000 7f570c190640 1 -- start start 2024-05-11T08:06:35.264 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.261+0000 7f570c190640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f57040b5e90 0x7f570415d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:35.265 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.261+0000 7f570c190640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f570415f360 0x7f57040af0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:35.265 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.261+0000 7f570c190640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f570415fc80 0x7f57040b1890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:35.265 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.261+0000 7f570c190640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f57040cdd20 con 0x7f57040b5e90 2024-05-11T08:06:35.265 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.261+0000 7f570c190640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f57040b7ad0 con 0x7f570415f360 2024-05-11T08:06:35.265 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.261+0000 7f570c190640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f57040cdea0 con 0x7f570415fc80 2024-05-11T08:06:35.265 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.261+0000 7f5709f2c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f57040b5e90 0x7f570415d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:35.265 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.261+0000 7f5709f2c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f57040b5e90 0x7f570415d500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:34764/0 (socket says 172.21.15.167:34764) 2024-05-11T08:06:35.265 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.261+0000 7f5709f2c640 1 -- 172.21.15.167:0/2854644403 learned_addr learned my addr 172.21.15.167:0/2854644403 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:35.265 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.261+0000 7f5709f2c640 1 -- 172.21.15.167:0/2854644403 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f570415fc80 msgr2=0x7f57040b1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:35.265 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.261+0000 7f5709f2c640 1 --2- 172.21.15.167:0/2854644403 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f570415fc80 0x7f57040b1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:35.265 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.261+0000 7f5709f2c640 1 -- 172.21.15.167:0/2854644403 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f570415f360 msgr2=0x7f57040af0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:35.265 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.261+0000 7f5709f2c640 1 --2- 172.21.15.167:0/2854644403 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f570415f360 0x7f57040af0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:35.266 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.261+0000 7f5709f2c640 1 -- 172.21.15.167:0/2854644403 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f570406fa30 con 0x7f57040b5e90 2024-05-11T08:06:35.266 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.261+0000 7f5709f2c640 1 --2- 172.21.15.167:0/2854644403 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f57040b5e90 0x7f570415d500 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f5700035040 tx=0x7f57000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:35.266 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.261+0000 7f56fa7fc640 1 -- 172.21.15.167:0/2854644403 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5700037040 con 0x7f57040b5e90 2024-05-11T08:06:35.266 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.261+0000 7f56fa7fc640 1 -- 172.21.15.167:0/2854644403 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5700004250 con 0x7f57040b5e90 2024-05-11T08:06:35.266 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.261+0000 7f56fa7fc640 1 -- 172.21.15.167:0/2854644403 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5700003610 con 0x7f57040b5e90 2024-05-11T08:06:35.266 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.261+0000 7f570c190640 1 -- 172.21.15.167:0/2854644403 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f570406fd10 con 0x7f57040b5e90 2024-05-11T08:06:35.266 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.262+0000 7f570c190640 1 -- 172.21.15.167:0/2854644403 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57040702d0 con 0x7f57040b5e90 2024-05-11T08:06:35.266 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.263+0000 7f570c190640 1 -- 172.21.15.167:0/2854644403 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5704069590 con 0x7f57040b5e90 2024-05-11T08:06:35.266 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.263+0000 7f56fa7fc640 1 -- 172.21.15.167:0/2854644403 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f57000037b0 con 0x7f57040b5e90 2024-05-11T08:06:35.266 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.264+0000 7f56fa7fc640 1 --2- 172.21.15.167:0/2854644403 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f56e80762d0 0x7f56e8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:35.266 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.264+0000 7f570972b640 1 --2- 172.21.15.167:0/2854644403 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f56e80762d0 0x7f56e8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:35.267 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.264+0000 7f56fa7fc640 1 -- 172.21.15.167:0/2854644403 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f57000b8740 con 0x7f57040b5e90 2024-05-11T08:06:35.267 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.264+0000 7f570972b640 1 --2- 172.21.15.167:0/2854644403 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f56e80762d0 0x7f56e8078790 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f56f4006fd0 tx=0x7f56f4008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:35.268 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.268+0000 7f56fa7fc640 1 -- 172.21.15.167:0/2854644403 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f5700081df0 con 0x7f57040b5e90 2024-05-11T08:06:35.489 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.489+0000 7f570c190640 1 -- 172.21.15.167:0/2854644403 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f57041618d0 con 0x7f57040b5e90 2024-05-11T08:06:35.493 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.493+0000 7f56fa7fc640 1 -- 172.21.15.167:0/2854644403 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5183 (secure 0 0 0) 0x7f57000816f0 con 0x7f57040b5e90 2024-05-11T08:06:35.493 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T08:06:35.493 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6834/506789040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":506789040},{"type":"v1","addr":"172.21.15.204:6834","nonce":506789040}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4287,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1448126000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":1448126000},{"type":"v1","addr":"172.21.15.167:6837","nonce":1448126000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4290,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/766834562","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":766834562},{"type":"v1","addr":"172.21.15.167:6836","nonce":766834562}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-11T08:05:54.711448+0000","modified":"2024-05-11T08:06:13.780553+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4305},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4305":{"gid":4305,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.204:6835/1247774569","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6833","nonce":1247774569},{"type":"v1","addr":"172.21.15.204:6835","nonce":1247774569}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4305,"qdb_cluster":[4305]},"id":1}]} 2024-05-11T08:06:35.493 INFO:teuthology.orchestra.run.smithi167.stderr:dumped fsmap epoch 9 2024-05-11T08:06:35.495 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.495+0000 7f570c190640 1 -- 172.21.15.167:0/2854644403 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f56e80762d0 msgr2=0x7f56e8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:35.495 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.495+0000 7f570c190640 1 --2- 172.21.15.167:0/2854644403 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f56e80762d0 0x7f56e8078790 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f56f4006fd0 tx=0x7f56f4008040 comp rx=0 tx=0).stop 2024-05-11T08:06:35.495 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.495+0000 7f570c190640 1 -- 172.21.15.167:0/2854644403 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f57040b5e90 msgr2=0x7f570415d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:35.495 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.495+0000 7f570c190640 1 --2- 172.21.15.167:0/2854644403 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f57040b5e90 0x7f570415d500 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f5700035040 tx=0x7f57000046c0 comp rx=0 tx=0).stop 2024-05-11T08:06:35.495 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.495+0000 7f570c190640 1 -- 172.21.15.167:0/2854644403 shutdown_connections 2024-05-11T08:06:35.495 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.495+0000 7f570c190640 1 --2- 172.21.15.167:0/2854644403 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f56e80762d0 0x7f56e8078790 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:35.496 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.495+0000 7f570c190640 1 --2- 172.21.15.167:0/2854644403 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f570415fc80 0x7f57040b1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:35.496 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.495+0000 7f570c190640 1 --2- 172.21.15.167:0/2854644403 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f570415f360 0x7f57040af0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:35.496 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.495+0000 7f570c190640 1 --2- 172.21.15.167:0/2854644403 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f57040b5e90 0x7f570415d500 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:35.496 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.495+0000 7f570c190640 1 -- 172.21.15.167:0/2854644403 >> 172.21.15.167:0/2854644403 conn(0x7f5704072730 msgr2=0x7f5704160ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:35.496 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.495+0000 7f570c190640 1 -- 172.21.15.167:0/2854644403 shutdown_connections 2024-05-11T08:06:35.496 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.495+0000 7f570c190640 1 -- 172.21.15.167:0/2854644403 wait complete. 2024-05-11T08:06:35.506 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-11T08:06:35.606 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.606+0000 7f5cd5396640 1 Processor -- start 2024-05-11T08:06:35.606 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.606+0000 7f5cd5396640 1 -- start start 2024-05-11T08:06:35.608 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.607+0000 7f5cd5396640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f5cd01598d0 0x7f5cd00d2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:35.608 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.607+0000 7f5cd5396640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5cd00d0640 0x7f5cd00d0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:35.608 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.607+0000 7f5cd5396640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5cd00d1190 con 0x7f5cd00cafb0 2024-05-11T08:06:35.608 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.607+0000 7f5cd5396640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f5cd006d500 con 0x7f5cd01598d0 2024-05-11T08:06:35.608 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.607+0000 7f5cd5396640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f5cd006d7e0 con 0x7f5cd00d0640 2024-05-11T08:06:35.608 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.607+0000 7f5cce7fc640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5cd00d0640 0x7f5cd00d0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:35.608 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.607+0000 7f5cceffd640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f5cd01598d0 0x7f5cd00d2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:35.608 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.607+0000 7f5cce7fc640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5cd00d0640 0x7f5cd00d0aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.167:60610/0 (socket says 172.21.15.167:60610) 2024-05-11T08:06:35.608 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.607+0000 7f5ccf7fe640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f5cd00cafb0 0x7f5cd006dfe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.167:44558/0 (socket says 172.21.15.167:44558) 2024-05-11T08:06:35.608 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.607+0000 7f5cce7fc640 1 -- 172.21.15.167:0/1412893943 learned_addr learned my addr 172.21.15.167:0/1412893943 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:35.608 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.608+0000 7f5cce7fc640 1 -- 172.21.15.167:0/1412893943 >> v2:172.21.15.204:3300/0 conn(0x7f5cd01598d0 msgr2=0x7f5cd00d2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:35.608 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.608+0000 7f5cce7fc640 1 --2- 172.21.15.167:0/1412893943 >> v2:172.21.15.204:3300/0 conn(0x7f5cd01598d0 0x7f5cd00d2cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:35.609 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.608+0000 7f5cce7fc640 1 -- 172.21.15.167:0/1412893943 >> v1:172.21.15.204:6789/0 conn(0x7f5cd00cafb0 legacy=0x7f5cd006dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:35.609 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.608+0000 7f5cce7fc640 1 -- 172.21.15.167:0/1412893943 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cd006dac0 con 0x7f5cd00d0640 2024-05-11T08:06:35.609 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.608+0000 7f5cceffd640 1 --2- 172.21.15.167:0/1412893943 >> v2:172.21.15.204:3300/0 conn(0x7f5cd01598d0 0x7f5cd00d2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T08:06:35.610 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.608+0000 7f5cce7fc640 1 --2- 172.21.15.167:0/1412893943 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5cd00d0640 0x7f5cd00d0aa0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f5cc4009870 tx=0x7f5cc402a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a4d7029a2c7e44ca server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:35.610 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.608+0000 7f5ccdffb640 1 -- 172.21.15.167:0/1412893943 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cc4037070 con 0x7f5cd00d0640 2024-05-11T08:06:35.610 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.609+0000 7f5ccdffb640 1 -- 172.21.15.167:0/1412893943 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5cc4032440 con 0x7f5cd00d0640 2024-05-11T08:06:35.610 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.609+0000 7f5ccdffb640 1 -- 172.21.15.167:0/1412893943 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cc4032810 con 0x7f5cd00d0640 2024-05-11T08:06:35.611 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.609+0000 7f5cd5396640 1 -- 172.21.15.167:0/1412893943 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5cd00d0640 msgr2=0x7f5cd00d0aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:35.611 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.609+0000 7f5cd5396640 1 --2- 172.21.15.167:0/1412893943 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5cd00d0640 0x7f5cd00d0aa0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f5cc4009870 tx=0x7f5cc402a270 comp rx=0 tx=0).stop 2024-05-11T08:06:35.611 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.609+0000 7f5cd5396640 1 -- 172.21.15.167:0/1412893943 shutdown_connections 2024-05-11T08:06:35.611 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.609+0000 7f5cd5396640 1 --2- 172.21.15.167:0/1412893943 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5cd00d0640 0x7f5cd00d0aa0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:35.611 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.609+0000 7f5cd5396640 1 --2- 172.21.15.167:0/1412893943 >> v2:172.21.15.204:3300/0 conn(0x7f5cd01598d0 0x7f5cd00d2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:35.611 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.609+0000 7f5cd5396640 1 -- 172.21.15.167:0/1412893943 >> 172.21.15.167:0/1412893943 conn(0x7f5cd00d1ba0 msgr2=0x7f5cd00d1fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:35.611 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.609+0000 7f5cd5396640 1 -- 172.21.15.167:0/1412893943 shutdown_connections 2024-05-11T08:06:35.611 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.609+0000 7f5cd5396640 1 -- 172.21.15.167:0/1412893943 wait complete. 2024-05-11T08:06:35.611 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.610+0000 7f5cd5396640 1 Processor -- start 2024-05-11T08:06:35.611 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.610+0000 7f5cd5396640 1 -- start start 2024-05-11T08:06:35.613 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.611+0000 7f5cd5396640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5cd00cafb0 0x7f5cd0150960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:35.613 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.611+0000 7f5cd5396640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5cd00d0640 0x7f5cd00ce040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:35.613 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.611+0000 7f5cd5396640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5cd01598d0 0x7f5cd00ce580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:35.613 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.611+0000 7f5cd5396640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5cd006d1b0 con 0x7f5cd00d0640 2024-05-11T08:06:35.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.611+0000 7f5cd5396640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5cd0159cb0 con 0x7f5cd01598d0 2024-05-11T08:06:35.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.611+0000 7f5cd5396640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f5cd00cb400 con 0x7f5cd00cafb0 2024-05-11T08:06:35.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.611+0000 7f5cce7fc640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5cd00d0640 0x7f5cd00ce040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:35.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.611+0000 7f5cce7fc640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5cd00d0640 0x7f5cd00ce040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:34774/0 (socket says 172.21.15.167:34774) 2024-05-11T08:06:35.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.611+0000 7f5cce7fc640 1 -- 172.21.15.167:0/3685656109 learned_addr learned my addr 172.21.15.167:0/3685656109 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:35.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.611+0000 7f5cceffd640 1 --2- 172.21.15.167:0/3685656109 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5cd00cafb0 0x7f5cd0150960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:35.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.611+0000 7f5ccf7fe640 1 --2- 172.21.15.167:0/3685656109 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5cd01598d0 0x7f5cd00ce580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:35.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.612+0000 7f5cce7fc640 1 -- 172.21.15.167:0/3685656109 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5cd00cafb0 msgr2=0x7f5cd0150960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:35.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.612+0000 7f5cce7fc640 1 --2- 172.21.15.167:0/3685656109 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5cd00cafb0 0x7f5cd0150960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:35.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.612+0000 7f5cce7fc640 1 -- 172.21.15.167:0/3685656109 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5cd01598d0 msgr2=0x7f5cd00ce580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:35.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.612+0000 7f5cce7fc640 1 --2- 172.21.15.167:0/3685656109 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5cd01598d0 0x7f5cd00ce580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:35.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.612+0000 7f5cce7fc640 1 -- 172.21.15.167:0/3685656109 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cd00cd100 con 0x7f5cd00d0640 2024-05-11T08:06:35.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.612+0000 7f5cce7fc640 1 --2- 172.21.15.167:0/3685656109 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5cd00d0640 0x7f5cd00ce040 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f5cc4000c00 tx=0x7f5cc40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:35.615 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.612+0000 7f5cb77fe640 1 -- 172.21.15.167:0/3685656109 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cc4037040 con 0x7f5cd00d0640 2024-05-11T08:06:35.615 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.612+0000 7f5cb77fe640 1 -- 172.21.15.167:0/3685656109 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5cc4042070 con 0x7f5cd00d0640 2024-05-11T08:06:35.615 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.612+0000 7f5cb77fe640 1 -- 172.21.15.167:0/3685656109 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cc4030860 con 0x7f5cd00d0640 2024-05-11T08:06:35.615 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.612+0000 7f5cd5396640 1 -- 172.21.15.167:0/3685656109 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cd00cd3e0 con 0x7f5cd00d0640 2024-05-11T08:06:35.615 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.612+0000 7f5cd5396640 1 -- 172.21.15.167:0/3685656109 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cd0153060 con 0x7f5cd00d0640 2024-05-11T08:06:35.615 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.613+0000 7f5cd5396640 1 -- 172.21.15.167:0/3685656109 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5cd0069590 con 0x7f5cd00d0640 2024-05-11T08:06:35.615 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.614+0000 7f5cb77fe640 1 -- 172.21.15.167:0/3685656109 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f5cc4030a00 con 0x7f5cd00d0640 2024-05-11T08:06:35.617 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.614+0000 7f5cb77fe640 1 --2- 172.21.15.167:0/3685656109 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f5cbc0762d0 0x7f5cbc078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:35.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.614+0000 7f5cb77fe640 1 -- 172.21.15.167:0/3685656109 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f5cc40b8050 con 0x7f5cd00d0640 2024-05-11T08:06:35.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.614+0000 7f5cceffd640 1 --2- 172.21.15.167:0/3685656109 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f5cbc0762d0 0x7f5cbc078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:35.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.615+0000 7f5cceffd640 1 --2- 172.21.15.167:0/3685656109 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f5cbc0762d0 0x7f5cbc078790 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f5cc0009bd0 tx=0x7f5cc000b040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:35.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.619+0000 7f5cb77fe640 1 -- 172.21.15.167:0/3685656109 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f5cc4081020 con 0x7f5cd00d0640 2024-05-11T08:06:35.838 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.838+0000 7f5cd5396640 1 -- 172.21.15.167:0/3685656109 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5cd0150ea0 con 0x7f5cd00d0640 2024-05-11T08:06:35.839 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.839+0000 7f5cb77fe640 1 -- 172.21.15.167:0/3685656109 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5183 (secure 0 0 0) 0x7f5cc4085320 con 0x7f5cd00d0640 2024-05-11T08:06:35.839 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T08:06:35.839 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6834/506789040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":506789040},{"type":"v1","addr":"172.21.15.204:6834","nonce":506789040}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4287,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1448126000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":1448126000},{"type":"v1","addr":"172.21.15.167:6837","nonce":1448126000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4290,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/766834562","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":766834562},{"type":"v1","addr":"172.21.15.167:6836","nonce":766834562}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-11T08:05:54.711448+0000","modified":"2024-05-11T08:06:13.780553+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4305},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4305":{"gid":4305,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.204:6835/1247774569","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6833","nonce":1247774569},{"type":"v1","addr":"172.21.15.204:6835","nonce":1247774569}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4305,"qdb_cluster":[4305]},"id":1}]} 2024-05-11T08:06:35.839 INFO:teuthology.orchestra.run.smithi167.stderr:dumped fsmap epoch 9 2024-05-11T08:06:35.841 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.841+0000 7f5cd5396640 1 -- 172.21.15.167:0/3685656109 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f5cbc0762d0 msgr2=0x7f5cbc078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:35.841 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.841+0000 7f5cd5396640 1 --2- 172.21.15.167:0/3685656109 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f5cbc0762d0 0x7f5cbc078790 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f5cc0009bd0 tx=0x7f5cc000b040 comp rx=0 tx=0).stop 2024-05-11T08:06:35.841 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.841+0000 7f5cd5396640 1 -- 172.21.15.167:0/3685656109 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5cd00d0640 msgr2=0x7f5cd00ce040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:35.841 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.841+0000 7f5cd5396640 1 --2- 172.21.15.167:0/3685656109 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5cd00d0640 0x7f5cd00ce040 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f5cc4000c00 tx=0x7f5cc40046c0 comp rx=0 tx=0).stop 2024-05-11T08:06:35.841 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.841+0000 7f5cd5396640 1 -- 172.21.15.167:0/3685656109 shutdown_connections 2024-05-11T08:06:35.842 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.841+0000 7f5cd5396640 1 --2- 172.21.15.167:0/3685656109 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f5cbc0762d0 0x7f5cbc078790 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:35.842 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.841+0000 7f5cd5396640 1 --2- 172.21.15.167:0/3685656109 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5cd01598d0 0x7f5cd00ce580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:35.842 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.841+0000 7f5cd5396640 1 --2- 172.21.15.167:0/3685656109 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5cd00d0640 0x7f5cd00ce040 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:35.842 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.841+0000 7f5cd5396640 1 --2- 172.21.15.167:0/3685656109 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5cd00cafb0 0x7f5cd0150960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:35.842 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.841+0000 7f5cd5396640 1 -- 172.21.15.167:0/3685656109 >> 172.21.15.167:0/3685656109 conn(0x7f5cd00d1ba0 msgr2=0x7f5cd01617f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:35.842 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.841+0000 7f5cd5396640 1 -- 172.21.15.167:0/3685656109 shutdown_connections 2024-05-11T08:06:35.842 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.841+0000 7f5cd5396640 1 -- 172.21.15.167:0/3685656109 wait complete. 2024-05-11T08:06:35.852 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-11T08:06:35.955 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.955+0000 7fd51c26d640 1 Processor -- start 2024-05-11T08:06:35.955 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.955+0000 7fd51c26d640 1 -- start start 2024-05-11T08:06:35.957 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.956+0000 7fd51c26d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd51415f330 0x7fd5140b6860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:35.957 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.956+0000 7fd51c26d640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fd5140737a0 0x7fd5140d0080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:35.957 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.956+0000 7fd51c26d640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd51406c930 con 0x7fd51406c450 2024-05-11T08:06:35.957 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.956+0000 7fd51c26d640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fd51406cdd0 con 0x7fd5140737a0 2024-05-11T08:06:35.957 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.956+0000 7fd51c26d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fd51406d0b0 con 0x7fd51415f330 2024-05-11T08:06:35.957 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.956+0000 7fd51a009640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd51415f330 0x7fd5140b6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:35.957 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.956+0000 7fd51a80a640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fd5140737a0 0x7fd5140d0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:35.957 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.956+0000 7fd51a009640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd51415f330 0x7fd5140b6860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.167:60632/0 (socket says 172.21.15.167:60632) 2024-05-11T08:06:35.957 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.956+0000 7fd519808640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7fd51406c450 0x7fd514073090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.167:44562/0 (socket says 172.21.15.167:44562) 2024-05-11T08:06:35.957 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.956+0000 7fd51a009640 1 -- 172.21.15.167:0/3093093574 learned_addr learned my addr 172.21.15.167:0/3093093574 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:35.957 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.957+0000 7fd51a009640 1 -- 172.21.15.167:0/3093093574 >> v2:172.21.15.204:3300/0 conn(0x7fd5140737a0 msgr2=0x7fd5140d0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:35.957 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.957+0000 7fd51a009640 1 --2- 172.21.15.167:0/3093093574 >> v2:172.21.15.204:3300/0 conn(0x7fd5140737a0 0x7fd5140d0080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:35.958 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.957+0000 7fd51a009640 1 -- 172.21.15.167:0/3093093574 >> v1:172.21.15.204:6789/0 conn(0x7fd51406c450 legacy=0x7fd514073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:35.958 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.957+0000 7fd51a009640 1 -- 172.21.15.167:0/3093093574 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd51406d390 con 0x7fd51415f330 2024-05-11T08:06:35.959 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.957+0000 7fd51a009640 1 --2- 172.21.15.167:0/3093093574 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd51415f330 0x7fd5140b6860 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd510009870 tx=0x7fd51002a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae489b32086a326a server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:35.959 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.958+0000 7fd519007640 1 -- 172.21.15.167:0/3093093574 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd510037070 con 0x7fd51415f330 2024-05-11T08:06:35.959 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.958+0000 7fd519007640 1 -- 172.21.15.167:0/3093093574 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd510032440 con 0x7fd51415f330 2024-05-11T08:06:35.959 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.958+0000 7fd519007640 1 -- 172.21.15.167:0/3093093574 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd510032810 con 0x7fd51415f330 2024-05-11T08:06:35.959 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.958+0000 7fd51c26d640 1 -- 172.21.15.167:0/3093093574 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd51415f330 msgr2=0x7fd5140b6860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:35.959 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.958+0000 7fd51c26d640 1 --2- 172.21.15.167:0/3093093574 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd51415f330 0x7fd5140b6860 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd510009870 tx=0x7fd51002a270 comp rx=0 tx=0).stop 2024-05-11T08:06:35.959 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.958+0000 7fd51c26d640 1 -- 172.21.15.167:0/3093093574 shutdown_connections 2024-05-11T08:06:35.959 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.958+0000 7fd51c26d640 1 --2- 172.21.15.167:0/3093093574 >> v2:172.21.15.204:3300/0 conn(0x7fd5140737a0 0x7fd5140d0080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:35.959 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.958+0000 7fd51c26d640 1 --2- 172.21.15.167:0/3093093574 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd51415f330 0x7fd5140b6860 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:35.959 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.958+0000 7fd51c26d640 1 -- 172.21.15.167:0/3093093574 >> 172.21.15.167:0/3093093574 conn(0x7fd514070290 msgr2=0x7fd514070670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:35.960 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.958+0000 7fd51c26d640 1 -- 172.21.15.167:0/3093093574 shutdown_connections 2024-05-11T08:06:35.960 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.958+0000 7fd51c26d640 1 -- 172.21.15.167:0/3093093574 wait complete. 2024-05-11T08:06:35.962 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.959+0000 7fd51c26d640 1 Processor -- start 2024-05-11T08:06:35.962 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.959+0000 7fd51c26d640 1 -- start start 2024-05-11T08:06:35.962 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.960+0000 7fd51c26d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd51406c450 0x7fd5140b12b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:35.962 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.960+0000 7fd51c26d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd5140737a0 0x7fd5140aea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:35.962 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.960+0000 7fd51c26d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd51415f330 0x7fd51415a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:35.962 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.960+0000 7fd51c26d640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd514073f00 con 0x7fd5140737a0 2024-05-11T08:06:35.962 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.960+0000 7fd51c26d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd514073d80 con 0x7fd51415f330 2024-05-11T08:06:35.962 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.960+0000 7fd51c26d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fd514074740 con 0x7fd51406c450 2024-05-11T08:06:35.962 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.960+0000 7fd519808640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd5140737a0 0x7fd5140aea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:35.962 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.960+0000 7fd519808640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd5140737a0 0x7fd5140aea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:34786/0 (socket says 172.21.15.167:34786) 2024-05-11T08:06:35.962 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.960+0000 7fd519808640 1 -- 172.21.15.167:0/2636201584 learned_addr learned my addr 172.21.15.167:0/2636201584 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:35.963 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.960+0000 7fd51a009640 1 --2- 172.21.15.167:0/2636201584 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd51406c450 0x7fd5140b12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:35.963 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.961+0000 7fd51a80a640 1 --2- 172.21.15.167:0/2636201584 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd51415f330 0x7fd51415a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:35.963 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.961+0000 7fd519808640 1 -- 172.21.15.167:0/2636201584 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd51406c450 msgr2=0x7fd5140b12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:35.963 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.961+0000 7fd519808640 1 --2- 172.21.15.167:0/2636201584 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd51406c450 0x7fd5140b12b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:35.963 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.961+0000 7fd519808640 1 -- 172.21.15.167:0/2636201584 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd51415f330 msgr2=0x7fd51415a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:35.963 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.961+0000 7fd519808640 1 --2- 172.21.15.167:0/2636201584 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd51415f330 0x7fd51415a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:35.963 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.961+0000 7fd519808640 1 -- 172.21.15.167:0/2636201584 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5140b17f0 con 0x7fd5140737a0 2024-05-11T08:06:35.963 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.961+0000 7fd519808640 1 --2- 172.21.15.167:0/2636201584 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd5140737a0 0x7fd5140aea00 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd50400c4d0 tx=0x7fd50400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:35.963 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.961+0000 7fd50affd640 1 -- 172.21.15.167:0/2636201584 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd504018070 con 0x7fd5140737a0 2024-05-11T08:06:35.963 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.961+0000 7fd50affd640 1 -- 172.21.15.167:0/2636201584 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd504007480 con 0x7fd5140737a0 2024-05-11T08:06:35.963 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.961+0000 7fd51c26d640 1 -- 172.21.15.167:0/2636201584 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5140b1da0 con 0x7fd5140737a0 2024-05-11T08:06:35.963 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.961+0000 7fd50affd640 1 -- 172.21.15.167:0/2636201584 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5040049e0 con 0x7fd5140737a0 2024-05-11T08:06:35.963 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.961+0000 7fd51c26d640 1 -- 172.21.15.167:0/2636201584 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd514159da0 con 0x7fd5140737a0 2024-05-11T08:06:35.964 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.962+0000 7fd51c26d640 1 -- 172.21.15.167:0/2636201584 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd514069590 con 0x7fd5140737a0 2024-05-11T08:06:35.964 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.963+0000 7fd50affd640 1 -- 172.21.15.167:0/2636201584 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fd50401b070 con 0x7fd5140737a0 2024-05-11T08:06:35.964 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.963+0000 7fd50affd640 1 --2- 172.21.15.167:0/2636201584 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fd4f80762d0 0x7fd4f8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:35.965 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.963+0000 7fd51a009640 1 --2- 172.21.15.167:0/2636201584 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fd4f80762d0 0x7fd4f8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:35.965 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.963+0000 7fd50affd640 1 -- 172.21.15.167:0/2636201584 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fd504096b70 con 0x7fd5140737a0 2024-05-11T08:06:35.965 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.964+0000 7fd51a009640 1 --2- 172.21.15.167:0/2636201584 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fd4f80762d0 0x7fd4f8078790 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd510034000 tx=0x7fd51002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:35.967 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:35.968+0000 7fd50affd640 1 -- 172.21.15.167:0/2636201584 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fd504007670 con 0x7fd5140737a0 2024-05-11T08:06:36.150 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.150+0000 7fd51c26d640 1 -- 172.21.15.167:0/2636201584 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd51415cce0 con 0x7fd5140737a0 2024-05-11T08:06:36.151 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.151+0000 7fd50affd640 1 -- 172.21.15.167:0/2636201584 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18562 (secure 0 0 0) 0x7fd504068210 con 0x7fd5140737a0 2024-05-11T08:06:36.151 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T08:06:36.152 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":27,"fsid":"2034778d-8ff1-4823-aa44-ded32b518241","created":"2024-05-11T08:05:31.612421+0000","modified":"2024-05-11T08:05:59.756806+0000","last_up_change":"2024-05-11T08:05:42.557382+0000","last_in_change":"2024-05-11T08:05:33.757311+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-11T08:05:43.931263+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-11T08:05:52.573726+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-11T08:05:53.670746+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-11T08:05:57.380272+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"587eaced-5ded-406d-884b-30c1caa8686b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":3863307356},{"type":"v1","addr":"172.21.15.167:6806","nonce":3863307356}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":3863307356},{"type":"v1","addr":"172.21.15.167:6810","nonce":3863307356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":3863307356},{"type":"v1","addr":"172.21.15.167:6815","nonce":3863307356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":3863307356},{"type":"v1","addr":"172.21.15.167:6813","nonce":3863307356}]},"public_addr":"172.21.15.167:6806/3863307356","cluster_addr":"172.21.15.167:6810/3863307356","heartbeat_back_addr":"172.21.15.167:6815/3863307356","heartbeat_front_addr":"172.21.15.167:6813/3863307356","state":["exists","up"]},{"osd":1,"uuid":"37c59104-2dd0-4cf9-85ea-56636f6b02a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":1585024048},{"type":"v1","addr":"172.21.15.167:6817","nonce":1585024048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1585024048},{"type":"v1","addr":"172.21.15.167:6819","nonce":1585024048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1585024048},{"type":"v1","addr":"172.21.15.167:6823","nonce":1585024048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":1585024048},{"type":"v1","addr":"172.21.15.167:6821","nonce":1585024048}]},"public_addr":"172.21.15.167:6817/1585024048","cluster_addr":"172.21.15.167:6819/1585024048","heartbeat_back_addr":"172.21.15.167:6823/1585024048","heartbeat_front_addr":"172.21.15.167:6821/1585024048","state":["exists","up"]},{"osd":2,"uuid":"2bb4f7f5-c698-451b-b4bb-78e716921291","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":3242166900},{"type":"v1","addr":"172.21.15.167:6801","nonce":3242166900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":3242166900},{"type":"v1","addr":"172.21.15.167:6803","nonce":3242166900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6809","nonce":3242166900},{"type":"v1","addr":"172.21.15.167:6811","nonce":3242166900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6805","nonce":3242166900},{"type":"v1","addr":"172.21.15.167:6807","nonce":3242166900}]},"public_addr":"172.21.15.167:6801/3242166900","cluster_addr":"172.21.15.167:6803/3242166900","heartbeat_back_addr":"172.21.15.167:6811/3242166900","heartbeat_front_addr":"172.21.15.167:6807/3242166900","state":["exists","up"]},{"osd":3,"uuid":"9dd26759-a338-49d2-8103-45d068b9c000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":453689293},{"type":"v1","addr":"172.21.15.167:6825","nonce":453689293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":453689293},{"type":"v1","addr":"172.21.15.167:6827","nonce":453689293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":453689293},{"type":"v1","addr":"172.21.15.167:6831","nonce":453689293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":453689293},{"type":"v1","addr":"172.21.15.167:6829","nonce":453689293}]},"public_addr":"172.21.15.167:6825/453689293","cluster_addr":"172.21.15.167:6827/453689293","heartbeat_back_addr":"172.21.15.167:6831/453689293","heartbeat_front_addr":"172.21.15.167:6829/453689293","state":["exists","up"]},{"osd":4,"uuid":"cb27250d-2a80-4c7b-8cef-3a7af05c8a16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1311831500},{"type":"v1","addr":"172.21.15.204:6801","nonce":1311831500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1311831500},{"type":"v1","addr":"172.21.15.204:6803","nonce":1311831500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1311831500},{"type":"v1","addr":"172.21.15.204:6807","nonce":1311831500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1311831500},{"type":"v1","addr":"172.21.15.204:6805","nonce":1311831500}]},"public_addr":"172.21.15.204:6801/1311831500","cluster_addr":"172.21.15.204:6803/1311831500","heartbeat_back_addr":"172.21.15.204:6807/1311831500","heartbeat_front_addr":"172.21.15.204:6805/1311831500","state":["exists","up"]},{"osd":5,"uuid":"92998541-ffec-430b-ab76-8c3cb7c2752d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2921847521},{"type":"v1","addr":"172.21.15.204:6825","nonce":2921847521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2921847521},{"type":"v1","addr":"172.21.15.204:6827","nonce":2921847521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2921847521},{"type":"v1","addr":"172.21.15.204:6831","nonce":2921847521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2921847521},{"type":"v1","addr":"172.21.15.204:6829","nonce":2921847521}]},"public_addr":"172.21.15.204:6825/2921847521","cluster_addr":"172.21.15.204:6827/2921847521","heartbeat_back_addr":"172.21.15.204:6831/2921847521","heartbeat_front_addr":"172.21.15.204:6829/2921847521","state":["exists","up"]},{"osd":6,"uuid":"9543479c-2ec5-42a8-8933-707bfed2e048","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":2889851077},{"type":"v1","addr":"172.21.15.204:6817","nonce":2889851077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":2889851077},{"type":"v1","addr":"172.21.15.204:6819","nonce":2889851077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":2889851077},{"type":"v1","addr":"172.21.15.204:6823","nonce":2889851077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":2889851077},{"type":"v1","addr":"172.21.15.204:6821","nonce":2889851077}]},"public_addr":"172.21.15.204:6817/2889851077","cluster_addr":"172.21.15.204:6819/2889851077","heartbeat_back_addr":"172.21.15.204:6823/2889851077","heartbeat_front_addr":"172.21.15.204:6821/2889851077","state":["exists","up"]},{"osd":7,"uuid":"c45a5647-8358-409e-af91-c05dda65f0b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":1949534184},{"type":"v1","addr":"172.21.15.204:6809","nonce":1949534184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":1949534184},{"type":"v1","addr":"172.21.15.204:6811","nonce":1949534184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1949534184},{"type":"v1","addr":"172.21.15.204:6815","nonce":1949534184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1949534184},{"type":"v1","addr":"172.21.15.204:6813","nonce":1949534184}]},"public_addr":"172.21.15.204:6809/1949534184","cluster_addr":"172.21.15.204:6811/1949534184","heartbeat_back_addr":"172.21.15.204:6815/1949534184","heartbeat_front_addr":"172.21.15.204:6813/1949534184","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.952253+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.940689+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.860682+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.929350+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.901627+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.957451+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.987249+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.951197+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-11T08:06:36.153 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.153+0000 7fd51c26d640 1 -- 172.21.15.167:0/2636201584 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fd4f80762d0 msgr2=0x7fd4f8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:36.154 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.153+0000 7fd51c26d640 1 --2- 172.21.15.167:0/2636201584 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fd4f80762d0 0x7fd4f8078790 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd510034000 tx=0x7fd51002e000 comp rx=0 tx=0).stop 2024-05-11T08:06:36.154 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.153+0000 7fd51c26d640 1 -- 172.21.15.167:0/2636201584 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd5140737a0 msgr2=0x7fd5140aea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:36.154 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.153+0000 7fd51c26d640 1 --2- 172.21.15.167:0/2636201584 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd5140737a0 0x7fd5140aea00 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd50400c4d0 tx=0x7fd50400c990 comp rx=0 tx=0).stop 2024-05-11T08:06:36.154 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.153+0000 7fd51c26d640 1 -- 172.21.15.167:0/2636201584 shutdown_connections 2024-05-11T08:06:36.154 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.153+0000 7fd51c26d640 1 --2- 172.21.15.167:0/2636201584 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fd4f80762d0 0x7fd4f8078790 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:36.154 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.153+0000 7fd51c26d640 1 --2- 172.21.15.167:0/2636201584 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd51415f330 0x7fd51415a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:36.154 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.153+0000 7fd51c26d640 1 --2- 172.21.15.167:0/2636201584 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd5140737a0 0x7fd5140aea00 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:36.154 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.153+0000 7fd51c26d640 1 --2- 172.21.15.167:0/2636201584 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd51406c450 0x7fd5140b12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:36.154 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.153+0000 7fd51c26d640 1 -- 172.21.15.167:0/2636201584 >> 172.21.15.167:0/2636201584 conn(0x7fd514070290 msgr2=0x7fd5140b2260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:36.154 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.153+0000 7fd51c26d640 1 -- 172.21.15.167:0/2636201584 shutdown_connections 2024-05-11T08:06:36.154 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.153+0000 7fd51c26d640 1 -- 172.21.15.167:0/2636201584 wait complete. 2024-05-11T08:06:36.165 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-11T08:06:36.165 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-11T08:06:36.266 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.266+0000 7f9ec8f04640 1 Processor -- start 2024-05-11T08:06:36.267 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.266+0000 7f9ec8f04640 1 -- start start 2024-05-11T08:06:36.268 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.267+0000 7f9ec8f04640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f9ec41598d0 0x7f9ec40d2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:36.268 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.267+0000 7f9ec8f04640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9ec40cb6c0 0x7f9ec40d0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:36.269 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.267+0000 7f9ec8f04640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ec40d1190 con 0x7f9ec40d0640 2024-05-11T08:06:36.269 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.267+0000 7f9ec8f04640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f9ec406d500 con 0x7f9ec41598d0 2024-05-11T08:06:36.269 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.267+0000 7f9ec8f04640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f9ec406d7e0 con 0x7f9ec40cb6c0 2024-05-11T08:06:36.269 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.267+0000 7f9eb9d9b640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f9ec40d0640 0x7f9ec40cafb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:49580/0 (socket says 172.21.15.167:49580) 2024-05-11T08:06:36.269 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.267+0000 7f9eb9d9b640 1 -- 172.21.15.167:0/2183024369 learned_addr learned my addr 172.21.15.167:0/2183024369 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T08:06:36.269 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.267+0000 7f9ec2d9d640 1 --2- 172.21.15.167:0/2183024369 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9ec40cb6c0 0x7f9ec40d0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:36.269 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.267+0000 7f9ec259c640 1 --2- 172.21.15.167:0/2183024369 >> v2:172.21.15.204:3300/0 conn(0x7f9ec41598d0 0x7f9ec40d2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:36.269 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.267+0000 7f9ec1d9b640 1 -- 172.21.15.167:0/2183024369 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 217122744 0 0) 0x7f9ec40d1190 con 0x7f9ec40d0640 2024-05-11T08:06:36.269 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.267+0000 7f9ec1d9b640 1 -- 172.21.15.167:0/2183024369 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9eb0003520 con 0x7f9ec40d0640 2024-05-11T08:06:36.269 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.268+0000 7f9ec2d9d640 1 -- 172.21.15.167:0/2183024369 >> v2:172.21.15.204:3300/0 conn(0x7f9ec41598d0 msgr2=0x7f9ec40d2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:36.269 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.268+0000 7f9ec2d9d640 1 --2- 172.21.15.167:0/2183024369 >> v2:172.21.15.204:3300/0 conn(0x7f9ec41598d0 0x7f9ec40d2cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:36.269 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.268+0000 7f9ec2d9d640 1 -- 172.21.15.167:0/2183024369 >> v1:172.21.15.167:6789/0 conn(0x7f9ec40d0640 legacy=0x7f9ec40cafb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:36.269 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.268+0000 7f9ec1d9b640 1 -- 172.21.15.167:0/2183024369 <== mon.0 v1:172.21.15.167:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3617776477 0 0) 0x7f9eb0003520 con 0x7f9ec40d0640 2024-05-11T08:06:36.270 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.268+0000 7f9ec2d9d640 1 -- 172.21.15.167:0/2183024369 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ec406dac0 con 0x7f9ec40cb6c0 2024-05-11T08:06:36.270 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.268+0000 7f9ec2d9d640 1 --2- 172.21.15.167:0/2183024369 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9ec40cb6c0 0x7f9ec40d0ad0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f9ebc009d90 tx=0x7f9ebc02a5f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e00d62439d2aa9f8 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:36.270 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.269+0000 7f9ec1d9b640 1 -- 172.21.15.167:0/2183024369 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eac0016f0 con 0x7f9ec40cb6c0 2024-05-11T08:06:36.270 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.269+0000 7f9ec1d9b640 1 -- 172.21.15.167:0/2183024369 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ebc007540 con 0x7f9ec40cb6c0 2024-05-11T08:06:36.270 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.269+0000 7f9ec1d9b640 1 -- 172.21.15.167:0/2183024369 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ebc0078f0 con 0x7f9ec40cb6c0 2024-05-11T08:06:36.273 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.269+0000 7f9ec8f04640 1 -- 172.21.15.167:0/2183024369 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9ec40cb6c0 msgr2=0x7f9ec40d0ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:36.273 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.269+0000 7f9ec8f04640 1 --2- 172.21.15.167:0/2183024369 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9ec40cb6c0 0x7f9ec40d0ad0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f9ebc009d90 tx=0x7f9ebc02a5f0 comp rx=0 tx=0).stop 2024-05-11T08:06:36.273 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.269+0000 7f9ec8f04640 1 -- 172.21.15.167:0/2183024369 shutdown_connections 2024-05-11T08:06:36.273 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.269+0000 7f9ec8f04640 1 --2- 172.21.15.167:0/2183024369 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9ec40cb6c0 0x7f9ec40d0ad0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:36.273 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.269+0000 7f9ec8f04640 1 --2- 172.21.15.167:0/2183024369 >> v2:172.21.15.204:3300/0 conn(0x7f9ec41598d0 0x7f9ec40d2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:36.273 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.269+0000 7f9ec8f04640 1 -- 172.21.15.167:0/2183024369 >> 172.21.15.167:0/2183024369 conn(0x7f9ec40d1ba0 msgr2=0x7f9ec40d1fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:36.273 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.269+0000 7f9ec8f04640 1 -- 172.21.15.167:0/2183024369 shutdown_connections 2024-05-11T08:06:36.273 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.269+0000 7f9ec8f04640 1 -- 172.21.15.167:0/2183024369 wait complete. 2024-05-11T08:06:36.273 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.271+0000 7f9ec8f04640 1 Processor -- start 2024-05-11T08:06:36.273 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.271+0000 7f9ec8f04640 1 -- start start 2024-05-11T08:06:36.273 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.271+0000 7f9ec8f04640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9ec40cb6c0 0x7f9ec4145a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:36.273 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.271+0000 7f9ec8f04640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9ec40d0640 0x7f9ec40ce650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:36.273 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.271+0000 7f9ec8f04640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ec41598d0 0x7f9ec40ceb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:36.274 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.271+0000 7f9ec8f04640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9ec406d1b0 con 0x7f9ec41598d0 2024-05-11T08:06:36.274 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.271+0000 7f9ec8f04640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f9ec4159cb0 con 0x7f9ec40d0640 2024-05-11T08:06:36.274 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.271+0000 7f9ec8f04640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f9ec406dfe0 con 0x7f9ec40cb6c0 2024-05-11T08:06:36.274 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.272+0000 7f9ec2d9d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ec41598d0 0x7f9ec40ceb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:36.274 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.272+0000 7f9ec2d9d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ec41598d0 0x7f9ec40ceb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:34788/0 (socket says 172.21.15.167:34788) 2024-05-11T08:06:36.274 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.272+0000 7f9ec2d9d640 1 -- 172.21.15.167:0/877498017 learned_addr learned my addr 172.21.15.167:0/877498017 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:36.274 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.272+0000 7f9ec259c640 1 --2- 172.21.15.167:0/877498017 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9ec40cb6c0 0x7f9ec4145a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:36.274 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.272+0000 7f9eb9d9b640 1 --2- 172.21.15.167:0/877498017 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9ec40d0640 0x7f9ec40ce650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:36.274 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.272+0000 7f9ec2d9d640 1 -- 172.21.15.167:0/877498017 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9ec40cb6c0 msgr2=0x7f9ec4145a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:36.274 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.272+0000 7f9ec2d9d640 1 --2- 172.21.15.167:0/877498017 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9ec40cb6c0 0x7f9ec4145a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:36.274 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.272+0000 7f9ec2d9d640 1 -- 172.21.15.167:0/877498017 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9ec40d0640 msgr2=0x7f9ec40ce650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:36.274 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.272+0000 7f9ec2d9d640 1 --2- 172.21.15.167:0/877498017 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9ec40d0640 0x7f9ec40ce650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:36.274 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.272+0000 7f9ec2d9d640 1 -- 172.21.15.167:0/877498017 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ec40cce40 con 0x7f9ec41598d0 2024-05-11T08:06:36.274 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.272+0000 7f9ec259c640 1 --2- 172.21.15.167:0/877498017 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9ec40cb6c0 0x7f9ec4145a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T08:06:36.275 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.272+0000 7f9ec2d9d640 1 --2- 172.21.15.167:0/877498017 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ec41598d0 0x7f9ec40ceb90 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f9ebc009eb0 tx=0x7f9ebc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:36.275 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.272+0000 7f9ebb7fe640 1 -- 172.21.15.167:0/877498017 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ebc040070 con 0x7f9ec41598d0 2024-05-11T08:06:36.275 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.272+0000 7f9ebb7fe640 1 -- 172.21.15.167:0/877498017 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ebc031c40 con 0x7f9ec41598d0 2024-05-11T08:06:36.275 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.272+0000 7f9ec8f04640 1 -- 172.21.15.167:0/877498017 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ec40cd120 con 0x7f9ec41598d0 2024-05-11T08:06:36.275 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.272+0000 7f9ebb7fe640 1 -- 172.21.15.167:0/877498017 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ebc009450 con 0x7f9ec41598d0 2024-05-11T08:06:36.275 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.272+0000 7f9ec8f04640 1 -- 172.21.15.167:0/877498017 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ec40cd6e0 con 0x7f9ec41598d0 2024-05-11T08:06:36.276 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.274+0000 7f9ec8f04640 1 -- 172.21.15.167:0/877498017 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ec4069590 con 0x7f9ec41598d0 2024-05-11T08:06:36.276 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.274+0000 7f9ebb7fe640 1 -- 172.21.15.167:0/877498017 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f9ebc039040 con 0x7f9ec41598d0 2024-05-11T08:06:36.276 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.275+0000 7f9ebb7fe640 1 --2- 172.21.15.167:0/877498017 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f9ea4076300 0x7f9ea40787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:36.276 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.275+0000 7f9ebb7fe640 1 -- 172.21.15.167:0/877498017 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f9ebc0b8be0 con 0x7f9ec41598d0 2024-05-11T08:06:36.276 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.275+0000 7f9ec259c640 1 --2- 172.21.15.167:0/877498017 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f9ea4076300 0x7f9ea40787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:36.276 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.275+0000 7f9ec259c640 1 --2- 172.21.15.167:0/877498017 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f9ea4076300 0x7f9ea40787c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f9eb4009e90 tx=0x7f9eb4009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:36.279 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.279+0000 7f9ebb7fe640 1 -- 172.21.15.167:0/877498017 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f9ebc081b20 con 0x7f9ec41598d0 2024-05-11T08:06:36.509 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.509+0000 7f9ec8f04640 1 -- 172.21.15.167:0/877498017 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9ec40cbfb0 con 0x7f9ec41598d0 2024-05-11T08:06:36.510 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.510+0000 7f9ebb7fe640 1 -- 172.21.15.167:0/877498017 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5183 (secure 0 0 0) 0x7f9ebc085e20 con 0x7f9ec41598d0 2024-05-11T08:06:36.510 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T08:06:36.510 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6834/506789040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":506789040},{"type":"v1","addr":"172.21.15.204:6834","nonce":506789040}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4287,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1448126000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":1448126000},{"type":"v1","addr":"172.21.15.167:6837","nonce":1448126000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4290,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/766834562","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":766834562},{"type":"v1","addr":"172.21.15.167:6836","nonce":766834562}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-11T08:05:54.711448+0000","modified":"2024-05-11T08:06:13.780553+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4305},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4305":{"gid":4305,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.204:6835/1247774569","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6833","nonce":1247774569},{"type":"v1","addr":"172.21.15.204:6835","nonce":1247774569}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4305,"qdb_cluster":[4305]},"id":1}]} 2024-05-11T08:06:36.510 INFO:teuthology.orchestra.run.smithi167.stderr:dumped fsmap epoch 9 2024-05-11T08:06:36.513 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.512+0000 7f9ec8f04640 1 -- 172.21.15.167:0/877498017 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f9ea4076300 msgr2=0x7f9ea40787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:36.513 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.512+0000 7f9ec8f04640 1 --2- 172.21.15.167:0/877498017 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f9ea4076300 0x7f9ea40787c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f9eb4009e90 tx=0x7f9eb4009190 comp rx=0 tx=0).stop 2024-05-11T08:06:36.513 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.512+0000 7f9ec8f04640 1 -- 172.21.15.167:0/877498017 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ec41598d0 msgr2=0x7f9ec40ceb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:36.513 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.512+0000 7f9ec8f04640 1 --2- 172.21.15.167:0/877498017 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ec41598d0 0x7f9ec40ceb90 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f9ebc009eb0 tx=0x7f9ebc004a40 comp rx=0 tx=0).stop 2024-05-11T08:06:36.513 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.512+0000 7f9ec8f04640 1 -- 172.21.15.167:0/877498017 shutdown_connections 2024-05-11T08:06:36.513 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.512+0000 7f9ec8f04640 1 --2- 172.21.15.167:0/877498017 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f9ea4076300 0x7f9ea40787c0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:36.513 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.512+0000 7f9ec8f04640 1 --2- 172.21.15.167:0/877498017 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ec41598d0 0x7f9ec40ceb90 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:36.513 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.512+0000 7f9ec8f04640 1 --2- 172.21.15.167:0/877498017 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9ec40d0640 0x7f9ec40ce650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:36.513 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.512+0000 7f9ec8f04640 1 --2- 172.21.15.167:0/877498017 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9ec40cb6c0 0x7f9ec4145a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:36.514 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.513+0000 7f9ec8f04640 1 -- 172.21.15.167:0/877498017 >> 172.21.15.167:0/877498017 conn(0x7f9ec40d1ba0 msgr2=0x7f9ec40d1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:36.514 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.513+0000 7f9ec8f04640 1 -- 172.21.15.167:0/877498017 shutdown_connections 2024-05-11T08:06:36.514 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.513+0000 7f9ec8f04640 1 -- 172.21.15.167:0/877498017 wait complete. 2024-05-11T08:06:36.524 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-11T08:06:36.624 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.624+0000 7fe2ba8da640 1 Processor -- start 2024-05-11T08:06:36.624 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.624+0000 7fe2ba8da640 1 -- start start 2024-05-11T08:06:36.627 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.624+0000 7fe2ba8da640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe2b40d2cf0 0x7fe2b40cafb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:36.627 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.624+0000 7fe2ba8da640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2b40d1190 con 0x7fe2b41598d0 2024-05-11T08:06:36.627 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.624+0000 7fe2ba8da640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2b406d500 con 0x7fe2b40cb4f0 2024-05-11T08:06:36.627 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.624+0000 7fe2ba8da640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fe2b40d3150 con 0x7fe2b40d2cf0 2024-05-11T08:06:36.627 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.625+0000 7fe2b3fff640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7fe2b41598d0 0x7fe2b40d0640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:49590/0 (socket says 172.21.15.167:49590) 2024-05-11T08:06:36.628 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.625+0000 7fe2b3fff640 1 -- 172.21.15.167:0/1754096826 learned_addr learned my addr 172.21.15.167:0/1754096826 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T08:06:36.628 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.625+0000 7fe2b37fe640 1 --2- 172.21.15.167:0/1754096826 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe2b40d2cf0 0x7fe2b40cafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:36.628 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.625+0000 7fe2b2ffd640 1 -- 172.21.15.167:0/1754096826 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3260762308 0 0) 0x7fe2b40d1190 con 0x7fe2b41598d0 2024-05-11T08:06:36.628 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.625+0000 7fe2b2ffd640 1 -- 172.21.15.167:0/1754096826 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe2a0003520 con 0x7fe2b41598d0 2024-05-11T08:06:36.628 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.625+0000 7fe2b2ffd640 1 -- 172.21.15.167:0/1754096826 <== mon.0 v1:172.21.15.167:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1998365599 0 0) 0x7fe2a0003520 con 0x7fe2b41598d0 2024-05-11T08:06:36.628 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.625+0000 7fe2b2ffd640 1 -- 172.21.15.167:0/1754096826 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe2b40d2cf0 msgr2=0x7fe2b40cafb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:36.628 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.625+0000 7fe2b2ffd640 1 --2- 172.21.15.167:0/1754096826 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe2b40d2cf0 0x7fe2b40cafb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:36.628 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.625+0000 7fe2b2ffd640 1 -- 172.21.15.167:0/1754096826 >> v1:172.21.15.204:6789/0 conn(0x7fe2b40cb4f0 legacy=0x7fe2b406dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:36.628 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.625+0000 7fe2b2ffd640 1 -- 172.21.15.167:0/1754096826 --> v1:172.21.15.167:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2b406d7e0 con 0x7fe2b41598d0 2024-05-11T08:06:36.628 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.625+0000 7fe2b2ffd640 1 -- 172.21.15.167:0/1754096826 <== mon.0 v1:172.21.15.167:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3602277846 0 0) 0x7fe2a40016f0 con 0x7fe2b41598d0 2024-05-11T08:06:36.628 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.625+0000 7fe2b2ffd640 1 -- 172.21.15.167:0/1754096826 >> v1:172.21.15.167:6789/0 conn(0x7fe2b41598d0 legacy=0x7fe2b40d0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:36.628 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.625+0000 7fe2b2ffd640 1 --2- 172.21.15.167:0/1754096826 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe2a00039e0 0x7fe2a0003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:36.628 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.625+0000 7fe2b2ffd640 1 --2- 172.21.15.167:0/1754096826 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe2b40d2cf0 0x7fe2a0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:36.629 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.625+0000 7fe2b2ffd640 1 --2- 172.21.15.167:0/1754096826 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe2a0004d40 0x7fe2a00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:36.629 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.625+0000 7fe2b2ffd640 1 -- 172.21.15.167:0/1754096826 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe2a000ba70 con 0x7fe2a00039e0 2024-05-11T08:06:36.629 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.625+0000 7fe2b2ffd640 1 -- 172.21.15.167:0/1754096826 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe2a000bc20 con 0x7fe2a0004d40 2024-05-11T08:06:36.629 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.625+0000 7fe2b2ffd640 1 -- 172.21.15.167:0/1754096826 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fe2a000be20 con 0x7fe2b40d2cf0 2024-05-11T08:06:36.629 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.625+0000 7fe2b3fff640 1 --2- 172.21.15.167:0/1754096826 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe2a00039e0 0x7fe2a0003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:36.629 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.625+0000 7fe2b8e77640 1 --2- 172.21.15.167:0/1754096826 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe2a0004d40 0x7fe2a00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:36.629 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.625+0000 7fe2b37fe640 1 --2- 172.21.15.167:0/1754096826 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe2b40d2cf0 0x7fe2a0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:36.629 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.626+0000 7fe2b3fff640 1 -- 172.21.15.167:0/1754096826 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe2b40d2cf0 msgr2=0x7fe2a0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:36.629 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.626+0000 7fe2b3fff640 1 --2- 172.21.15.167:0/1754096826 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe2b40d2cf0 0x7fe2a0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:36.629 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.626+0000 7fe2b3fff640 1 -- 172.21.15.167:0/1754096826 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe2a0004d40 msgr2=0x7fe2a00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:36.629 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.626+0000 7fe2b3fff640 1 --2- 172.21.15.167:0/1754096826 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe2a0004d40 0x7fe2a00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:36.629 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.626+0000 7fe2b3fff640 1 -- 172.21.15.167:0/1754096826 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2a0003520 con 0x7fe2a00039e0 2024-05-11T08:06:36.629 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.626+0000 7fe2b37fe640 1 --2- 172.21.15.167:0/1754096826 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe2b40d2cf0 0x7fe2a0004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T08:06:36.629 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.626+0000 7fe2b3fff640 1 --2- 172.21.15.167:0/1754096826 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe2a00039e0 0x7fe2a0003dc0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fe2a4007fd0 tx=0x7fe2a0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ec8d4642abf40f8 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:36.630 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.626+0000 7fe2b2ffd640 1 -- 172.21.15.167:0/1754096826 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a4007d50 con 0x7fe2a00039e0 2024-05-11T08:06:36.630 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.626+0000 7fe2b2ffd640 1 -- 172.21.15.167:0/1754096826 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2a4005ad0 con 0x7fe2a00039e0 2024-05-11T08:06:36.630 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.626+0000 7fe2b2ffd640 1 -- 172.21.15.167:0/1754096826 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a4007070 con 0x7fe2a00039e0 2024-05-11T08:06:36.630 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.627+0000 7fe2ba8da640 1 -- 172.21.15.167:0/1754096826 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe2a00039e0 msgr2=0x7fe2a0003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:36.630 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.627+0000 7fe2ba8da640 1 --2- 172.21.15.167:0/1754096826 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe2a00039e0 0x7fe2a0003dc0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fe2a4007fd0 tx=0x7fe2a0005830 comp rx=0 tx=0).stop 2024-05-11T08:06:36.630 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.627+0000 7fe2b8e77640 1 -- 172.21.15.167:0/1754096826 reap_dead start 2024-05-11T08:06:36.630 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.627+0000 7fe2ba8da640 1 -- 172.21.15.167:0/1754096826 shutdown_connections 2024-05-11T08:06:36.630 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.627+0000 7fe2ba8da640 1 -- 172.21.15.167:0/1754096826 >> 172.21.15.167:0/1754096826 conn(0x7fe2b40d1ba0 msgr2=0x7fe2b40d1fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:36.630 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.627+0000 7fe2ba8da640 1 -- 172.21.15.167:0/1754096826 shutdown_connections 2024-05-11T08:06:36.630 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.627+0000 7fe2ba8da640 1 -- 172.21.15.167:0/1754096826 wait complete. 2024-05-11T08:06:36.633 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.628+0000 7fe2ba8da640 1 Processor -- start 2024-05-11T08:06:36.633 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.628+0000 7fe2ba8da640 1 -- start start 2024-05-11T08:06:36.633 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.629+0000 7fe2ba8da640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe2a00039e0 0x7fe2b41f4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:36.633 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.629+0000 7fe2ba8da640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe2b4153520 0x7fe2b40ba700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:36.633 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.629+0000 7fe2ba8da640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe2b41539a0 0x7fe2b40bac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:36.633 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.629+0000 7fe2ba8da640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b41607c0 con 0x7fe2b4153520 2024-05-11T08:06:36.633 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.629+0000 7fe2ba8da640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b40cb940 con 0x7fe2b41539a0 2024-05-11T08:06:36.633 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.629+0000 7fe2ba8da640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fe2b4159cb0 con 0x7fe2a00039e0 2024-05-11T08:06:36.633 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.629+0000 7fe2b37fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe2b4153520 0x7fe2b40ba700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:36.633 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.629+0000 7fe2b37fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe2b4153520 0x7fe2b40ba700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:34812/0 (socket says 172.21.15.167:34812) 2024-05-11T08:06:36.633 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.629+0000 7fe2b37fe640 1 -- 172.21.15.167:0/1025720433 learned_addr learned my addr 172.21.15.167:0/1025720433 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:36.634 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.629+0000 7fe2b8e77640 1 --2- 172.21.15.167:0/1025720433 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe2b41539a0 0x7fe2b40bac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:36.634 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.629+0000 7fe2b3fff640 1 --2- 172.21.15.167:0/1025720433 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe2a00039e0 0x7fe2b41f4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:36.634 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.629+0000 7fe2b37fe640 1 -- 172.21.15.167:0/1025720433 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe2a00039e0 msgr2=0x7fe2b41f4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:36.634 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.629+0000 7fe2b37fe640 1 --2- 172.21.15.167:0/1025720433 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe2a00039e0 0x7fe2b41f4ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:36.634 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.629+0000 7fe2b37fe640 1 -- 172.21.15.167:0/1025720433 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe2b41539a0 msgr2=0x7fe2b40bac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:36.634 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.629+0000 7fe2b37fe640 1 --2- 172.21.15.167:0/1025720433 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe2b41539a0 0x7fe2b40bac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:36.634 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.629+0000 7fe2b37fe640 1 -- 172.21.15.167:0/1025720433 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2b40b7ea0 con 0x7fe2b4153520 2024-05-11T08:06:36.634 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.629+0000 7fe2b3fff640 1 --2- 172.21.15.167:0/1025720433 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe2a00039e0 0x7fe2b41f4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T08:06:36.634 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.630+0000 7fe2b37fe640 1 --2- 172.21.15.167:0/1025720433 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe2b4153520 0x7fe2b40ba700 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fe29c0116d0 tx=0x7fe29c011b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:36.634 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.630+0000 7fe2b0ff9640 1 -- 172.21.15.167:0/1025720433 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe29c00f9c0 con 0x7fe2b4153520 2024-05-11T08:06:36.634 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.630+0000 7fe2b0ff9640 1 -- 172.21.15.167:0/1025720433 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe29c00fb60 con 0x7fe2b4153520 2024-05-11T08:06:36.634 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.630+0000 7fe2ba8da640 1 -- 172.21.15.167:0/1025720433 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2b40b8140 con 0x7fe2b4153520 2024-05-11T08:06:36.634 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.630+0000 7fe2b0ff9640 1 -- 172.21.15.167:0/1025720433 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe29c015460 con 0x7fe2b4153520 2024-05-11T08:06:36.635 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.630+0000 7fe2ba8da640 1 -- 172.21.15.167:0/1025720433 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2b40b8700 con 0x7fe2b4153520 2024-05-11T08:06:36.635 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.631+0000 7fe2ba8da640 1 -- 172.21.15.167:0/1025720433 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2b4069590 con 0x7fe2b4153520 2024-05-11T08:06:36.635 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.632+0000 7fe2b0ff9640 1 -- 172.21.15.167:0/1025720433 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fe29c015600 con 0x7fe2b4153520 2024-05-11T08:06:36.635 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.632+0000 7fe2b0ff9640 1 --2- 172.21.15.167:0/1025720433 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fe2940762d0 0x7fe294078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:36.635 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.632+0000 7fe2b3fff640 1 --2- 172.21.15.167:0/1025720433 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fe2940762d0 0x7fe294078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:36.635 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.632+0000 7fe2b0ff9640 1 -- 172.21.15.167:0/1025720433 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7fe29c098320 con 0x7fe2b4153520 2024-05-11T08:06:36.635 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.632+0000 7fe2b3fff640 1 --2- 172.21.15.167:0/1025720433 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fe2940762d0 0x7fe294078790 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fe2a4032000 tx=0x7fe2a4009030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:36.636 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.636+0000 7fe2b0ff9640 1 -- 172.21.15.167:0/1025720433 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fe29c019030 con 0x7fe2b4153520 2024-05-11T08:06:36.856 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.856+0000 7fe2ba8da640 1 -- 172.21.15.167:0/1025720433 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe2b406d240 con 0x7fe2b4153520 2024-05-11T08:06:36.856 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.856+0000 7fe2b0ff9640 1 -- 172.21.15.167:0/1025720433 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5183 (secure 0 0 0) 0x7fe29c061880 con 0x7fe2b4153520 2024-05-11T08:06:36.856 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T08:06:36.857 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6834/506789040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":506789040},{"type":"v1","addr":"172.21.15.204:6834","nonce":506789040}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4287,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1448126000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":1448126000},{"type":"v1","addr":"172.21.15.167:6837","nonce":1448126000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4290,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/766834562","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":766834562},{"type":"v1","addr":"172.21.15.167:6836","nonce":766834562}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-11T08:05:54.711448+0000","modified":"2024-05-11T08:06:13.780553+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4305},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4305":{"gid":4305,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.204:6835/1247774569","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6833","nonce":1247774569},{"type":"v1","addr":"172.21.15.204:6835","nonce":1247774569}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4305,"qdb_cluster":[4305]},"id":1}]} 2024-05-11T08:06:36.857 INFO:teuthology.orchestra.run.smithi167.stderr:dumped fsmap epoch 9 2024-05-11T08:06:36.859 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.858+0000 7fe2ba8da640 1 -- 172.21.15.167:0/1025720433 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fe2940762d0 msgr2=0x7fe294078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:36.859 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.858+0000 7fe2ba8da640 1 --2- 172.21.15.167:0/1025720433 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fe2940762d0 0x7fe294078790 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fe2a4032000 tx=0x7fe2a4009030 comp rx=0 tx=0).stop 2024-05-11T08:06:36.859 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.858+0000 7fe2ba8da640 1 -- 172.21.15.167:0/1025720433 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe2b4153520 msgr2=0x7fe2b40ba700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:36.859 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.858+0000 7fe2ba8da640 1 --2- 172.21.15.167:0/1025720433 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe2b4153520 0x7fe2b40ba700 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fe29c0116d0 tx=0x7fe29c011b90 comp rx=0 tx=0).stop 2024-05-11T08:06:36.859 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.858+0000 7fe2ba8da640 1 -- 172.21.15.167:0/1025720433 shutdown_connections 2024-05-11T08:06:36.859 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.858+0000 7fe2ba8da640 1 --2- 172.21.15.167:0/1025720433 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fe2940762d0 0x7fe294078790 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:36.859 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.858+0000 7fe2ba8da640 1 --2- 172.21.15.167:0/1025720433 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe2b41539a0 0x7fe2b40bac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:36.859 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.858+0000 7fe2ba8da640 1 --2- 172.21.15.167:0/1025720433 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe2b4153520 0x7fe2b40ba700 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:36.859 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.858+0000 7fe2ba8da640 1 --2- 172.21.15.167:0/1025720433 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe2a00039e0 0x7fe2b41f4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:36.859 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.858+0000 7fe2ba8da640 1 -- 172.21.15.167:0/1025720433 >> 172.21.15.167:0/1025720433 conn(0x7fe2b40d1ba0 msgr2=0x7fe2b41577d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:36.859 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.858+0000 7fe2ba8da640 1 -- 172.21.15.167:0/1025720433 shutdown_connections 2024-05-11T08:06:36.859 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.858+0000 7fe2ba8da640 1 -- 172.21.15.167:0/1025720433 wait complete. 2024-05-11T08:06:36.870 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-11T08:06:36.970 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.970+0000 7fa571e6d640 1 Processor -- start 2024-05-11T08:06:36.970 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.970+0000 7fa571e6d640 1 -- start start 2024-05-11T08:06:36.973 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.971+0000 7fa571e6d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa56c1598d0 0x7fa56c0d2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:36.973 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.971+0000 7fa571e6d640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fa56c0cb6c0 0x7fa56c0d0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:36.973 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.971+0000 7fa571e6d640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa56c0d1190 con 0x7fa56c0d0640 2024-05-11T08:06:36.973 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.971+0000 7fa571e6d640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7fa56c06d500 con 0x7fa56c0cb6c0 2024-05-11T08:06:36.973 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.971+0000 7fa571e6d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fa56c06d7e0 con 0x7fa56c1598d0 2024-05-11T08:06:36.973 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.971+0000 7fa56bfff640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fa56c0cb6c0 0x7fa56c0d0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:36.973 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.971+0000 7fa56affd640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7fa56c0d0640 0x7fa56c0cafb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:49592/0 (socket says 172.21.15.167:49592) 2024-05-11T08:06:36.973 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.971+0000 7fa56bfff640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fa56c0cb6c0 0x7fa56c0d0aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:34816/0 (socket says 172.21.15.167:34816) 2024-05-11T08:06:36.973 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.971+0000 7fa56affd640 1 -- 172.21.15.167:0/2187668832 learned_addr learned my addr 172.21.15.167:0/2187668832 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T08:06:36.973 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.971+0000 7fa56b7fe640 1 --2- 172.21.15.167:0/2187668832 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa56c1598d0 0x7fa56c0d2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:36.974 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.971+0000 7fa56bfff640 1 -- 172.21.15.167:0/2187668832 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa56c1598d0 msgr2=0x7fa56c0d2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:36.974 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.971+0000 7fa56bfff640 1 --2- 172.21.15.167:0/2187668832 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa56c1598d0 0x7fa56c0d2cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:36.974 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.971+0000 7fa56bfff640 1 -- 172.21.15.167:0/2187668832 >> v1:172.21.15.167:6789/0 conn(0x7fa56c0d0640 legacy=0x7fa56c0cafb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:36.974 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.971+0000 7fa56a7fc640 1 -- 172.21.15.167:0/2187668832 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1526079720 0 0) 0x7fa56c0d1190 con 0x7fa56c0d0640 2024-05-11T08:06:36.974 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.971+0000 7fa56bfff640 1 -- 172.21.15.167:0/2187668832 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa56c06dac0 con 0x7fa56c0cb6c0 2024-05-11T08:06:36.974 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.972+0000 7fa56bfff640 1 --2- 172.21.15.167:0/2187668832 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa56c0cb6c0 0x7fa56c0d0aa0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fa564009d90 tx=0x7fa56402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a50d9933ab61e41 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:36.974 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.972+0000 7fa56a7fc640 1 -- 172.21.15.167:0/2187668832 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa564007740 con 0x7fa56c0cb6c0 2024-05-11T08:06:36.974 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.972+0000 7fa56a7fc640 1 -- 172.21.15.167:0/2187668832 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5640078e0 con 0x7fa56c0cb6c0 2024-05-11T08:06:36.974 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.972+0000 7fa56a7fc640 1 -- 172.21.15.167:0/2187668832 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa564007cb0 con 0x7fa56c0cb6c0 2024-05-11T08:06:36.974 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.973+0000 7fa571e6d640 1 -- 172.21.15.167:0/2187668832 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa56c0cb6c0 msgr2=0x7fa56c0d0aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:36.974 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.973+0000 7fa571e6d640 1 --2- 172.21.15.167:0/2187668832 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa56c0cb6c0 0x7fa56c0d0aa0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fa564009d90 tx=0x7fa56402a660 comp rx=0 tx=0).stop 2024-05-11T08:06:36.974 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.973+0000 7fa571e6d640 1 -- 172.21.15.167:0/2187668832 shutdown_connections 2024-05-11T08:06:36.974 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.973+0000 7fa571e6d640 1 --2- 172.21.15.167:0/2187668832 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa56c0cb6c0 0x7fa56c0d0aa0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:36.975 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.973+0000 7fa571e6d640 1 --2- 172.21.15.167:0/2187668832 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa56c1598d0 0x7fa56c0d2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:36.975 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.973+0000 7fa571e6d640 1 -- 172.21.15.167:0/2187668832 >> 172.21.15.167:0/2187668832 conn(0x7fa56c0d1ba0 msgr2=0x7fa56c0d1fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:36.975 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.973+0000 7fa571e6d640 1 -- 172.21.15.167:0/2187668832 shutdown_connections 2024-05-11T08:06:36.975 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.973+0000 7fa571e6d640 1 -- 172.21.15.167:0/2187668832 wait complete. 2024-05-11T08:06:36.977 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.974+0000 7fa571e6d640 1 Processor -- start 2024-05-11T08:06:36.978 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.974+0000 7fa571e6d640 1 -- start start 2024-05-11T08:06:36.978 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.975+0000 7fa571e6d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa56c0cb6c0 0x7fa56c150f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:36.978 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.975+0000 7fa571e6d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa56c0d0640 0x7fa56c0b7e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:36.978 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.975+0000 7fa571e6d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa56c1598d0 0x7fa56c153080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:36.978 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.975+0000 7fa571e6d640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa56c06d1b0 con 0x7fa56c1598d0 2024-05-11T08:06:36.978 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.975+0000 7fa571e6d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa56c159cb0 con 0x7fa56c0d0640 2024-05-11T08:06:36.978 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.975+0000 7fa571e6d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fa56c06dfe0 con 0x7fa56c0cb6c0 2024-05-11T08:06:36.978 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.975+0000 7fa56bfff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa56c1598d0 0x7fa56c153080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:36.978 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.975+0000 7fa56bfff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa56c1598d0 0x7fa56c153080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:34826/0 (socket says 172.21.15.167:34826) 2024-05-11T08:06:36.978 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.975+0000 7fa56bfff640 1 -- 172.21.15.167:0/746751946 learned_addr learned my addr 172.21.15.167:0/746751946 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:36.978 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.975+0000 7fa56b7fe640 1 --2- 172.21.15.167:0/746751946 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa56c0cb6c0 0x7fa56c150f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:36.978 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.975+0000 7fa56affd640 1 --2- 172.21.15.167:0/746751946 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa56c0d0640 0x7fa56c0b7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:36.979 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.975+0000 7fa56bfff640 1 -- 172.21.15.167:0/746751946 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa56c0cb6c0 msgr2=0x7fa56c150f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:36.979 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.975+0000 7fa56bfff640 1 --2- 172.21.15.167:0/746751946 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa56c0cb6c0 0x7fa56c150f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:36.979 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.975+0000 7fa56bfff640 1 -- 172.21.15.167:0/746751946 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa56c0d0640 msgr2=0x7fa56c0b7e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:36.979 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.975+0000 7fa56bfff640 1 --2- 172.21.15.167:0/746751946 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa56c0d0640 0x7fa56c0b7e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:36.979 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.975+0000 7fa56bfff640 1 -- 172.21.15.167:0/746751946 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa56c1514b0 con 0x7fa56c1598d0 2024-05-11T08:06:36.979 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.975+0000 7fa56b7fe640 1 --2- 172.21.15.167:0/746751946 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa56c0cb6c0 0x7fa56c150f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T08:06:36.979 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.975+0000 7fa56bfff640 1 --2- 172.21.15.167:0/746751946 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa56c1598d0 0x7fa56c153080 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fa564009eb0 tx=0x7fa564004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:36.979 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.975+0000 7fa553fff640 1 -- 172.21.15.167:0/746751946 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa564040070 con 0x7fa56c1598d0 2024-05-11T08:06:36.979 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.975+0000 7fa553fff640 1 -- 172.21.15.167:0/746751946 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa564031c40 con 0x7fa56c1598d0 2024-05-11T08:06:36.979 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.975+0000 7fa571e6d640 1 -- 172.21.15.167:0/746751946 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa56c1f5970 con 0x7fa56c1598d0 2024-05-11T08:06:36.979 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.975+0000 7fa553fff640 1 -- 172.21.15.167:0/746751946 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa564009450 con 0x7fa56c1598d0 2024-05-11T08:06:36.979 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.975+0000 7fa571e6d640 1 -- 172.21.15.167:0/746751946 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa56c151a30 con 0x7fa56c1598d0 2024-05-11T08:06:36.979 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.977+0000 7fa571e6d640 1 -- 172.21.15.167:0/746751946 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa56c069590 con 0x7fa56c1598d0 2024-05-11T08:06:36.980 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.977+0000 7fa553fff640 1 -- 172.21.15.167:0/746751946 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fa564039040 con 0x7fa56c1598d0 2024-05-11T08:06:36.980 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.978+0000 7fa553fff640 1 --2- 172.21.15.167:0/746751946 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fa5580762d0 0x7fa558078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:36.980 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.978+0000 7fa56b7fe640 1 --2- 172.21.15.167:0/746751946 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fa5580762d0 0x7fa558078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:36.980 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.978+0000 7fa553fff640 1 -- 172.21.15.167:0/746751946 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7fa5640b87f0 con 0x7fa56c1598d0 2024-05-11T08:06:36.980 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.978+0000 7fa56b7fe640 1 --2- 172.21.15.167:0/746751946 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fa5580762d0 0x7fa558078790 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa56c0b9fa0 tx=0x7fa55c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:36.982 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:36.982+0000 7fa553fff640 1 -- 172.21.15.167:0/746751946 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fa564081de0 con 0x7fa56c1598d0 2024-05-11T08:06:37.167 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.167+0000 7fa571e6d640 1 -- 172.21.15.167:0/746751946 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa56c0cbfb0 con 0x7fa56c1598d0 2024-05-11T08:06:37.168 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.168+0000 7fa553fff640 1 -- 172.21.15.167:0/746751946 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) ==== 74+0+18951 (secure 0 0 0) 0x7fa5640816e0 con 0x7fa56c1598d0 2024-05-11T08:06:37.168 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T08:06:37.168 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":28,"fsid":"2034778d-8ff1-4823-aa44-ded32b518241","created":"2024-05-11T08:05:31.612421+0000","modified":"2024-05-11T08:06:36.489215+0000","last_up_change":"2024-05-11T08:05:42.557382+0000","last_in_change":"2024-05-11T08:05:33.757311+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-11T08:05:43.931263+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-11T08:05:52.573726+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-11T08:05:53.670746+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-11T08:05:57.380272+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"587eaced-5ded-406d-884b-30c1caa8686b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":3863307356},{"type":"v1","addr":"172.21.15.167:6806","nonce":3863307356}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":3863307356},{"type":"v1","addr":"172.21.15.167:6810","nonce":3863307356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":3863307356},{"type":"v1","addr":"172.21.15.167:6815","nonce":3863307356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":3863307356},{"type":"v1","addr":"172.21.15.167:6813","nonce":3863307356}]},"public_addr":"172.21.15.167:6806/3863307356","cluster_addr":"172.21.15.167:6810/3863307356","heartbeat_back_addr":"172.21.15.167:6815/3863307356","heartbeat_front_addr":"172.21.15.167:6813/3863307356","state":["exists","up"]},{"osd":1,"uuid":"37c59104-2dd0-4cf9-85ea-56636f6b02a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":1585024048},{"type":"v1","addr":"172.21.15.167:6817","nonce":1585024048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1585024048},{"type":"v1","addr":"172.21.15.167:6819","nonce":1585024048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1585024048},{"type":"v1","addr":"172.21.15.167:6823","nonce":1585024048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":1585024048},{"type":"v1","addr":"172.21.15.167:6821","nonce":1585024048}]},"public_addr":"172.21.15.167:6817/1585024048","cluster_addr":"172.21.15.167:6819/1585024048","heartbeat_back_addr":"172.21.15.167:6823/1585024048","heartbeat_front_addr":"172.21.15.167:6821/1585024048","state":["exists","up"]},{"osd":2,"uuid":"2bb4f7f5-c698-451b-b4bb-78e716921291","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":3242166900},{"type":"v1","addr":"172.21.15.167:6801","nonce":3242166900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":3242166900},{"type":"v1","addr":"172.21.15.167:6803","nonce":3242166900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6809","nonce":3242166900},{"type":"v1","addr":"172.21.15.167:6811","nonce":3242166900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6805","nonce":3242166900},{"type":"v1","addr":"172.21.15.167:6807","nonce":3242166900}]},"public_addr":"172.21.15.167:6801/3242166900","cluster_addr":"172.21.15.167:6803/3242166900","heartbeat_back_addr":"172.21.15.167:6811/3242166900","heartbeat_front_addr":"172.21.15.167:6807/3242166900","state":["exists","up"]},{"osd":3,"uuid":"9dd26759-a338-49d2-8103-45d068b9c000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":453689293},{"type":"v1","addr":"172.21.15.167:6825","nonce":453689293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":453689293},{"type":"v1","addr":"172.21.15.167:6827","nonce":453689293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":453689293},{"type":"v1","addr":"172.21.15.167:6831","nonce":453689293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":453689293},{"type":"v1","addr":"172.21.15.167:6829","nonce":453689293}]},"public_addr":"172.21.15.167:6825/453689293","cluster_addr":"172.21.15.167:6827/453689293","heartbeat_back_addr":"172.21.15.167:6831/453689293","heartbeat_front_addr":"172.21.15.167:6829/453689293","state":["exists","up"]},{"osd":4,"uuid":"cb27250d-2a80-4c7b-8cef-3a7af05c8a16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1311831500},{"type":"v1","addr":"172.21.15.204:6801","nonce":1311831500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1311831500},{"type":"v1","addr":"172.21.15.204:6803","nonce":1311831500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1311831500},{"type":"v1","addr":"172.21.15.204:6807","nonce":1311831500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1311831500},{"type":"v1","addr":"172.21.15.204:6805","nonce":1311831500}]},"public_addr":"172.21.15.204:6801/1311831500","cluster_addr":"172.21.15.204:6803/1311831500","heartbeat_back_addr":"172.21.15.204:6807/1311831500","heartbeat_front_addr":"172.21.15.204:6805/1311831500","state":["exists","up"]},{"osd":5,"uuid":"92998541-ffec-430b-ab76-8c3cb7c2752d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2921847521},{"type":"v1","addr":"172.21.15.204:6825","nonce":2921847521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2921847521},{"type":"v1","addr":"172.21.15.204:6827","nonce":2921847521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2921847521},{"type":"v1","addr":"172.21.15.204:6831","nonce":2921847521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2921847521},{"type":"v1","addr":"172.21.15.204:6829","nonce":2921847521}]},"public_addr":"172.21.15.204:6825/2921847521","cluster_addr":"172.21.15.204:6827/2921847521","heartbeat_back_addr":"172.21.15.204:6831/2921847521","heartbeat_front_addr":"172.21.15.204:6829/2921847521","state":["exists","up"]},{"osd":6,"uuid":"9543479c-2ec5-42a8-8933-707bfed2e048","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":2889851077},{"type":"v1","addr":"172.21.15.204:6817","nonce":2889851077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":2889851077},{"type":"v1","addr":"172.21.15.204:6819","nonce":2889851077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":2889851077},{"type":"v1","addr":"172.21.15.204:6823","nonce":2889851077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":2889851077},{"type":"v1","addr":"172.21.15.204:6821","nonce":2889851077}]},"public_addr":"172.21.15.204:6817/2889851077","cluster_addr":"172.21.15.204:6819/2889851077","heartbeat_back_addr":"172.21.15.204:6823/2889851077","heartbeat_front_addr":"172.21.15.204:6821/2889851077","state":["exists","up"]},{"osd":7,"uuid":"c45a5647-8358-409e-af91-c05dda65f0b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":1949534184},{"type":"v1","addr":"172.21.15.204:6809","nonce":1949534184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":1949534184},{"type":"v1","addr":"172.21.15.204:6811","nonce":1949534184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1949534184},{"type":"v1","addr":"172.21.15.204:6815","nonce":1949534184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1949534184},{"type":"v1","addr":"172.21.15.204:6813","nonce":1949534184}]},"public_addr":"172.21.15.204:6809/1949534184","cluster_addr":"172.21.15.204:6811/1949534184","heartbeat_back_addr":"172.21.15.204:6815/1949534184","heartbeat_front_addr":"172.21.15.204:6813/1949534184","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.952253+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.940689+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.860682+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.929350+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.901627+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.957451+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.987249+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.951197+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.8","mappings":[{"from":3,"to":2}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.2b","mappings":[{"from":0,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"4.f","mappings":[{"from":1,"to":2}]},{"pgid":"4.18","mappings":[{"from":1,"to":2}]},{"pgid":"4.1f","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-11T08:06:37.170 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.170+0000 7fa571e6d640 1 -- 172.21.15.167:0/746751946 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fa5580762d0 msgr2=0x7fa558078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:37.170 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.170+0000 7fa571e6d640 1 --2- 172.21.15.167:0/746751946 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fa5580762d0 0x7fa558078790 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa56c0b9fa0 tx=0x7fa55c008040 comp rx=0 tx=0).stop 2024-05-11T08:06:37.170 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.170+0000 7fa571e6d640 1 -- 172.21.15.167:0/746751946 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa56c1598d0 msgr2=0x7fa56c153080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:37.170 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.170+0000 7fa571e6d640 1 --2- 172.21.15.167:0/746751946 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa56c1598d0 0x7fa56c153080 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fa564009eb0 tx=0x7fa564004a40 comp rx=0 tx=0).stop 2024-05-11T08:06:37.171 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.170+0000 7fa571e6d640 1 -- 172.21.15.167:0/746751946 shutdown_connections 2024-05-11T08:06:37.171 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.170+0000 7fa571e6d640 1 --2- 172.21.15.167:0/746751946 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fa5580762d0 0x7fa558078790 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:37.171 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.170+0000 7fa571e6d640 1 --2- 172.21.15.167:0/746751946 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa56c1598d0 0x7fa56c153080 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:37.171 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.170+0000 7fa571e6d640 1 --2- 172.21.15.167:0/746751946 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa56c0d0640 0x7fa56c0b7e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:37.171 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.170+0000 7fa571e6d640 1 --2- 172.21.15.167:0/746751946 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa56c0cb6c0 0x7fa56c150f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:37.171 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.170+0000 7fa571e6d640 1 -- 172.21.15.167:0/746751946 >> 172.21.15.167:0/746751946 conn(0x7fa56c0d1ba0 msgr2=0x7fa56c0d3630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:37.171 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.170+0000 7fa571e6d640 1 -- 172.21.15.167:0/746751946 shutdown_connections 2024-05-11T08:06:37.171 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.170+0000 7fa571e6d640 1 -- 172.21.15.167:0/746751946 wait complete. 2024-05-11T08:06:37.182 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-11T08:06:37.283 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.283+0000 7f377e847640 1 Processor -- start 2024-05-11T08:06:37.284 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.284+0000 7f377e847640 1 -- start start 2024-05-11T08:06:37.287 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.284+0000 7f377e847640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3778071800 0x7f377815ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:37.287 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.284+0000 7f377e847640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37780b4f50 con 0x7f3778148900 2024-05-11T08:06:37.287 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.284+0000 7f377e847640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37780b5230 con 0x7f3778160470 2024-05-11T08:06:37.287 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.284+0000 7f377e847640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f37780b5510 con 0x7f3778071800 2024-05-11T08:06:37.287 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.285+0000 7f3777fff640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f3778148900 0x7f37780710f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:49596/0 (socket says 172.21.15.167:49596) 2024-05-11T08:06:37.287 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.285+0000 7f3777fff640 1 -- 172.21.15.167:0/2975841250 learned_addr learned my addr 172.21.15.167:0/2975841250 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T08:06:37.287 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.285+0000 7f376ffff640 1 --2- 172.21.15.167:0/2975841250 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3778071800 0x7f377815ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:37.287 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.285+0000 7f37777fe640 1 -- 172.21.15.167:0/2975841250 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1938682337 0 0) 0x7f37780b4f50 con 0x7f3778148900 2024-05-11T08:06:37.287 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.285+0000 7f37777fe640 1 -- 172.21.15.167:0/2975841250 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3764003520 con 0x7f3778148900 2024-05-11T08:06:37.287 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.285+0000 7f37777fe640 1 -- 172.21.15.167:0/2975841250 <== mon.0 v1:172.21.15.167:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1362301510 0 0) 0x7f3764003520 con 0x7f3778148900 2024-05-11T08:06:37.287 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.285+0000 7f37777fe640 1 -- 172.21.15.167:0/2975841250 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3778071800 msgr2=0x7f377815ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:37.287 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.285+0000 7f37777fe640 1 --2- 172.21.15.167:0/2975841250 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3778071800 0x7f377815ff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:37.288 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.285+0000 7f37777fe640 1 -- 172.21.15.167:0/2975841250 >> v1:172.21.15.204:6789/0 conn(0x7f3778160470 legacy=0x7f3778160920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:37.288 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.285+0000 7f37777fe640 1 -- 172.21.15.167:0/2975841250 --> v1:172.21.15.167:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37780b57f0 con 0x7f3778148900 2024-05-11T08:06:37.288 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.285+0000 7f37777fe640 1 -- 172.21.15.167:0/2975841250 <== mon.0 v1:172.21.15.167:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3602277846 0 0) 0x7f37680016f0 con 0x7f3778148900 2024-05-11T08:06:37.288 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.285+0000 7f37777fe640 1 -- 172.21.15.167:0/2975841250 >> v1:172.21.15.167:6789/0 conn(0x7f3778148900 legacy=0x7f37780710f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:37.288 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.285+0000 7f37777fe640 1 --2- 172.21.15.167:0/2975841250 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3764003a30 0x7f3764003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:37.288 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.285+0000 7f37777fe640 1 --2- 172.21.15.167:0/2975841250 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3778071800 0x7f3764004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:37.288 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.285+0000 7f37777fe640 1 --2- 172.21.15.167:0/2975841250 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3764004d40 0x7f37640051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:37.288 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.285+0000 7f37777fe640 1 -- 172.21.15.167:0/2975841250 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f37780b5510 con 0x7f3764003a30 2024-05-11T08:06:37.288 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.285+0000 7f37777fe640 1 -- 172.21.15.167:0/2975841250 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f376400baf0 con 0x7f3764004d40 2024-05-11T08:06:37.288 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.285+0000 7f37777fe640 1 -- 172.21.15.167:0/2975841250 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f376400bcf0 con 0x7f3778071800 2024-05-11T08:06:37.288 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.285+0000 7f3777fff640 1 --2- 172.21.15.167:0/2975841250 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3764003a30 0x7f3764003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:37.288 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.285+0000 7f376ffff640 1 --2- 172.21.15.167:0/2975841250 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3778071800 0x7f3764004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:37.288 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.285+0000 7f377cde4640 1 --2- 172.21.15.167:0/2975841250 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3764004d40 0x7f37640051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:37.288 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.286+0000 7f3777fff640 1 -- 172.21.15.167:0/2975841250 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3778071800 msgr2=0x7f3764004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:37.289 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.286+0000 7f3777fff640 1 --2- 172.21.15.167:0/2975841250 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3778071800 0x7f3764004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:37.289 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.286+0000 7f3777fff640 1 -- 172.21.15.167:0/2975841250 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3764004d40 msgr2=0x7f37640051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:37.289 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.286+0000 7f3777fff640 1 --2- 172.21.15.167:0/2975841250 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3764004d40 0x7f37640051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:37.289 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.286+0000 7f3777fff640 1 -- 172.21.15.167:0/2975841250 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3764003520 con 0x7f3764003a30 2024-05-11T08:06:37.289 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.286+0000 7f3777fff640 1 --2- 172.21.15.167:0/2975841250 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3764003a30 0x7f3764003e50 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f376800c240 tx=0x7f3764005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ace11073a47c9a68 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:37.289 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.287+0000 7f37777fe640 1 -- 172.21.15.167:0/2975841250 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3768007ce0 con 0x7f3764003a30 2024-05-11T08:06:37.289 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.287+0000 7f37777fe640 1 -- 172.21.15.167:0/2975841250 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3768005ad0 con 0x7f3764003a30 2024-05-11T08:06:37.289 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.287+0000 7f37777fe640 1 -- 172.21.15.167:0/2975841250 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3768007070 con 0x7f3764003a30 2024-05-11T08:06:37.292 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.287+0000 7f377e847640 1 -- 172.21.15.167:0/2975841250 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3764003a30 msgr2=0x7f3764003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:37.292 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.287+0000 7f377e847640 1 --2- 172.21.15.167:0/2975841250 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3764003a30 0x7f3764003e50 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f376800c240 tx=0x7f3764005830 comp rx=0 tx=0).stop 2024-05-11T08:06:37.292 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.287+0000 7f377cde4640 1 -- 172.21.15.167:0/2975841250 reap_dead start 2024-05-11T08:06:37.292 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.287+0000 7f377e847640 1 -- 172.21.15.167:0/2975841250 shutdown_connections 2024-05-11T08:06:37.292 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.287+0000 7f377e847640 1 -- 172.21.15.167:0/2975841250 >> 172.21.15.167:0/2975841250 conn(0x7f37780d1e40 msgr2=0x7f37780d2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:37.292 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.287+0000 7f377e847640 1 -- 172.21.15.167:0/2975841250 shutdown_connections 2024-05-11T08:06:37.292 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.287+0000 7f377e847640 1 -- 172.21.15.167:0/2975841250 wait complete. 2024-05-11T08:06:37.293 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.288+0000 7f377e847640 1 Processor -- start 2024-05-11T08:06:37.293 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.288+0000 7f377e847640 1 -- start start 2024-05-11T08:06:37.293 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.289+0000 7f377e847640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3764003a30 0x7f3778142450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:37.293 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.289+0000 7f377e847640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f37781f50e0 0x7f3778147090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:37.293 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.289+0000 7f377e847640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f37781f5540 0x7f37781475d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:37.293 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.289+0000 7f377e847640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f377815f900 con 0x7f3764003a30 2024-05-11T08:06:37.293 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.289+0000 7f377e847640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f3778148ce0 con 0x7f37781f50e0 2024-05-11T08:06:37.293 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.289+0000 7f377e847640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f377815fa80 con 0x7f37781f5540 2024-05-11T08:06:37.293 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.289+0000 7f3777fff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3764003a30 0x7f3778142450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:37.293 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.289+0000 7f3777fff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3764003a30 0x7f3778142450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:34848/0 (socket says 172.21.15.167:34848) 2024-05-11T08:06:37.293 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.289+0000 7f3777fff640 1 -- 172.21.15.167:0/993593117 learned_addr learned my addr 172.21.15.167:0/993593117 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:37.293 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.290+0000 7f377cde4640 1 --2- 172.21.15.167:0/993593117 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f37781f5540 0x7f37781475d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:37.293 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.290+0000 7f376ffff640 1 --2- 172.21.15.167:0/993593117 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f37781f50e0 0x7f3778147090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:37.293 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.290+0000 7f3777fff640 1 -- 172.21.15.167:0/993593117 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f37781f5540 msgr2=0x7f37781475d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:37.294 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.290+0000 7f3777fff640 1 --2- 172.21.15.167:0/993593117 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f37781f5540 0x7f37781475d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:37.294 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.290+0000 7f3777fff640 1 -- 172.21.15.167:0/993593117 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f37781f50e0 msgr2=0x7f3778147090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:37.294 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.290+0000 7f3777fff640 1 --2- 172.21.15.167:0/993593117 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f37781f50e0 0x7f3778147090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:37.294 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.290+0000 7f3777fff640 1 -- 172.21.15.167:0/993593117 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37781401d0 con 0x7f3764003a30 2024-05-11T08:06:37.294 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.290+0000 7f377cde4640 1 --2- 172.21.15.167:0/993593117 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f37781f5540 0x7f37781475d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T08:06:37.294 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.290+0000 7f3777fff640 1 --2- 172.21.15.167:0/993593117 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3764003a30 0x7f3778142450 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f3768000c00 tx=0x7f37780b56e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:37.294 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.290+0000 7f37757fa640 1 -- 172.21.15.167:0/993593117 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3768041020 con 0x7f3764003a30 2024-05-11T08:06:37.294 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.290+0000 7f37757fa640 1 -- 172.21.15.167:0/993593117 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37680014b0 con 0x7f3764003a30 2024-05-11T08:06:37.294 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.290+0000 7f377e847640 1 -- 172.21.15.167:0/993593117 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37781404b0 con 0x7f3764003a30 2024-05-11T08:06:37.294 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.290+0000 7f37757fa640 1 -- 172.21.15.167:0/993593117 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3768007070 con 0x7f3764003a30 2024-05-11T08:06:37.294 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.290+0000 7f377e847640 1 -- 172.21.15.167:0/993593117 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3778140a70 con 0x7f3764003a30 2024-05-11T08:06:37.294 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.291+0000 7f377e847640 1 -- 172.21.15.167:0/993593117 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3778069bc0 con 0x7f3764003a30 2024-05-11T08:06:37.294 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.292+0000 7f37757fa640 1 -- 172.21.15.167:0/993593117 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f3768044020 con 0x7f3764003a30 2024-05-11T08:06:37.294 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.292+0000 7f37757fa640 1 --2- 172.21.15.167:0/993593117 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f3758076300 0x7f37580787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:37.295 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.292+0000 7f37757fa640 1 -- 172.21.15.167:0/993593117 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7f37680b7de0 con 0x7f3764003a30 2024-05-11T08:06:37.295 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.292+0000 7f376ffff640 1 --2- 172.21.15.167:0/993593117 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f3758076300 0x7f37580787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:37.295 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.293+0000 7f376ffff640 1 --2- 172.21.15.167:0/993593117 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f3758076300 0x7f37580787c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f377815d240 tx=0x7f3760008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:37.296 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.297+0000 7f37757fa640 1 -- 172.21.15.167:0/993593117 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f3768005aa0 con 0x7f3764003a30 2024-05-11T08:06:37.521 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.521+0000 7f377e847640 1 -- 172.21.15.167:0/993593117 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f377815e010 con 0x7f3764003a30 2024-05-11T08:06:37.522 INFO:tasks.ceph.mon.a.smithi167.stderr:2024-05-11T08:06:37.522+0000 7feeb5228640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-11T08:06:37.526 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.526+0000 7f37757fa640 1 -- 172.21.15.167:0/993593117 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v10) ==== 161+0+0 (secure 0 0 0) 0x7f3768081340 con 0x7f3764003a30 2024-05-11T08:06:37.526 INFO:teuthology.orchestra.run.smithi167.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-11T08:06:37.528 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.528+0000 7f377e847640 1 -- 172.21.15.167:0/993593117 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f3758076300 msgr2=0x7f37580787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:37.528 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.528+0000 7f377e847640 1 --2- 172.21.15.167:0/993593117 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f3758076300 0x7f37580787c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f377815d240 tx=0x7f3760008040 comp rx=0 tx=0).stop 2024-05-11T08:06:37.528 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.528+0000 7f377e847640 1 -- 172.21.15.167:0/993593117 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3764003a30 msgr2=0x7f3778142450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:37.529 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.528+0000 7f377e847640 1 --2- 172.21.15.167:0/993593117 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3764003a30 0x7f3778142450 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f3768000c00 tx=0x7f37780b56e0 comp rx=0 tx=0).stop 2024-05-11T08:06:37.529 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.528+0000 7f377e847640 1 -- 172.21.15.167:0/993593117 shutdown_connections 2024-05-11T08:06:37.529 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.528+0000 7f377e847640 1 --2- 172.21.15.167:0/993593117 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f3758076300 0x7f37580787c0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:37.529 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.528+0000 7f377e847640 1 --2- 172.21.15.167:0/993593117 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f37781f5540 0x7f37781475d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:37.529 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.528+0000 7f377e847640 1 --2- 172.21.15.167:0/993593117 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f37781f50e0 0x7f3778147090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:37.529 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.528+0000 7f377e847640 1 --2- 172.21.15.167:0/993593117 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3764003a30 0x7f3778142450 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:37.529 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.528+0000 7f377e847640 1 -- 172.21.15.167:0/993593117 >> 172.21.15.167:0/993593117 conn(0x7f37780d1e40 msgr2=0x7f377815e5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:37.529 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.528+0000 7f377e847640 1 -- 172.21.15.167:0/993593117 shutdown_connections 2024-05-11T08:06:37.529 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.528+0000 7f377e847640 1 -- 172.21.15.167:0/993593117 wait complete. 2024-05-11T08:06:37.532 INFO:tasks.ceph.mds.b.smithi204.stderr: -4194> 2024-05-11T08:06:00.330+0000 7fb112f60640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-11T08:06:37.540 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-11T08:06:37.564 INFO:tasks.ceph.mds.b.smithi204.stdout:starting mds.b at 2024-05-11T08:06:37.642 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.642+0000 7f33a8124640 1 Processor -- start 2024-05-11T08:06:37.642 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.642+0000 7f33a8124640 1 -- start start 2024-05-11T08:06:37.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.643+0000 7f33a8124640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f33a0161380 0x7f33a0074260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:37.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.643+0000 7f33a8124640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f33a00b3d50 0x7f33a0072950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:37.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.643+0000 7f33a8124640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f33a00b65e0 con 0x7f33a015ac80 2024-05-11T08:06:37.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.643+0000 7f33a8124640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f33a006e3c0 con 0x7f33a00b3d50 2024-05-11T08:06:37.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.643+0000 7f33a8124640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f33a015b060 con 0x7f33a0161380 2024-05-11T08:06:37.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.643+0000 7f33a5ec0640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f33a015ac80 0x7f33a0160c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:49604/0 (socket says 172.21.15.167:49604) 2024-05-11T08:06:37.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.643+0000 7f33a5ec0640 1 -- 172.21.15.167:0/1803964008 learned_addr learned my addr 172.21.15.167:0/1803964008 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T08:06:37.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.643+0000 7f33a56bf640 1 --2- 172.21.15.167:0/1803964008 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f33a0161380 0x7f33a0074260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:37.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.643+0000 7f33a66c1640 1 --2- 172.21.15.167:0/1803964008 >> v2:172.21.15.204:3300/0 conn(0x7f33a00b3d50 0x7f33a0072950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:37.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.643+0000 7f33a4ebe640 1 -- 172.21.15.167:0/1803964008 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1899098138 0 0) 0x7f33a00b65e0 con 0x7f33a015ac80 2024-05-11T08:06:37.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.643+0000 7f33a4ebe640 1 -- 172.21.15.167:0/1803964008 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f338c003520 con 0x7f33a015ac80 2024-05-11T08:06:37.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.643+0000 7f33a56bf640 1 -- 172.21.15.167:0/1803964008 >> v2:172.21.15.204:3300/0 conn(0x7f33a00b3d50 msgr2=0x7f33a0072950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:37.645 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.643+0000 7f33a56bf640 1 --2- 172.21.15.167:0/1803964008 >> v2:172.21.15.204:3300/0 conn(0x7f33a00b3d50 0x7f33a0072950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:37.645 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.643+0000 7f33a56bf640 1 -- 172.21.15.167:0/1803964008 >> v1:172.21.15.167:6789/0 conn(0x7f33a015ac80 legacy=0x7f33a0160c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:37.645 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.643+0000 7f33a56bf640 1 -- 172.21.15.167:0/1803964008 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33a00b68c0 con 0x7f33a0161380 2024-05-11T08:06:37.646 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.644+0000 7f33a56bf640 1 --2- 172.21.15.167:0/1803964008 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f33a0161380 0x7f33a0074260 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f33900097a0 tx=0x7f339002a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=44f4a4879eea32ff server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:37.646 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.644+0000 7f33a4ebe640 1 -- 172.21.15.167:0/1803964008 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3390037070 con 0x7f33a0161380 2024-05-11T08:06:37.646 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.644+0000 7f33a4ebe640 1 -- 172.21.15.167:0/1803964008 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3390032440 con 0x7f33a0161380 2024-05-11T08:06:37.646 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.644+0000 7f33a4ebe640 1 -- 172.21.15.167:0/1803964008 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33900327f0 con 0x7f33a0161380 2024-05-11T08:06:37.647 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.645+0000 7f33a8124640 1 -- 172.21.15.167:0/1803964008 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f33a0161380 msgr2=0x7f33a0074260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:37.647 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.645+0000 7f33a8124640 1 --2- 172.21.15.167:0/1803964008 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f33a0161380 0x7f33a0074260 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f33900097a0 tx=0x7f339002a180 comp rx=0 tx=0).stop 2024-05-11T08:06:37.647 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.645+0000 7f33a8124640 1 -- 172.21.15.167:0/1803964008 shutdown_connections 2024-05-11T08:06:37.647 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.645+0000 7f33a8124640 1 --2- 172.21.15.167:0/1803964008 >> v2:172.21.15.204:3300/0 conn(0x7f33a00b3d50 0x7f33a0072950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:37.647 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.645+0000 7f33a8124640 1 --2- 172.21.15.167:0/1803964008 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f33a0161380 0x7f33a0074260 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:37.647 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.645+0000 7f33a8124640 1 -- 172.21.15.167:0/1803964008 >> 172.21.15.167:0/1803964008 conn(0x7f33a006c450 msgr2=0x7f33a006c860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:37.647 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.645+0000 7f33a8124640 1 -- 172.21.15.167:0/1803964008 shutdown_connections 2024-05-11T08:06:37.647 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.645+0000 7f33a8124640 1 -- 172.21.15.167:0/1803964008 wait complete. 2024-05-11T08:06:37.647 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.646+0000 7f33a8124640 1 Processor -- start 2024-05-11T08:06:37.647 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.646+0000 7f33a8124640 1 -- start start 2024-05-11T08:06:37.650 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.647+0000 7f33a8124640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f33a00b3d50 0x7f33a0141a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:37.650 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.647+0000 7f33a8124640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f33a015ac80 0x7f33a01439c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:37.650 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.647+0000 7f33a8124640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f33a0161380 0x7f33a00ad120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:37.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.647+0000 7f33a8124640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f33a00b44f0 con 0x7f33a0161380 2024-05-11T08:06:37.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.647+0000 7f33a8124640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f33a00b41e0 con 0x7f33a00b3d50 2024-05-11T08:06:37.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.647+0000 7f33a8124640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f33a0072e90 con 0x7f33a015ac80 2024-05-11T08:06:37.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.647+0000 7f33a66c1640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f33a0161380 0x7f33a00ad120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:37.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.647+0000 7f33a66c1640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f33a0161380 0x7f33a00ad120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:34858/0 (socket says 172.21.15.167:34858) 2024-05-11T08:06:37.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.647+0000 7f33a66c1640 1 -- 172.21.15.167:0/2866552711 learned_addr learned my addr 172.21.15.167:0/2866552711 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:37.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.647+0000 7f33a5ec0640 1 --2- 172.21.15.167:0/2866552711 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f33a00b3d50 0x7f33a0141a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:37.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.647+0000 7f33a56bf640 1 --2- 172.21.15.167:0/2866552711 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f33a015ac80 0x7f33a01439c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:37.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.647+0000 7f33a66c1640 1 -- 172.21.15.167:0/2866552711 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f33a015ac80 msgr2=0x7f33a01439c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:37.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.647+0000 7f33a66c1640 1 --2- 172.21.15.167:0/2866552711 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f33a015ac80 0x7f33a01439c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:37.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.647+0000 7f33a66c1640 1 -- 172.21.15.167:0/2866552711 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f33a00b3d50 msgr2=0x7f33a0141a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:37.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.647+0000 7f33a66c1640 1 --2- 172.21.15.167:0/2866552711 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f33a00b3d50 0x7f33a0141a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:37.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.647+0000 7f33a66c1640 1 -- 172.21.15.167:0/2866552711 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33a0142920 con 0x7f33a0161380 2024-05-11T08:06:37.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.647+0000 7f33a66c1640 1 --2- 172.21.15.167:0/2866552711 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f33a0161380 0x7f33a00ad120 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f339800e930 tx=0x7f339800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:37.652 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.647+0000 7f33967fc640 1 -- 172.21.15.167:0/2866552711 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3398017070 con 0x7f33a0161380 2024-05-11T08:06:37.652 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.648+0000 7f33967fc640 1 -- 172.21.15.167:0/2866552711 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f339800c980 con 0x7f33a0161380 2024-05-11T08:06:37.652 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.648+0000 7f33a8124640 1 -- 172.21.15.167:0/2866552711 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33a0142c00 con 0x7f33a0161380 2024-05-11T08:06:37.652 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.648+0000 7f33967fc640 1 -- 172.21.15.167:0/2866552711 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f339800cd70 con 0x7f33a0161380 2024-05-11T08:06:37.652 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.648+0000 7f33a8124640 1 -- 172.21.15.167:0/2866552711 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33a01431c0 con 0x7f33a0161380 2024-05-11T08:06:37.652 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.649+0000 7f33a8124640 1 -- 172.21.15.167:0/2866552711 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33a0069520 con 0x7f33a0161380 2024-05-11T08:06:37.652 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.649+0000 7f33967fc640 1 -- 172.21.15.167:0/2866552711 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f339801a070 con 0x7f33a0161380 2024-05-11T08:06:37.652 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.650+0000 7f33967fc640 1 --2- 172.21.15.167:0/2866552711 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f33840762d0 0x7f3384078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:37.652 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.650+0000 7f33a5ec0640 1 --2- 172.21.15.167:0/2866552711 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f33840762d0 0x7f3384078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:37.652 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.650+0000 7f33967fc640 1 -- 172.21.15.167:0/2866552711 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6821+0+0 (secure 0 0 0) 0x7f3398097de0 con 0x7f33a0161380 2024-05-11T08:06:37.652 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.650+0000 7f33a5ec0640 1 --2- 172.21.15.167:0/2866552711 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f33840762d0 0x7f3384078790 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f339c001800 tx=0x7f339c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:37.654 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.654+0000 7f33967fc640 1 -- 172.21.15.167:0/2866552711 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f33980613b0 con 0x7f33a0161380 2024-05-11T08:06:37.877 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:37.877+0000 7f33a8124640 1 -- 172.21.15.167:0/2866552711 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f33a015d230 con 0x7f33a0161380 2024-05-11T08:06:38.531 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:38.531+0000 7f33967fc640 1 -- 172.21.15.167:0/2866552711 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v11) ==== 104+0+0 (secure 0 0 0) 0x7f3398060cb0 con 0x7f33a0161380 2024-05-11T08:06:38.534 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:38.534+0000 7f33a8124640 1 -- 172.21.15.167:0/2866552711 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f33840762d0 msgr2=0x7f3384078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:38.534 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:38.534+0000 7f33a8124640 1 --2- 172.21.15.167:0/2866552711 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f33840762d0 0x7f3384078790 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f339c001800 tx=0x7f339c009040 comp rx=0 tx=0).stop 2024-05-11T08:06:38.534 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:38.534+0000 7f33a8124640 1 -- 172.21.15.167:0/2866552711 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f33a0161380 msgr2=0x7f33a00ad120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:38.534 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:38.534+0000 7f33a8124640 1 --2- 172.21.15.167:0/2866552711 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f33a0161380 0x7f33a00ad120 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f339800e930 tx=0x7f339800edf0 comp rx=0 tx=0).stop 2024-05-11T08:06:38.534 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:38.534+0000 7f33a8124640 1 -- 172.21.15.167:0/2866552711 shutdown_connections 2024-05-11T08:06:38.534 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:38.534+0000 7f33a8124640 1 --2- 172.21.15.167:0/2866552711 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f33840762d0 0x7f3384078790 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:38.534 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:38.534+0000 7f33a8124640 1 --2- 172.21.15.167:0/2866552711 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f33a0161380 0x7f33a00ad120 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:38.535 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:38.534+0000 7f33a8124640 1 --2- 172.21.15.167:0/2866552711 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f33a015ac80 0x7f33a01439c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:38.535 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:38.534+0000 7f33a8124640 1 --2- 172.21.15.167:0/2866552711 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f33a00b3d50 0x7f33a0141a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:38.535 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:38.534+0000 7f33a8124640 1 -- 172.21.15.167:0/2866552711 >> 172.21.15.167:0/2866552711 conn(0x7f33a006c450 msgr2=0x7f33a0159a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:38.535 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:38.534+0000 7f33a8124640 1 -- 172.21.15.167:0/2866552711 shutdown_connections 2024-05-11T08:06:38.535 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:38.534+0000 7f33a8124640 1 -- 172.21.15.167:0/2866552711 wait complete. 2024-05-11T08:06:38.545 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-11T08:06:38.650 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:38.649+0000 7f786b4c8640 1 Processor -- start 2024-05-11T08:06:38.650 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:38.650+0000 7f786b4c8640 1 -- start start 2024-05-11T08:06:38.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:38.650+0000 7f786b4c8640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f78640b77e0 0x7f78640d11a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:38.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:38.650+0000 7f786b4c8640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f78640b54f0 0x7f786406ee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:38.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:38.651+0000 7f786b4c8640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7864145cf0 con 0x7f7864145910 2024-05-11T08:06:38.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:38.651+0000 7f786b4c8640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f78640d16e0 con 0x7f78640b77e0 2024-05-11T08:06:38.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:38.651+0000 7f786b4c8640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f78640d0100 con 0x7f78640b54f0 2024-05-11T08:06:38.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:38.651+0000 7f7868a63640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f78640b77e0 0x7f78640d11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:38.652 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:38.651+0000 7f7868a63640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f78640b77e0 0x7f78640d11a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:34872/0 (socket says 172.21.15.167:34872) 2024-05-11T08:06:38.652 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:38.651+0000 7f7868a63640 1 -- 172.21.15.167:0/1079080970 learned_addr learned my addr 172.21.15.167:0/1079080970 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:38.652 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:38.651+0000 7f7868a63640 1 -- 172.21.15.167:0/1079080970 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f78640b54f0 msgr2=0x7f786406ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:38.652 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:38.651+0000 7f7868a63640 1 --2- 172.21.15.167:0/1079080970 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f78640b54f0 0x7f786406ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:38.652 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:38.651+0000 7f7868a63640 1 -- 172.21.15.167:0/1079080970 >> v1:172.21.15.204:6789/0 conn(0x7f7864145910 legacy=0x7f78640b70d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:38.652 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:38.651+0000 7f7868a63640 1 -- 172.21.15.167:0/1079080970 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78640d03e0 con 0x7f78640b77e0 2024-05-11T08:06:38.653 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:38.652+0000 7f7868a63640 1 --2- 172.21.15.167:0/1079080970 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f78640b77e0 0x7f78640d11a0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f786000bb10 tx=0x7f786002c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2ee8c8bcae3094 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:38.653 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:38.652+0000 7f7857fff640 1 -- 172.21.15.167:0/1079080970 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7860032770 con 0x7f78640b77e0 2024-05-11T08:06:38.653 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:38.652+0000 7f7857fff640 1 -- 172.21.15.167:0/1079080970 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7860038070 con 0x7f78640b77e0 2024-05-11T08:06:38.653 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:38.652+0000 7f7857fff640 1 -- 172.21.15.167:0/1079080970 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7860032b30 con 0x7f78640b77e0 2024-05-11T08:06:38.653 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:38.652+0000 7f786b4c8640 1 -- 172.21.15.167:0/1079080970 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f78640b77e0 msgr2=0x7f78640d11a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:38.653 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:38.652+0000 7f786b4c8640 1 --2- 172.21.15.167:0/1079080970 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f78640b77e0 0x7f78640d11a0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f786000bb10 tx=0x7f786002c870 comp rx=0 tx=0).stop 2024-05-11T08:06:38.654 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:38.652+0000 7f786b4c8640 1 -- 172.21.15.167:0/1079080970 shutdown_connections 2024-05-11T08:06:38.654 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:38.652+0000 7f786b4c8640 1 --2- 172.21.15.167:0/1079080970 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f78640b54f0 0x7f786406ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:38.654 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:38.652+0000 7f786b4c8640 1 --2- 172.21.15.167:0/1079080970 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f78640b77e0 0x7f78640d11a0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:38.654 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:38.652+0000 7f786b4c8640 1 -- 172.21.15.167:0/1079080970 >> 172.21.15.167:0/1079080970 conn(0x7f78640ce900 msgr2=0x7f78640ced10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:38.654 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:38.652+0000 7f786b4c8640 1 -- 172.21.15.167:0/1079080970 shutdown_connections 2024-05-11T08:06:38.654 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:38.652+0000 7f786b4c8640 1 -- 172.21.15.167:0/1079080970 wait complete. 2024-05-11T08:06:38.656 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:38.654+0000 7f786b4c8640 1 Processor -- start 2024-05-11T08:06:38.656 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:38.654+0000 7f786b4c8640 1 -- start start 2024-05-11T08:06:38.656 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:38.654+0000 7f786b4c8640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f78640b54f0 0x7f7864138340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:38.657 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:38.654+0000 7f786b4c8640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f78640b77e0 0x7f78640b0d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:38.657 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:38.654+0000 7f786b4c8640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7864145910 0x7f7864157040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:38.657 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:38.654+0000 7f786b4c8640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f78640d0c80 con 0x7f78640b77e0 2024-05-11T08:06:38.657 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:38.654+0000 7f786b4c8640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f78640d09a0 con 0x7f7864145910 2024-05-11T08:06:38.657 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:38.654+0000 7f786b4c8640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f78640cfd20 con 0x7f78640b54f0 2024-05-11T08:06:38.657 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:38.654+0000 7f7868a63640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f78640b77e0 0x7f78640b0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:38.657 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:38.655+0000 7f7868a63640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f78640b77e0 0x7f78640b0d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:34884/0 (socket says 172.21.15.167:34884) 2024-05-11T08:06:38.657 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:38.655+0000 7f7868a63640 1 -- 172.21.15.167:0/1502638212 learned_addr learned my addr 172.21.15.167:0/1502638212 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:38.657 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:38.655+0000 7f7869264640 1 --2- 172.21.15.167:0/1502638212 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f78640b54f0 0x7f7864138340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:38.657 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:38.655+0000 7f7869a65640 1 --2- 172.21.15.167:0/1502638212 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7864145910 0x7f7864157040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:38.657 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:38.655+0000 7f7868a63640 1 -- 172.21.15.167:0/1502638212 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f78640b54f0 msgr2=0x7f7864138340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:38.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:38.655+0000 7f7868a63640 1 --2- 172.21.15.167:0/1502638212 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f78640b54f0 0x7f7864138340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:38.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:38.655+0000 7f7868a63640 1 -- 172.21.15.167:0/1502638212 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7864145910 msgr2=0x7f7864157040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:38.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:38.655+0000 7f7868a63640 1 --2- 172.21.15.167:0/1502638212 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7864145910 0x7f7864157040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:38.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:38.655+0000 7f7868a63640 1 -- 172.21.15.167:0/1502638212 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78640b12b0 con 0x7f78640b77e0 2024-05-11T08:06:38.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:38.655+0000 7f7868a63640 1 --2- 172.21.15.167:0/1502638212 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f78640b77e0 0x7f78640b0d70 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f786000b2c0 tx=0x7f7860004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:38.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:38.655+0000 7f7855ffb640 1 -- 172.21.15.167:0/1502638212 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7860043070 con 0x7f78640b77e0 2024-05-11T08:06:38.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:38.655+0000 7f7855ffb640 1 -- 172.21.15.167:0/1502638212 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7860033a30 con 0x7f78640b77e0 2024-05-11T08:06:38.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:38.655+0000 7f786b4c8640 1 -- 172.21.15.167:0/1502638212 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78641578b0 con 0x7f78640b77e0 2024-05-11T08:06:38.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:38.655+0000 7f7855ffb640 1 -- 172.21.15.167:0/1502638212 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7860033e20 con 0x7f78640b77e0 2024-05-11T08:06:38.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:38.655+0000 7f786b4c8640 1 -- 172.21.15.167:0/1502638212 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7864157e70 con 0x7f78640b77e0 2024-05-11T08:06:38.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:38.656+0000 7f786b4c8640 1 -- 172.21.15.167:0/1502638212 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7864069bc0 con 0x7f78640b77e0 2024-05-11T08:06:38.659 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:38.657+0000 7f7855ffb640 1 -- 172.21.15.167:0/1502638212 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f7860046070 con 0x7f78640b77e0 2024-05-11T08:06:38.659 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:38.658+0000 7f7855ffb640 1 --2- 172.21.15.167:0/1502638212 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f78480762d0 0x7f7848078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:38.659 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:38.658+0000 7f7869264640 1 --2- 172.21.15.167:0/1502638212 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f78480762d0 0x7f7848078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:38.659 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:38.658+0000 7f7855ffb640 1 -- 172.21.15.167:0/1502638212 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6821+0+0 (secure 0 0 0) 0x7f78600b8300 con 0x7f78640b77e0 2024-05-11T08:06:38.660 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:38.658+0000 7f7869264640 1 --2- 172.21.15.167:0/1502638212 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f78480762d0 0x7f7848078790 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f7858006fd0 tx=0x7f7858009060 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:38.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:38.662+0000 7f7855ffb640 1 -- 172.21.15.167:0/1502638212 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f78600812e0 con 0x7f78640b77e0 2024-05-11T08:06:38.847 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:38.847+0000 7f786b4c8640 1 -- 172.21.15.167:0/1502638212 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f7864070ba0 con 0x7f78640b77e0 2024-05-11T08:06:39.533 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:39.533+0000 7f7855ffb640 1 -- 172.21.15.167:0/1502638212 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v31) ==== 181+0+0 (secure 0 0 0) 0x7f78600855e0 con 0x7f78640b77e0 2024-05-11T08:06:39.533 INFO:teuthology.orchestra.run.smithi167.stderr:pool 'cephfs_metadata' removed 2024-05-11T08:06:39.536 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:39.536+0000 7f786b4c8640 1 -- 172.21.15.167:0/1502638212 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f78480762d0 msgr2=0x7f7848078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:39.536 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:39.536+0000 7f786b4c8640 1 --2- 172.21.15.167:0/1502638212 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f78480762d0 0x7f7848078790 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f7858006fd0 tx=0x7f7858009060 comp rx=0 tx=0).stop 2024-05-11T08:06:39.536 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:39.536+0000 7f786b4c8640 1 -- 172.21.15.167:0/1502638212 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f78640b77e0 msgr2=0x7f78640b0d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:39.537 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:39.536+0000 7f786b4c8640 1 --2- 172.21.15.167:0/1502638212 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f78640b77e0 0x7f78640b0d70 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f786000b2c0 tx=0x7f7860004730 comp rx=0 tx=0).stop 2024-05-11T08:06:39.537 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:39.536+0000 7f786b4c8640 1 -- 172.21.15.167:0/1502638212 shutdown_connections 2024-05-11T08:06:39.537 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:39.536+0000 7f786b4c8640 1 --2- 172.21.15.167:0/1502638212 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f78480762d0 0x7f7848078790 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:39.537 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:39.536+0000 7f786b4c8640 1 --2- 172.21.15.167:0/1502638212 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7864145910 0x7f7864157040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:39.537 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:39.536+0000 7f786b4c8640 1 --2- 172.21.15.167:0/1502638212 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f78640b77e0 0x7f78640b0d70 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:39.537 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:39.536+0000 7f786b4c8640 1 --2- 172.21.15.167:0/1502638212 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f78640b54f0 0x7f7864138340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:39.537 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:39.536+0000 7f786b4c8640 1 -- 172.21.15.167:0/1502638212 >> 172.21.15.167:0/1502638212 conn(0x7f78640ce900 msgr2=0x7f786415eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:39.537 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:39.536+0000 7f786b4c8640 1 -- 172.21.15.167:0/1502638212 shutdown_connections 2024-05-11T08:06:39.537 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:39.537+0000 7f786b4c8640 1 -- 172.21.15.167:0/1502638212 wait complete. 2024-05-11T08:06:39.550 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-11T08:06:39.656 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:39.656+0000 7f4b53015640 1 Processor -- start 2024-05-11T08:06:39.656 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:39.656+0000 7f4b53015640 1 -- start start 2024-05-11T08:06:39.660 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:39.657+0000 7f4b53015640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f4b4c1439a0 0x7f4b4c0cfee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:39.660 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:39.657+0000 7f4b53015640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b4c0d0420 con 0x7f4b4c0cf820 2024-05-11T08:06:39.660 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:39.657+0000 7f4b53015640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b4c0b3f30 con 0x7f4b4c0b59f0 2024-05-11T08:06:39.660 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:39.657+0000 7f4b53015640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f4b4c0b3ce0 con 0x7f4b4c1439a0 2024-05-11T08:06:39.660 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:39.657+0000 7f4b4bfff640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f4b4c0cf820 0x7f4b4c0b52e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:49618/0 (socket says 172.21.15.167:49618) 2024-05-11T08:06:39.660 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:39.657+0000 7f4b4bfff640 1 -- 172.21.15.167:0/1508249577 learned_addr learned my addr 172.21.15.167:0/1508249577 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T08:06:39.660 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:39.657+0000 7f4b50db1640 1 --2- 172.21.15.167:0/1508249577 >> v2:172.21.15.204:3300/0 conn(0x7f4b4c1439a0 0x7f4b4c0cfee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:39.661 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:39.657+0000 7f4b4b7fe640 1 -- 172.21.15.167:0/1508249577 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 174675910 0 0) 0x7f4b4c0d0420 con 0x7f4b4c0cf820 2024-05-11T08:06:39.661 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:39.657+0000 7f4b4b7fe640 1 -- 172.21.15.167:0/1508249577 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4b3c003520 con 0x7f4b4c0cf820 2024-05-11T08:06:39.661 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:39.657+0000 7f4b4b7fe640 1 -- 172.21.15.167:0/1508249577 <== mon.0 v1:172.21.15.167:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3253896457 0 0) 0x7f4b3c003520 con 0x7f4b4c0cf820 2024-05-11T08:06:39.661 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:39.657+0000 7f4b4b7fe640 1 -- 172.21.15.167:0/1508249577 >> v2:172.21.15.204:3300/0 conn(0x7f4b4c1439a0 msgr2=0x7f4b4c0cfee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:39.661 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:39.657+0000 7f4b4b7fe640 1 --2- 172.21.15.167:0/1508249577 >> v2:172.21.15.204:3300/0 conn(0x7f4b4c1439a0 0x7f4b4c0cfee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:39.661 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:39.657+0000 7f4b4b7fe640 1 -- 172.21.15.167:0/1508249577 >> v1:172.21.15.204:6789/0 conn(0x7f4b4c0b59f0 legacy=0x7f4b4c073220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:39.661 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:39.657+0000 7f4b4b7fe640 1 -- 172.21.15.167:0/1508249577 --> v1:172.21.15.167:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b4c0b4210 con 0x7f4b4c0cf820 2024-05-11T08:06:39.661 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:39.658+0000 7f4b4b7fe640 1 -- 172.21.15.167:0/1508249577 <== mon.0 v1:172.21.15.167:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3602277846 0 0) 0x7f4b380016f0 con 0x7f4b4c0cf820 2024-05-11T08:06:39.661 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:39.658+0000 7f4b4b7fe640 1 -- 172.21.15.167:0/1508249577 >> v1:172.21.15.167:6789/0 conn(0x7f4b4c0cf820 legacy=0x7f4b4c0b52e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:39.661 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:39.658+0000 7f4b4b7fe640 1 --2- 172.21.15.167:0/1508249577 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4b3c003a50 0x7f4b3c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:39.661 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:39.658+0000 7f4b4b7fe640 1 --2- 172.21.15.167:0/1508249577 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4b3c004800 0x7f4b3c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:39.661 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:39.658+0000 7f4b4b7fe640 1 --2- 172.21.15.167:0/1508249577 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4b3c0051c0 0x7f4b3c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:39.661 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:39.658+0000 7f4b4b7fe640 1 -- 172.21.15.167:0/1508249577 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4b4c0b3ce0 con 0x7f4b3c004800 2024-05-11T08:06:39.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:39.658+0000 7f4b4b7fe640 1 -- 172.21.15.167:0/1508249577 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4b3c00c160 con 0x7f4b3c0051c0 2024-05-11T08:06:39.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:39.658+0000 7f4b4b7fe640 1 -- 172.21.15.167:0/1508249577 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f4b3c00c360 con 0x7f4b3c003a50 2024-05-11T08:06:39.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:39.658+0000 7f4b4bfff640 1 --2- 172.21.15.167:0/1508249577 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4b3c004800 0x7f4b3c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:39.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:39.658+0000 7f4b50db1640 1 --2- 172.21.15.167:0/1508249577 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4b3c003a50 0x7f4b3c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:39.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:39.658+0000 7f4b515b2640 1 --2- 172.21.15.167:0/1508249577 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4b3c0051c0 0x7f4b3c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:39.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:39.658+0000 7f4b4bfff640 1 -- 172.21.15.167:0/1508249577 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4b3c003a50 msgr2=0x7f4b3c003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:39.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:39.658+0000 7f4b4bfff640 1 --2- 172.21.15.167:0/1508249577 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4b3c003a50 0x7f4b3c003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:39.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:39.658+0000 7f4b4bfff640 1 -- 172.21.15.167:0/1508249577 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4b3c0051c0 msgr2=0x7f4b3c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:39.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:39.658+0000 7f4b4bfff640 1 --2- 172.21.15.167:0/1508249577 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4b3c0051c0 0x7f4b3c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:39.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:39.658+0000 7f4b4bfff640 1 -- 172.21.15.167:0/1508249577 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b3c003520 con 0x7f4b3c004800 2024-05-11T08:06:39.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:39.658+0000 7f4b50db1640 1 --2- 172.21.15.167:0/1508249577 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4b3c003a50 0x7f4b3c003e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T08:06:39.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:39.658+0000 7f4b515b2640 1 --2- 172.21.15.167:0/1508249577 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4b3c0051c0 0x7f4b3c00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T08:06:39.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:39.658+0000 7f4b515b2640 1 -- 172.21.15.167:0/1508249577 reap_dead start 2024-05-11T08:06:39.663 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:39.659+0000 7f4b4bfff640 1 --2- 172.21.15.167:0/1508249577 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4b3c004800 0x7f4b3c004c80 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f4b38001050 tx=0x7f4b3c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=898402ac4d892251 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:39.663 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:39.659+0000 7f4b4b7fe640 1 -- 172.21.15.167:0/1508249577 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b38007d10 con 0x7f4b3c004800 2024-05-11T08:06:39.663 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:39.659+0000 7f4b4b7fe640 1 -- 172.21.15.167:0/1508249577 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b38005ad0 con 0x7f4b3c004800 2024-05-11T08:06:39.663 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:39.659+0000 7f4b4b7fe640 1 -- 172.21.15.167:0/1508249577 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b38007070 con 0x7f4b3c004800 2024-05-11T08:06:39.663 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:39.659+0000 7f4b53015640 1 -- 172.21.15.167:0/1508249577 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4b3c004800 msgr2=0x7f4b3c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:39.663 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:39.659+0000 7f4b53015640 1 --2- 172.21.15.167:0/1508249577 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4b3c004800 0x7f4b3c004c80 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f4b38001050 tx=0x7f4b3c005830 comp rx=0 tx=0).stop 2024-05-11T08:06:39.663 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:39.659+0000 7f4b53015640 1 -- 172.21.15.167:0/1508249577 shutdown_connections 2024-05-11T08:06:39.663 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:39.659+0000 7f4b53015640 1 --2- 172.21.15.167:0/1508249577 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4b3c004800 0x7f4b3c004c80 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:39.663 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:39.659+0000 7f4b53015640 1 -- 172.21.15.167:0/1508249577 >> 172.21.15.167:0/1508249577 conn(0x7f4b4c0d0c70 msgr2=0x7f4b4c0d1080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:39.663 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:39.659+0000 7f4b53015640 1 -- 172.21.15.167:0/1508249577 shutdown_connections 2024-05-11T08:06:39.663 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:39.659+0000 7f4b53015640 1 -- 172.21.15.167:0/1508249577 wait complete. 2024-05-11T08:06:39.666 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:39.661+0000 7f4b53015640 1 Processor -- start 2024-05-11T08:06:39.666 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:39.661+0000 7f4b53015640 1 -- start start 2024-05-11T08:06:39.666 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:39.661+0000 7f4b53015640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4b3c004800 0x7f4b4c15d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:39.666 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:39.661+0000 7f4b53015640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4b4c0aaff0 0x7f4b4c0aa620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:39.666 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:39.661+0000 7f4b53015640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4b4c13e8a0 0x7f4b4c13ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:39.667 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:39.661+0000 7f4b53015640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4b4c0b4ab0 con 0x7f4b4c13e8a0 2024-05-11T08:06:39.667 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:39.661+0000 7f4b53015640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4b4c0b47d0 con 0x7f4b4c0aaff0 2024-05-11T08:06:39.667 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:39.661+0000 7f4b53015640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f4b4c0cfee0 con 0x7f4b3c004800 2024-05-11T08:06:39.667 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:39.662+0000 7f4b515b2640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4b4c13e8a0 0x7f4b4c13ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:39.667 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:39.662+0000 7f4b515b2640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4b4c13e8a0 0x7f4b4c13ed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:34904/0 (socket says 172.21.15.167:34904) 2024-05-11T08:06:39.667 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:39.662+0000 7f4b515b2640 1 -- 172.21.15.167:0/50582605 learned_addr learned my addr 172.21.15.167:0/50582605 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:39.667 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:39.662+0000 7f4b50db1640 1 --2- 172.21.15.167:0/50582605 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4b3c004800 0x7f4b4c15d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:39.667 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:39.662+0000 7f4b4bfff640 1 --2- 172.21.15.167:0/50582605 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4b4c0aaff0 0x7f4b4c0aa620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:39.667 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:39.662+0000 7f4b515b2640 1 -- 172.21.15.167:0/50582605 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4b3c004800 msgr2=0x7f4b4c15d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:39.667 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:39.662+0000 7f4b515b2640 1 --2- 172.21.15.167:0/50582605 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4b3c004800 0x7f4b4c15d510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:39.667 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:39.662+0000 7f4b515b2640 1 -- 172.21.15.167:0/50582605 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4b4c0aaff0 msgr2=0x7f4b4c0aa620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:39.667 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:39.662+0000 7f4b515b2640 1 --2- 172.21.15.167:0/50582605 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4b4c0aaff0 0x7f4b4c0aa620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:39.667 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:39.662+0000 7f4b515b2640 1 -- 172.21.15.167:0/50582605 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b4c0ab560 con 0x7f4b4c13e8a0 2024-05-11T08:06:39.667 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:39.662+0000 7f4b515b2640 1 --2- 172.21.15.167:0/50582605 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4b4c13e8a0 0x7f4b4c13ed20 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c0d1ab0 tx=0x7f4b4400ead0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:39.668 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:39.662+0000 7f4b497fa640 1 -- 172.21.15.167:0/50582605 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b44003b80 con 0x7f4b4c13e8a0 2024-05-11T08:06:39.668 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:39.662+0000 7f4b497fa640 1 -- 172.21.15.167:0/50582605 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b44003d40 con 0x7f4b4c13e8a0 2024-05-11T08:06:39.668 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:39.662+0000 7f4b53015640 1 -- 172.21.15.167:0/50582605 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b4c0ab840 con 0x7f4b4c13e8a0 2024-05-11T08:06:39.668 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:39.662+0000 7f4b497fa640 1 -- 172.21.15.167:0/50582605 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b44006b50 con 0x7f4b4c13e8a0 2024-05-11T08:06:39.668 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:39.662+0000 7f4b53015640 1 -- 172.21.15.167:0/50582605 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b4c0abb20 con 0x7f4b4c13e8a0 2024-05-11T08:06:39.668 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:39.663+0000 7f4b53015640 1 -- 172.21.15.167:0/50582605 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b4c069590 con 0x7f4b4c13e8a0 2024-05-11T08:06:39.668 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:39.664+0000 7f4b497fa640 1 -- 172.21.15.167:0/50582605 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f4b4401a020 con 0x7f4b4c13e8a0 2024-05-11T08:06:39.668 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:39.665+0000 7f4b497fa640 1 --2- 172.21.15.167:0/50582605 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f4b300762d0 0x7f4b30078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:39.668 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:39.665+0000 7f4b50db1640 1 --2- 172.21.15.167:0/50582605 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f4b300762d0 0x7f4b30078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:39.668 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:39.665+0000 7f4b497fa640 1 -- 172.21.15.167:0/50582605 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6383+0+0 (secure 0 0 0) 0x7f4b44096cb0 con 0x7f4b4c13e8a0 2024-05-11T08:06:39.668 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:39.665+0000 7f4b50db1640 1 --2- 172.21.15.167:0/50582605 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f4b300762d0 0x7f4b30078790 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f4b40009bd0 tx=0x7f4b40008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:39.669 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:39.669+0000 7f4b497fa640 1 -- 172.21.15.167:0/50582605 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f4b44060400 con 0x7f4b4c13e8a0 2024-05-11T08:06:39.854 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:39.854+0000 7f4b53015640 1 -- 172.21.15.167:0/50582605 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f4b4c0b2780 con 0x7f4b4c13e8a0 2024-05-11T08:06:40.537 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:40.537+0000 7f4b497fa640 1 -- 172.21.15.167:0/50582605 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v32) ==== 169+0+0 (secure 0 0 0) 0x7f4b4405fd00 con 0x7f4b4c13e8a0 2024-05-11T08:06:40.538 INFO:teuthology.orchestra.run.smithi167.stderr:pool 'cephfs_data' removed 2024-05-11T08:06:40.541 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:40.541+0000 7f4b53015640 1 -- 172.21.15.167:0/50582605 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f4b300762d0 msgr2=0x7f4b30078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:40.542 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:40.541+0000 7f4b53015640 1 --2- 172.21.15.167:0/50582605 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f4b300762d0 0x7f4b30078790 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f4b40009bd0 tx=0x7f4b40008040 comp rx=0 tx=0).stop 2024-05-11T08:06:40.542 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:40.541+0000 7f4b53015640 1 -- 172.21.15.167:0/50582605 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4b4c13e8a0 msgr2=0x7f4b4c13ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:40.542 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:40.541+0000 7f4b53015640 1 --2- 172.21.15.167:0/50582605 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4b4c13e8a0 0x7f4b4c13ed20 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c0d1ab0 tx=0x7f4b4400ead0 comp rx=0 tx=0).stop 2024-05-11T08:06:40.542 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:40.541+0000 7f4b53015640 1 -- 172.21.15.167:0/50582605 shutdown_connections 2024-05-11T08:06:40.542 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:40.541+0000 7f4b53015640 1 --2- 172.21.15.167:0/50582605 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f4b300762d0 0x7f4b30078790 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:40.542 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:40.541+0000 7f4b53015640 1 --2- 172.21.15.167:0/50582605 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4b4c13e8a0 0x7f4b4c13ed20 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:40.542 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:40.541+0000 7f4b53015640 1 --2- 172.21.15.167:0/50582605 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4b4c0aaff0 0x7f4b4c0aa620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:40.542 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:40.541+0000 7f4b53015640 1 --2- 172.21.15.167:0/50582605 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4b3c004800 0x7f4b4c15d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:40.542 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:40.541+0000 7f4b53015640 1 -- 172.21.15.167:0/50582605 >> 172.21.15.167:0/50582605 conn(0x7f4b4c0d0c70 msgr2=0x7f4b4c073220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:40.542 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:40.542+0000 7f4b53015640 1 -- 172.21.15.167:0/50582605 shutdown_connections 2024-05-11T08:06:40.542 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:40.542+0000 7f4b53015640 1 -- 172.21.15.167:0/50582605 wait complete. 2024-05-11T08:06:40.555 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-05-11T08:06:40.659 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:40.659+0000 7f6257074640 1 Processor -- start 2024-05-11T08:06:40.659 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:40.659+0000 7f6257074640 1 -- start start 2024-05-11T08:06:40.663 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:40.660+0000 7f6257074640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f625006fc30 0x7f6250070090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:40.663 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:40.660+0000 7f6257074640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f625006eb90 con 0x7f6250072150 2024-05-11T08:06:40.663 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:40.660+0000 7f6257074640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f625006ee70 con 0x7f6250142740 2024-05-11T08:06:40.663 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:40.660+0000 7f6257074640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f625006f7a0 con 0x7f625006fc30 2024-05-11T08:06:40.663 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:40.660+0000 7f6255611640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f6250072150 0x7f625015fcb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:49622/0 (socket says 172.21.15.167:49622) 2024-05-11T08:06:40.663 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:40.660+0000 7f6255611640 1 -- 172.21.15.167:0/3492267628 learned_addr learned my addr 172.21.15.167:0/3492267628 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T08:06:40.663 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:40.661+0000 7f624f7fe640 1 -- 172.21.15.167:0/3492267628 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4044378376 0 0) 0x7f625006eb90 con 0x7f6250072150 2024-05-11T08:06:40.663 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:40.661+0000 7f624ffff640 1 --2- 172.21.15.167:0/3492267628 >> v2:172.21.15.204:3300/0 conn(0x7f625006fc30 0x7f6250070090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:40.664 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:40.661+0000 7f624f7fe640 1 -- 172.21.15.167:0/3492267628 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f623c003520 con 0x7f6250072150 2024-05-11T08:06:40.664 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:40.661+0000 7f624f7fe640 1 -- 172.21.15.167:0/3492267628 <== mon.0 v1:172.21.15.167:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3268493660 0 0) 0x7f623c003520 con 0x7f6250072150 2024-05-11T08:06:40.664 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:40.661+0000 7f624f7fe640 1 -- 172.21.15.167:0/3492267628 >> v2:172.21.15.204:3300/0 conn(0x7f625006fc30 msgr2=0x7f6250070090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:40.664 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:40.661+0000 7f624f7fe640 1 --2- 172.21.15.167:0/3492267628 >> v2:172.21.15.204:3300/0 conn(0x7f625006fc30 0x7f6250070090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:40.664 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:40.661+0000 7f624f7fe640 1 -- 172.21.15.167:0/3492267628 >> v1:172.21.15.204:6789/0 conn(0x7f6250142740 legacy=0x7f62500b4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:40.664 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:40.661+0000 7f624f7fe640 1 -- 172.21.15.167:0/3492267628 --> v1:172.21.15.167:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f625006f150 con 0x7f6250072150 2024-05-11T08:06:40.664 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:40.661+0000 7f624f7fe640 1 -- 172.21.15.167:0/3492267628 <== mon.0 v1:172.21.15.167:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3602277846 0 0) 0x7f6248001a70 con 0x7f6250072150 2024-05-11T08:06:40.664 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:40.661+0000 7f624f7fe640 1 -- 172.21.15.167:0/3492267628 >> v1:172.21.15.167:6789/0 conn(0x7f6250072150 legacy=0x7f625015fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:40.664 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:40.661+0000 7f624f7fe640 1 --2- 172.21.15.167:0/3492267628 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f623c003a50 0x7f623c003e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:40.664 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:40.661+0000 7f624f7fe640 1 --2- 172.21.15.167:0/3492267628 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f623c004800 0x7f623c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:40.664 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:40.661+0000 7f624f7fe640 1 --2- 172.21.15.167:0/3492267628 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f623c0052e0 0x7f623c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:40.664 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:40.661+0000 7f624f7fe640 1 -- 172.21.15.167:0/3492267628 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f625006f7a0 con 0x7f623c003a50 2024-05-11T08:06:40.664 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:40.661+0000 7f624f7fe640 1 -- 172.21.15.167:0/3492267628 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f623c00c1b0 con 0x7f623c004800 2024-05-11T08:06:40.664 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:40.661+0000 7f624f7fe640 1 -- 172.21.15.167:0/3492267628 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f623c00c3b0 con 0x7f623c0052e0 2024-05-11T08:06:40.665 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:40.661+0000 7f6254e10640 1 --2- 172.21.15.167:0/3492267628 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f623c003a50 0x7f623c003e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:40.665 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:40.661+0000 7f624ffff640 1 --2- 172.21.15.167:0/3492267628 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f623c004800 0x7f623c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:40.665 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:40.661+0000 7f6255611640 1 --2- 172.21.15.167:0/3492267628 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f623c0052e0 0x7f623c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:40.665 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:40.661+0000 7f6254e10640 1 -- 172.21.15.167:0/3492267628 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f623c0052e0 msgr2=0x7f623c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:40.665 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:40.661+0000 7f6254e10640 1 --2- 172.21.15.167:0/3492267628 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f623c0052e0 0x7f623c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:40.665 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:40.661+0000 7f6254e10640 1 -- 172.21.15.167:0/3492267628 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f623c004800 msgr2=0x7f623c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:40.665 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:40.661+0000 7f6254e10640 1 --2- 172.21.15.167:0/3492267628 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f623c004800 0x7f623c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:40.665 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:40.661+0000 7f6254e10640 1 -- 172.21.15.167:0/3492267628 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f623c003520 con 0x7f623c003a50 2024-05-11T08:06:40.665 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:40.661+0000 7f6255611640 1 -- 172.21.15.167:0/3492267628 reap_dead start 2024-05-11T08:06:40.665 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:40.662+0000 7f6254e10640 1 --2- 172.21.15.167:0/3492267628 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f623c003a50 0x7f623c003e90 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f6238001790 tx=0x7f623c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe5de46a130a5d57 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:40.665 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:40.662+0000 7f624f7fe640 1 -- 172.21.15.167:0/3492267628 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62380013b0 con 0x7f623c003a50 2024-05-11T08:06:40.665 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:40.662+0000 7f624f7fe640 1 -- 172.21.15.167:0/3492267628 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62380052d0 con 0x7f623c003a50 2024-05-11T08:06:40.665 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:40.662+0000 7f624f7fe640 1 -- 172.21.15.167:0/3492267628 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f623802aaa0 con 0x7f623c003a50 2024-05-11T08:06:40.665 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:40.663+0000 7f6257074640 1 -- 172.21.15.167:0/3492267628 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f623c003a50 msgr2=0x7f623c003e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:40.665 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:40.663+0000 7f6257074640 1 --2- 172.21.15.167:0/3492267628 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f623c003a50 0x7f623c003e90 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f6238001790 tx=0x7f623c005830 comp rx=0 tx=0).stop 2024-05-11T08:06:40.666 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:40.663+0000 7f6257074640 1 -- 172.21.15.167:0/3492267628 shutdown_connections 2024-05-11T08:06:40.666 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:40.663+0000 7f6257074640 1 --2- 172.21.15.167:0/3492267628 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f623c003a50 0x7f623c003e90 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:40.666 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:40.663+0000 7f6257074640 1 -- 172.21.15.167:0/3492267628 >> 172.21.15.167:0/3492267628 conn(0x7f62500d0c70 msgr2=0x7f62500d1080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:40.666 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:40.663+0000 7f6257074640 1 -- 172.21.15.167:0/3492267628 shutdown_connections 2024-05-11T08:06:40.666 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:40.663+0000 7f6257074640 1 -- 172.21.15.167:0/3492267628 wait complete. 2024-05-11T08:06:40.668 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:40.664+0000 7f6257074640 1 Processor -- start 2024-05-11T08:06:40.668 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:40.664+0000 7f6257074640 1 -- start start 2024-05-11T08:06:40.669 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:40.665+0000 7f6257074640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f623c003a50 0x7f62501300d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:40.669 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:40.665+0000 7f6257074640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f625015ba90 0x7f6250072d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:40.669 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:40.665+0000 7f6257074640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f62500aa410 0x7f62500aa890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:40.669 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:40.665+0000 7f6257074640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f62501609a0 con 0x7f625015ba90 2024-05-11T08:06:40.669 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:40.665+0000 7f6257074640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6250142b20 con 0x7f623c003a50 2024-05-11T08:06:40.669 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:40.665+0000 7f6257074640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f6250160b20 con 0x7f62500aa410 2024-05-11T08:06:40.669 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:40.665+0000 7f624ffff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f625015ba90 0x7f6250072d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:40.669 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:40.665+0000 7f624ffff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f625015ba90 0x7f6250072d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:34918/0 (socket says 172.21.15.167:34918) 2024-05-11T08:06:40.669 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:40.665+0000 7f624ffff640 1 -- 172.21.15.167:0/2006964993 learned_addr learned my addr 172.21.15.167:0/2006964993 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:40.669 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:40.665+0000 7f624ffff640 1 -- 172.21.15.167:0/2006964993 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f62500aa410 msgr2=0x7f62500aa890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:40.669 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:40.665+0000 7f624ffff640 1 --2- 172.21.15.167:0/2006964993 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f62500aa410 0x7f62500aa890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:40.669 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:40.665+0000 7f624ffff640 1 -- 172.21.15.167:0/2006964993 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f623c003a50 msgr2=0x7f62501300d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:40.669 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:40.665+0000 7f624ffff640 1 --2- 172.21.15.167:0/2006964993 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f623c003a50 0x7f62501300d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:40.670 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:40.665+0000 7f624ffff640 1 -- 172.21.15.167:0/2006964993 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6250132160 con 0x7f625015ba90 2024-05-11T08:06:40.670 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:40.665+0000 7f624ffff640 1 --2- 172.21.15.167:0/2006964993 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f625015ba90 0x7f6250072d40 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f62400118c0 tx=0x7f6240011d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:40.670 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:40.666+0000 7f624d7fa640 1 -- 172.21.15.167:0/2006964993 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6240019070 con 0x7f625015ba90 2024-05-11T08:06:40.670 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:40.666+0000 7f624d7fa640 1 -- 172.21.15.167:0/2006964993 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6240004240 con 0x7f625015ba90 2024-05-11T08:06:40.670 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:40.666+0000 7f6257074640 1 -- 172.21.15.167:0/2006964993 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6250132440 con 0x7f625015ba90 2024-05-11T08:06:40.670 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:40.666+0000 7f624d7fa640 1 -- 172.21.15.167:0/2006964993 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f624000f820 con 0x7f625015ba90 2024-05-11T08:06:40.670 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:40.666+0000 7f6257074640 1 -- 172.21.15.167:0/2006964993 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6250132a00 con 0x7f625015ba90 2024-05-11T08:06:40.670 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:40.667+0000 7f6257074640 1 -- 172.21.15.167:0/2006964993 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6250069590 con 0x7f625015ba90 2024-05-11T08:06:40.670 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:40.667+0000 7f624d7fa640 1 -- 172.21.15.167:0/2006964993 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f624001c070 con 0x7f625015ba90 2024-05-11T08:06:40.670 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:40.668+0000 7f624d7fa640 1 --2- 172.21.15.167:0/2006964993 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f6230076300 0x7f62300787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:40.670 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:40.668+0000 7f6254e10640 1 --2- 172.21.15.167:0/2006964993 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f6230076300 0x7f62300787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:40.670 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:40.668+0000 7f624d7fa640 1 -- 172.21.15.167:0/2006964993 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5853+0+0 (secure 0 0 0) 0x7f6240097a00 con 0x7f625015ba90 2024-05-11T08:06:40.670 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:40.668+0000 7f6254e10640 1 --2- 172.21.15.167:0/2006964993 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f6230076300 0x7f62300787c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f6238032000 tx=0x7f623802c000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:40.672 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:40.672+0000 7f624d7fa640 1 -- 172.21.15.167:0/2006964993 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f6240061250 con 0x7f625015ba90 2024-05-11T08:06:40.863 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:40.863+0000 7f6257074640 1 -- 172.21.15.167:0/2006964993 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7f625006e2c0 con 0x7f625015ba90 2024-05-11T08:06:41.539 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:41.539+0000 7f624d7fa640 1 -- 172.21.15.167:0/2006964993 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v33) ==== 178+0+0 (secure 0 0 0) 0x7f6240060b50 con 0x7f625015ba90 2024-05-11T08:06:41.539 INFO:teuthology.orchestra.run.smithi167.stderr:pool 'cephfs_data_ec' removed 2024-05-11T08:06:41.543 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:41.543+0000 7f6257074640 1 -- 172.21.15.167:0/2006964993 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f6230076300 msgr2=0x7f62300787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:41.544 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:41.543+0000 7f6257074640 1 --2- 172.21.15.167:0/2006964993 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f6230076300 0x7f62300787c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f6238032000 tx=0x7f623802c000 comp rx=0 tx=0).stop 2024-05-11T08:06:41.544 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:41.543+0000 7f6257074640 1 -- 172.21.15.167:0/2006964993 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f625015ba90 msgr2=0x7f6250072d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:41.544 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:41.543+0000 7f6257074640 1 --2- 172.21.15.167:0/2006964993 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f625015ba90 0x7f6250072d40 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f62400118c0 tx=0x7f6240011d80 comp rx=0 tx=0).stop 2024-05-11T08:06:41.545 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:41.543+0000 7f6257074640 1 -- 172.21.15.167:0/2006964993 shutdown_connections 2024-05-11T08:06:41.545 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:41.543+0000 7f6257074640 1 --2- 172.21.15.167:0/2006964993 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f6230076300 0x7f62300787c0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:41.545 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:41.543+0000 7f6257074640 1 --2- 172.21.15.167:0/2006964993 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f62500aa410 0x7f62500aa890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:41.545 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:41.543+0000 7f6257074640 1 --2- 172.21.15.167:0/2006964993 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f625015ba90 0x7f6250072d40 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:41.545 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:41.543+0000 7f6257074640 1 --2- 172.21.15.167:0/2006964993 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f623c003a50 0x7f62501300d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:41.545 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:41.543+0000 7f6257074640 1 -- 172.21.15.167:0/2006964993 >> 172.21.15.167:0/2006964993 conn(0x7f62500d0c70 msgr2=0x7f625015e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:41.545 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:41.544+0000 7f6257074640 1 -- 172.21.15.167:0/2006964993 shutdown_connections 2024-05-11T08:06:41.545 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:41.545+0000 7f6257074640 1 -- 172.21.15.167:0/2006964993 wait complete. 2024-05-11T08:06:41.559 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-11T08:06:41.559 INFO:tasks.ceph.mds.a:Stopping old one... 2024-05-11T08:06:41.560 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-11T08:06:41.560 INFO:teuthology.orchestra.run:waiting for 300 2024-05-11T08:06:41.560 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-11T08:06:41.560 INFO:tasks.ceph.mds.c:Stopping old one... 2024-05-11T08:06:41.560 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-11T08:06:41.560 INFO:teuthology.orchestra.run:waiting for 300 2024-05-11T08:06:41.560 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-11T08:06:41.561 INFO:tasks.ceph.mds.b:Stopping old one... 2024-05-11T08:06:41.561 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-11T08:06:41.561 INFO:teuthology.orchestra.run:waiting for 300 2024-05-11T08:06:41.561 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-11T08:06:41.561 INFO:tasks.ceph.mds.d:Stopping old one... 2024-05-11T08:06:41.561 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-11T08:06:41.561 INFO:teuthology.orchestra.run:waiting for 300 2024-05-11T08:06:41.569 INFO:tasks.ceph.mds.a:Stopped 2024-05-11T08:06:41.569 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-11T08:06:41.570 INFO:tasks.ceph.mds.c:Stopped 2024-05-11T08:06:41.570 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-11T08:06:41.572 INFO:tasks.ceph.mds.b:Stopped 2024-05-11T08:06:41.573 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-11T08:06:41.573 INFO:tasks.ceph.mds.d:Stopped 2024-05-11T08:06:41.573 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-11T08:06:41.612 INFO:tasks.ceph.mds.a:Started 2024-05-11T08:06:41.612 DEBUG:teuthology.parallel:result is None 2024-05-11T08:06:41.613 INFO:tasks.ceph.mds.c:Started 2024-05-11T08:06:41.613 DEBUG:teuthology.parallel:result is None 2024-05-11T08:06:41.616 INFO:tasks.ceph.mds.b:Started 2024-05-11T08:06:41.616 DEBUG:teuthology.parallel:result is None 2024-05-11T08:06:41.616 INFO:tasks.ceph.mds.d:Started 2024-05-11T08:06:41.617 DEBUG:teuthology.parallel:result is None 2024-05-11T08:06:41.617 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-05-11T08:06:41.689 INFO:tasks.ceph.mds.a.smithi167.stdout:starting mds.a at 2024-05-11T08:06:41.689 INFO:tasks.ceph.mds.c.smithi167.stdout:starting mds.c at 2024-05-11T08:06:41.696 INFO:tasks.ceph.mds.d.smithi204.stdout:starting mds.d at 2024-05-11T08:06:41.696 INFO:tasks.ceph.mds.b.smithi204.stdout:starting mds.b at 2024-05-11T08:06:41.722 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:41.722+0000 7f7d368ea640 1 Processor -- start 2024-05-11T08:06:41.722 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:41.723+0000 7f7d368ea640 1 -- start start 2024-05-11T08:06:41.726 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:41.723+0000 7f7d368ea640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f7d300cb6c0 0x7f7d3006dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:41.726 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:41.723+0000 7f7d368ea640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d300d1190 con 0x7f7d300d2cf0 2024-05-11T08:06:41.726 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:41.723+0000 7f7d368ea640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d3006d500 con 0x7f7d301598d0 2024-05-11T08:06:41.726 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:41.723+0000 7f7d368ea640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f7d3006d7e0 con 0x7f7d300cb6c0 2024-05-11T08:06:41.726 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:41.724+0000 7f7d2f7fe640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f7d300d2cf0 0x7f7d300cafb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:49660/0 (socket says 172.21.15.167:49660) 2024-05-11T08:06:41.726 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:41.724+0000 7f7d2f7fe640 1 -- 172.21.15.167:0/4091378117 learned_addr learned my addr 172.21.15.167:0/4091378117 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T08:06:41.726 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:41.724+0000 7f7d34e87640 1 --2- 172.21.15.167:0/4091378117 >> v2:172.21.15.204:3300/0 conn(0x7f7d300cb6c0 0x7f7d3006dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:41.726 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:41.724+0000 7f7d2effd640 1 -- 172.21.15.167:0/4091378117 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3116575113 0 0) 0x7f7d300d1190 con 0x7f7d300d2cf0 2024-05-11T08:06:41.727 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:41.724+0000 7f7d2effd640 1 -- 172.21.15.167:0/4091378117 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7d18003520 con 0x7f7d300d2cf0 2024-05-11T08:06:41.727 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:41.724+0000 7f7d2effd640 1 -- 172.21.15.167:0/4091378117 <== mon.0 v1:172.21.15.167:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1725644177 0 0) 0x7f7d18003520 con 0x7f7d300d2cf0 2024-05-11T08:06:41.727 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:41.724+0000 7f7d2effd640 1 -- 172.21.15.167:0/4091378117 >> v2:172.21.15.204:3300/0 conn(0x7f7d300cb6c0 msgr2=0x7f7d3006dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:41.727 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:41.724+0000 7f7d2effd640 1 --2- 172.21.15.167:0/4091378117 >> v2:172.21.15.204:3300/0 conn(0x7f7d300cb6c0 0x7f7d3006dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:41.727 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:41.724+0000 7f7d2effd640 1 -- 172.21.15.167:0/4091378117 >> v1:172.21.15.204:6789/0 conn(0x7f7d301598d0 legacy=0x7f7d300d0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:41.727 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:41.724+0000 7f7d2effd640 1 -- 172.21.15.167:0/4091378117 --> v1:172.21.15.167:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d3006dac0 con 0x7f7d300d2cf0 2024-05-11T08:06:41.727 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:41.724+0000 7f7d2effd640 1 -- 172.21.15.167:0/4091378117 <== mon.0 v1:172.21.15.167:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3602277846 0 0) 0x7f7d240016f0 con 0x7f7d300d2cf0 2024-05-11T08:06:41.727 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:41.724+0000 7f7d2effd640 1 -- 172.21.15.167:0/4091378117 >> v1:172.21.15.167:6789/0 conn(0x7f7d300d2cf0 legacy=0x7f7d300cafb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:41.727 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:41.724+0000 7f7d2effd640 1 --2- 172.21.15.167:0/4091378117 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d18003a50 0x7f7d18003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:41.727 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:41.724+0000 7f7d2effd640 1 --2- 172.21.15.167:0/4091378117 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d18004800 0x7f7d18004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:41.727 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:41.724+0000 7f7d2effd640 1 --2- 172.21.15.167:0/4091378117 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7d180051c0 0x7f7d1800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:41.727 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:41.724+0000 7f7d2effd640 1 -- 172.21.15.167:0/4091378117 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7d3006d7e0 con 0x7f7d18003a50 2024-05-11T08:06:41.727 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:41.724+0000 7f7d2effd640 1 -- 172.21.15.167:0/4091378117 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7d1800c160 con 0x7f7d18004800 2024-05-11T08:06:41.727 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:41.724+0000 7f7d2effd640 1 -- 172.21.15.167:0/4091378117 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f7d1800c360 con 0x7f7d180051c0 2024-05-11T08:06:41.728 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:41.724+0000 7f7d2ffff640 1 --2- 172.21.15.167:0/4091378117 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d18003a50 0x7f7d18003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:41.728 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:41.724+0000 7f7d2f7fe640 1 --2- 172.21.15.167:0/4091378117 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d18004800 0x7f7d18004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:41.728 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:41.724+0000 7f7d34e87640 1 --2- 172.21.15.167:0/4091378117 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7d180051c0 0x7f7d1800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:41.728 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:41.725+0000 7f7d2ffff640 1 -- 172.21.15.167:0/4091378117 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7d180051c0 msgr2=0x7f7d1800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:41.728 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:41.725+0000 7f7d2ffff640 1 --2- 172.21.15.167:0/4091378117 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7d180051c0 0x7f7d1800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:41.728 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:41.725+0000 7f7d2ffff640 1 -- 172.21.15.167:0/4091378117 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d18004800 msgr2=0x7f7d18004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:41.728 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:41.725+0000 7f7d2ffff640 1 --2- 172.21.15.167:0/4091378117 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d18004800 0x7f7d18004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:41.728 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:41.725+0000 7f7d34e87640 1 -- 172.21.15.167:0/4091378117 reap_dead start 2024-05-11T08:06:41.728 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:41.725+0000 7f7d2ffff640 1 -- 172.21.15.167:0/4091378117 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d18003520 con 0x7f7d18003a50 2024-05-11T08:06:41.728 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:41.725+0000 7f7d2ffff640 1 --2- 172.21.15.167:0/4091378117 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d18003a50 0x7f7d18003e70 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f7d20001790 tx=0x7f7d18005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3828c58218db5a7b server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:41.728 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:41.725+0000 7f7d2effd640 1 -- 172.21.15.167:0/4091378117 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d200013b0 con 0x7f7d18003a50 2024-05-11T08:06:41.728 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:41.725+0000 7f7d2effd640 1 -- 172.21.15.167:0/4091378117 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d200052d0 con 0x7f7d18003a50 2024-05-11T08:06:41.728 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:41.725+0000 7f7d2effd640 1 -- 172.21.15.167:0/4091378117 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d2002aaa0 con 0x7f7d18003a50 2024-05-11T08:06:41.728 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:41.726+0000 7f7d368ea640 1 -- 172.21.15.167:0/4091378117 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d18003a50 msgr2=0x7f7d18003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:41.728 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:41.726+0000 7f7d368ea640 1 --2- 172.21.15.167:0/4091378117 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d18003a50 0x7f7d18003e70 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f7d20001790 tx=0x7f7d18005830 comp rx=0 tx=0).stop 2024-05-11T08:06:41.729 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:41.726+0000 7f7d368ea640 1 -- 172.21.15.167:0/4091378117 shutdown_connections 2024-05-11T08:06:41.729 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:41.726+0000 7f7d368ea640 1 --2- 172.21.15.167:0/4091378117 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d18003a50 0x7f7d18003e70 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:41.729 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:41.726+0000 7f7d368ea640 1 -- 172.21.15.167:0/4091378117 >> 172.21.15.167:0/4091378117 conn(0x7f7d300d1ba0 msgr2=0x7f7d300d1fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:41.729 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:41.726+0000 7f7d368ea640 1 -- 172.21.15.167:0/4091378117 shutdown_connections 2024-05-11T08:06:41.729 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:41.726+0000 7f7d368ea640 1 -- 172.21.15.167:0/4091378117 wait complete. 2024-05-11T08:06:41.731 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:41.727+0000 7f7d368ea640 1 Processor -- start 2024-05-11T08:06:41.731 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:41.727+0000 7f7d368ea640 1 -- start start 2024-05-11T08:06:41.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:41.728+0000 7f7d368ea640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7d18003a50 0x7f7d300ba8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:41.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:41.728+0000 7f7d368ea640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d30152460 0x7f7d300bae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:41.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:41.728+0000 7f7d368ea640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d3014fcc0 0x7f7d30150140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:41.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:41.728+0000 7f7d368ea640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7d3006d1b0 con 0x7f7d3014fcc0 2024-05-11T08:06:41.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:41.728+0000 7f7d368ea640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7d30159cb0 con 0x7f7d30152460 2024-05-11T08:06:41.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:41.728+0000 7f7d368ea640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f7d300cafb0 con 0x7f7d18003a50 2024-05-11T08:06:41.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:41.728+0000 7f7d34e87640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d3014fcc0 0x7f7d30150140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:41.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:41.728+0000 7f7d34e87640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d3014fcc0 0x7f7d30150140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:34986/0 (socket says 172.21.15.167:34986) 2024-05-11T08:06:41.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:41.728+0000 7f7d34e87640 1 -- 172.21.15.167:0/633662432 learned_addr learned my addr 172.21.15.167:0/633662432 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:41.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:41.728+0000 7f7d2f7fe640 1 --2- 172.21.15.167:0/633662432 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d30152460 0x7f7d300bae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:41.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:41.728+0000 7f7d34e87640 1 -- 172.21.15.167:0/633662432 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7d18003a50 msgr2=0x7f7d300ba8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:41.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:41.729+0000 7f7d2ffff640 1 --2- 172.21.15.167:0/633662432 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7d18003a50 0x7f7d300ba8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:41.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:41.729+0000 7f7d34e87640 1 --2- 172.21.15.167:0/633662432 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7d18003a50 0x7f7d300ba8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:41.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:41.729+0000 7f7d34e87640 1 -- 172.21.15.167:0/633662432 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d30152460 msgr2=0x7f7d300bae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:41.733 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:41.729+0000 7f7d34e87640 1 --2- 172.21.15.167:0/633662432 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d30152460 0x7f7d300bae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:41.733 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:41.729+0000 7f7d34e87640 1 -- 172.21.15.167:0/633662432 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d30150680 con 0x7f7d3014fcc0 2024-05-11T08:06:41.733 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:41.729+0000 7f7d2f7fe640 1 --2- 172.21.15.167:0/633662432 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d30152460 0x7f7d300bae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T08:06:41.733 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:41.729+0000 7f7d2ffff640 1 --2- 172.21.15.167:0/633662432 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7d18003a50 0x7f7d300ba8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T08:06:41.733 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:41.729+0000 7f7d34e87640 1 --2- 172.21.15.167:0/633662432 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d3014fcc0 0x7f7d30150140 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f7d300d2be0 tx=0x7f7d2800f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:41.733 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:41.729+0000 7f7d2cff9640 1 -- 172.21.15.167:0/633662432 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d2800fd80 con 0x7f7d3014fcc0 2024-05-11T08:06:41.733 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:41.729+0000 7f7d368ea640 1 -- 172.21.15.167:0/633662432 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d301f55e0 con 0x7f7d3014fcc0 2024-05-11T08:06:41.733 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:41.729+0000 7f7d2cff9640 1 -- 172.21.15.167:0/633662432 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d2801d070 con 0x7f7d3014fcc0 2024-05-11T08:06:41.733 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:41.729+0000 7f7d2cff9640 1 -- 172.21.15.167:0/633662432 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d28015670 con 0x7f7d3014fcc0 2024-05-11T08:06:41.733 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:41.729+0000 7f7d368ea640 1 -- 172.21.15.167:0/633662432 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d301f5ba0 con 0x7f7d3014fcc0 2024-05-11T08:06:41.733 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:41.730+0000 7f7d368ea640 1 -- 172.21.15.167:0/633662432 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d30069590 con 0x7f7d3014fcc0 2024-05-11T08:06:41.733 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:41.731+0000 7f7d2cff9640 1 -- 172.21.15.167:0/633662432 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f7d28015810 con 0x7f7d3014fcc0 2024-05-11T08:06:41.733 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:41.731+0000 7f7d2cff9640 1 --2- 172.21.15.167:0/633662432 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f7d1c0762d0 0x7f7d1c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:41.733 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:41.731+0000 7f7d2cff9640 1 -- 172.21.15.167:0/633662432 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5335+0+0 (secure 0 0 0) 0x7f7d28097e40 con 0x7f7d3014fcc0 2024-05-11T08:06:41.733 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:41.731+0000 7f7d2ffff640 1 --2- 172.21.15.167:0/633662432 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f7d1c0762d0 0x7f7d1c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:41.733 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:41.732+0000 7f7d2ffff640 1 --2- 172.21.15.167:0/633662432 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f7d1c0762d0 0x7f7d1c078790 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f7d20032000 tx=0x7f7d2002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:41.735 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:41.735+0000 7f7d2cff9640 1 -- 172.21.15.167:0/633662432 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f7d28010040 con 0x7f7d3014fcc0 2024-05-11T08:06:41.927 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:41.926+0000 7f7d368ea640 1 -- 172.21.15.167:0/633662432 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f7d301591f0 con 0x7f7d3014fcc0 2024-05-11T08:06:42.559 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:42.559+0000 7f7d2cff9640 1 -- 172.21.15.167:0/633662432 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v35) ==== 97+0+0 (secure 0 0 0) 0x7f7d28061890 con 0x7f7d3014fcc0 2024-05-11T08:06:42.559 INFO:teuthology.orchestra.run.smithi167.stderr: removed all blocklist entries 2024-05-11T08:06:42.562 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:42.561+0000 7f7d368ea640 1 -- 172.21.15.167:0/633662432 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f7d1c0762d0 msgr2=0x7f7d1c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:42.562 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:42.561+0000 7f7d368ea640 1 --2- 172.21.15.167:0/633662432 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f7d1c0762d0 0x7f7d1c078790 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f7d20032000 tx=0x7f7d2002e000 comp rx=0 tx=0).stop 2024-05-11T08:06:42.562 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:42.561+0000 7f7d368ea640 1 -- 172.21.15.167:0/633662432 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d3014fcc0 msgr2=0x7f7d30150140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:42.562 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:42.561+0000 7f7d368ea640 1 --2- 172.21.15.167:0/633662432 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d3014fcc0 0x7f7d30150140 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f7d300d2be0 tx=0x7f7d2800f580 comp rx=0 tx=0).stop 2024-05-11T08:06:42.562 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:42.561+0000 7f7d368ea640 1 -- 172.21.15.167:0/633662432 shutdown_connections 2024-05-11T08:06:42.562 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:42.561+0000 7f7d368ea640 1 --2- 172.21.15.167:0/633662432 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f7d1c0762d0 0x7f7d1c078790 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:42.562 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:42.561+0000 7f7d368ea640 1 --2- 172.21.15.167:0/633662432 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d3014fcc0 0x7f7d30150140 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:42.562 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:42.561+0000 7f7d368ea640 1 --2- 172.21.15.167:0/633662432 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d30152460 0x7f7d300bae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:42.562 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:42.561+0000 7f7d368ea640 1 --2- 172.21.15.167:0/633662432 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7d18003a50 0x7f7d300ba8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:42.562 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:42.561+0000 7f7d368ea640 1 -- 172.21.15.167:0/633662432 >> 172.21.15.167:0/633662432 conn(0x7f7d300d1ba0 msgr2=0x7f7d300c9fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:42.562 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:42.561+0000 7f7d368ea640 1 -- 172.21.15.167:0/633662432 shutdown_connections 2024-05-11T08:06:42.562 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:42.561+0000 7f7d368ea640 1 -- 172.21.15.167:0/633662432 wait complete. 2024-05-11T08:06:42.574 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-05-11T08:06:42.675 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:42.675+0000 7f5cce540640 1 Processor -- start 2024-05-11T08:06:42.675 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:42.675+0000 7f5cce540640 1 -- start start 2024-05-11T08:06:42.676 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:42.676+0000 7f5cce540640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f5cc80b5e90 0x7f5cc80cfba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:42.676 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:42.676+0000 7f5cce540640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5cc80d24a0 con 0x7f5cc80d00e0 2024-05-11T08:06:42.676 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:42.676+0000 7f5cce540640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5cc80cf3e0 con 0x7f5cc80b8a40 2024-05-11T08:06:42.677 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:42.676+0000 7f5cce540640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f5cc80d2740 con 0x7f5cc80b5e90 2024-05-11T08:06:42.677 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:42.676+0000 7f5cc77fe640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f5cc80b5e90 0x7f5cc80cfba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:42.677 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:42.676+0000 7f5cc77fe640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f5cc80b5e90 0x7f5cc80cfba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:34990/0 (socket says 172.21.15.167:34990) 2024-05-11T08:06:42.677 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:42.676+0000 7f5cccadd640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f5cc80d00e0 0x7f5cc80d0ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:49670/0 (socket says 172.21.15.167:49670) 2024-05-11T08:06:42.677 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:42.676+0000 7f5cc77fe640 1 -- 172.21.15.167:0/2021304736 learned_addr learned my addr 172.21.15.167:0/2021304736 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:42.677 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:42.676+0000 7f5cc77fe640 1 -- 172.21.15.167:0/2021304736 >> v1:172.21.15.204:6789/0 conn(0x7f5cc80b8a40 legacy=0x7f5cc8160540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:42.677 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:42.676+0000 7f5cc77fe640 1 -- 172.21.15.167:0/2021304736 >> v1:172.21.15.167:6789/0 conn(0x7f5cc80d00e0 legacy=0x7f5cc80d0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:42.677 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:42.676+0000 7f5cc6ffd640 1 -- 172.21.15.167:0/2021304736 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 28242017 0 0) 0x7f5cc80d24a0 con 0x7f5cc80d00e0 2024-05-11T08:06:42.677 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:42.676+0000 7f5cc77fe640 1 -- 172.21.15.167:0/2021304736 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cc80d2a20 con 0x7f5cc80b5e90 2024-05-11T08:06:42.678 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:42.677+0000 7f5cc77fe640 1 --2- 172.21.15.167:0/2021304736 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5cc80b5e90 0x7f5cc80cfba0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc009870 tx=0x7f5cbc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bcdb50f4d1fb057 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:42.678 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:42.677+0000 7f5cc6ffd640 1 -- 172.21.15.167:0/2021304736 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cbc037070 con 0x7f5cc80b5e90 2024-05-11T08:06:42.679 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:42.677+0000 7f5cc6ffd640 1 -- 172.21.15.167:0/2021304736 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5cbc032440 con 0x7f5cc80b5e90 2024-05-11T08:06:42.679 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:42.677+0000 7f5cc6ffd640 1 -- 172.21.15.167:0/2021304736 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cbc032810 con 0x7f5cc80b5e90 2024-05-11T08:06:42.679 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:42.678+0000 7f5cce540640 1 -- 172.21.15.167:0/2021304736 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5cc80b5e90 msgr2=0x7f5cc80cfba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:42.679 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:42.678+0000 7f5cce540640 1 --2- 172.21.15.167:0/2021304736 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5cc80b5e90 0x7f5cc80cfba0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc009870 tx=0x7f5cbc02a270 comp rx=0 tx=0).stop 2024-05-11T08:06:42.679 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:42.678+0000 7f5cce540640 1 -- 172.21.15.167:0/2021304736 shutdown_connections 2024-05-11T08:06:42.679 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:42.678+0000 7f5cce540640 1 --2- 172.21.15.167:0/2021304736 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5cc80b5e90 0x7f5cc80cfba0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:42.679 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:42.678+0000 7f5cce540640 1 -- 172.21.15.167:0/2021304736 >> 172.21.15.167:0/2021304736 conn(0x7f5cc8072730 msgr2=0x7f5cc80d2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:42.679 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:42.678+0000 7f5cce540640 1 -- 172.21.15.167:0/2021304736 shutdown_connections 2024-05-11T08:06:42.679 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:42.678+0000 7f5cce540640 1 -- 172.21.15.167:0/2021304736 wait complete. 2024-05-11T08:06:42.681 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:42.679+0000 7f5cce540640 1 Processor -- start 2024-05-11T08:06:42.681 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:42.679+0000 7f5cce540640 1 -- start start 2024-05-11T08:06:42.681 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:42.680+0000 7f5cce540640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5cc80b5e90 0x7f5cc80ca300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:42.681 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:42.680+0000 7f5cce540640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5cc80b8a40 0x7f5cc81f8e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:42.682 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:42.680+0000 7f5cce540640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5cc80d00e0 0x7f5cc81f9380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:42.682 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:42.680+0000 7f5cce540640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5cc80b8fa0 con 0x7f5cc80d00e0 2024-05-11T08:06:42.682 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:42.680+0000 7f5cce540640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5cc80b8e20 con 0x7f5cc80b5e90 2024-05-11T08:06:42.682 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:42.680+0000 7f5cce540640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f5cc80d15d0 con 0x7f5cc80b8a40 2024-05-11T08:06:42.682 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:42.680+0000 7f5cccadd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5cc80d00e0 0x7f5cc81f9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:42.682 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:42.680+0000 7f5cccadd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5cc80d00e0 0x7f5cc81f9380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:35000/0 (socket says 172.21.15.167:35000) 2024-05-11T08:06:42.682 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:42.680+0000 7f5cccadd640 1 -- 172.21.15.167:0/2169794562 learned_addr learned my addr 172.21.15.167:0/2169794562 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:42.682 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:42.680+0000 7f5cc77fe640 1 --2- 172.21.15.167:0/2169794562 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5cc80b8a40 0x7f5cc81f8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:42.682 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:42.680+0000 7f5cc7fff640 1 --2- 172.21.15.167:0/2169794562 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5cc80b5e90 0x7f5cc80ca300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:42.682 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:42.680+0000 7f5cccadd640 1 -- 172.21.15.167:0/2169794562 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5cc80b8a40 msgr2=0x7f5cc81f8e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:42.682 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:42.680+0000 7f5cccadd640 1 --2- 172.21.15.167:0/2169794562 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5cc80b8a40 0x7f5cc81f8e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:42.682 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:42.681+0000 7f5cccadd640 1 -- 172.21.15.167:0/2169794562 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5cc80b5e90 msgr2=0x7f5cc80ca300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:42.682 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:42.681+0000 7f5cccadd640 1 --2- 172.21.15.167:0/2169794562 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5cc80b5e90 0x7f5cc80ca300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:42.683 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:42.681+0000 7f5cccadd640 1 -- 172.21.15.167:0/2169794562 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cc81e7400 con 0x7f5cc80d00e0 2024-05-11T08:06:42.683 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:42.681+0000 7f5cc77fe640 1 --2- 172.21.15.167:0/2169794562 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5cc80b8a40 0x7f5cc81f8e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T08:06:42.683 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:42.681+0000 7f5cccadd640 1 --2- 172.21.15.167:0/2169794562 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5cc80d00e0 0x7f5cc81f9380 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f5cc000c6b0 tx=0x7f5cc000cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:42.683 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:42.681+0000 7f5cc4ff9640 1 -- 172.21.15.167:0/2169794562 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cc0018070 con 0x7f5cc80d00e0 2024-05-11T08:06:42.683 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:42.681+0000 7f5cc4ff9640 1 -- 172.21.15.167:0/2169794562 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5cc0004da0 con 0x7f5cc80d00e0 2024-05-11T08:06:42.683 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:42.681+0000 7f5cce540640 1 -- 172.21.15.167:0/2169794562 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cc81e76e0 con 0x7f5cc80d00e0 2024-05-11T08:06:42.683 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:42.681+0000 7f5cc4ff9640 1 -- 172.21.15.167:0/2169794562 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cc000a880 con 0x7f5cc80d00e0 2024-05-11T08:06:42.683 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:42.681+0000 7f5cce540640 1 -- 172.21.15.167:0/2169794562 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cc81e7ca0 con 0x7f5cc80d00e0 2024-05-11T08:06:42.684 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:42.682+0000 7f5cce540640 1 -- 172.21.15.167:0/2169794562 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5cc8069590 con 0x7f5cc80d00e0 2024-05-11T08:06:42.684 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:42.683+0000 7f5cc4ff9640 1 -- 172.21.15.167:0/2169794562 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f5cc001b070 con 0x7f5cc80d00e0 2024-05-11T08:06:42.684 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:42.683+0000 7f5cc4ff9640 1 --2- 172.21.15.167:0/2169794562 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f5cb40762d0 0x7f5cb4078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:42.684 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:42.683+0000 7f5cc7fff640 1 --2- 172.21.15.167:0/2169794562 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f5cb40762d0 0x7f5cb4078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:42.684 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:42.683+0000 7f5cc4ff9640 1 -- 172.21.15.167:0/2169794562 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5249+0+0 (secure 0 0 0) 0x7f5cc00969f0 con 0x7f5cc80d00e0 2024-05-11T08:06:42.684 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:42.684+0000 7f5cc7fff640 1 --2- 172.21.15.167:0/2169794562 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f5cb40762d0 0x7f5cb4078790 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f5cb8000f60 tx=0x7f5cb8009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:42.688 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:42.688+0000 7f5cc4ff9640 1 -- 172.21.15.167:0/2169794562 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f5cc00605a0 con 0x7f5cc80d00e0 2024-05-11T08:06:42.939 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:42.939+0000 7f5cce540640 1 -- 172.21.15.167:0/2169794562 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f5cc81e5830 con 0x7f5cc80d00e0 2024-05-11T08:06:42.939 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:42.939+0000 7f5cc4ff9640 1 -- 172.21.15.167:0/2169794562 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v2) ==== 80+0+6645 (secure 0 0 0) 0x7f5cc005fea0 con 0x7f5cc80d00e0 2024-05-11T08:06:42.939 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T08:06:42.940 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-05-11T08:06:42.940 INFO:teuthology.orchestra.run.smithi167.stdout: "auth_dump": [ 2024-05-11T08:06:42.940 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T08:06:42.940 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "mds.a", 2024-05-11T08:06:42.940 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQCpJj9mdAurOBAAHcD6a4z7xCorLPtMvKt7+g==", 2024-05-11T08:06:42.940 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T08:06:42.940 INFO:teuthology.orchestra.run.smithi167.stdout: "mds": "allow", 2024-05-11T08:06:42.940 INFO:teuthology.orchestra.run.smithi167.stdout: "mgr": "allow *", 2024-05-11T08:06:42.940 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow *", 2024-05-11T08:06:42.940 INFO:teuthology.orchestra.run.smithi167.stdout: "osd": "allow *" 2024-05-11T08:06:42.940 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T08:06:42.941 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T08:06:42.941 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T08:06:42.941 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "mds.b", 2024-05-11T08:06:42.941 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQCqJj9mCXXOCBAAWLxzC/EaKyn/AP/ScQ6arA==", 2024-05-11T08:06:42.941 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T08:06:42.941 INFO:teuthology.orchestra.run.smithi167.stdout: "mds": "allow", 2024-05-11T08:06:42.941 INFO:teuthology.orchestra.run.smithi167.stdout: "mgr": "allow *", 2024-05-11T08:06:42.941 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow *", 2024-05-11T08:06:42.941 INFO:teuthology.orchestra.run.smithi167.stdout: "osd": "allow *" 2024-05-11T08:06:42.941 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T08:06:42.941 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T08:06:42.941 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T08:06:42.941 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "mds.c", 2024-05-11T08:06:42.941 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQCqJj9mWHUoBBAATca00N8k4vAPQSLss3QaKQ==", 2024-05-11T08:06:42.941 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T08:06:42.942 INFO:teuthology.orchestra.run.smithi167.stdout: "mds": "allow", 2024-05-11T08:06:42.942 INFO:teuthology.orchestra.run.smithi167.stdout: "mgr": "allow *", 2024-05-11T08:06:42.942 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow *", 2024-05-11T08:06:42.942 INFO:teuthology.orchestra.run.smithi167.stdout: "osd": "allow *" 2024-05-11T08:06:42.942 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T08:06:42.942 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T08:06:42.942 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T08:06:42.942 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "mds.d", 2024-05-11T08:06:42.942 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQCqJj9m7N3qDxAAGgpj+5XiYNtiwB0to3JsHQ==", 2024-05-11T08:06:42.942 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T08:06:42.942 INFO:teuthology.orchestra.run.smithi167.stdout: "mds": "allow", 2024-05-11T08:06:42.942 INFO:teuthology.orchestra.run.smithi167.stdout: "mgr": "allow *", 2024-05-11T08:06:42.942 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow *", 2024-05-11T08:06:42.943 INFO:teuthology.orchestra.run.smithi167.stdout: "osd": "allow *" 2024-05-11T08:06:42.943 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T08:06:42.943 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T08:06:42.943 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T08:06:42.943 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "osd.0", 2024-05-11T08:06:42.943 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQCtJj9mPzPPMhAADIgOJxI1YIZwLPuF04h4Iw==", 2024-05-11T08:06:42.943 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T08:06:42.943 INFO:teuthology.orchestra.run.smithi167.stdout: "mgr": "allow profile osd", 2024-05-11T08:06:42.943 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow profile osd", 2024-05-11T08:06:42.943 INFO:teuthology.orchestra.run.smithi167.stdout: "osd": "allow *" 2024-05-11T08:06:42.943 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T08:06:42.943 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T08:06:42.943 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T08:06:42.943 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "osd.1", 2024-05-11T08:06:42.944 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQCwJj9msz+BMBAAgvNo97yu9vWqVMi6TLFWZA==", 2024-05-11T08:06:42.944 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T08:06:42.944 INFO:teuthology.orchestra.run.smithi167.stdout: "mgr": "allow profile osd", 2024-05-11T08:06:42.944 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow profile osd", 2024-05-11T08:06:42.944 INFO:teuthology.orchestra.run.smithi167.stdout: "osd": "allow *" 2024-05-11T08:06:42.944 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T08:06:42.944 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T08:06:42.944 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T08:06:42.944 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "osd.2", 2024-05-11T08:06:42.944 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQCzJj9mDa5gLhAAdzaIlR9nyF6mYHi/xxXxqA==", 2024-05-11T08:06:42.944 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T08:06:42.944 INFO:teuthology.orchestra.run.smithi167.stdout: "mgr": "allow profile osd", 2024-05-11T08:06:42.944 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow profile osd", 2024-05-11T08:06:42.944 INFO:teuthology.orchestra.run.smithi167.stdout: "osd": "allow *" 2024-05-11T08:06:42.945 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T08:06:42.945 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T08:06:42.945 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T08:06:42.945 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "osd.3", 2024-05-11T08:06:42.945 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQC2Jj9mHlmiLBAAu28f59Qm6L7ZozDdjcHpEA==", 2024-05-11T08:06:42.945 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T08:06:42.945 INFO:teuthology.orchestra.run.smithi167.stdout: "mgr": "allow profile osd", 2024-05-11T08:06:42.945 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow profile osd", 2024-05-11T08:06:42.945 INFO:teuthology.orchestra.run.smithi167.stdout: "osd": "allow *" 2024-05-11T08:06:42.945 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T08:06:42.945 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T08:06:42.945 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T08:06:42.945 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "osd.4", 2024-05-11T08:06:42.945 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQC8Jj9mOdjVMxAAoEhXYMeNeoqPqwVB8nHTpQ==", 2024-05-11T08:06:42.945 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T08:06:42.946 INFO:teuthology.orchestra.run.smithi167.stdout: "mgr": "allow profile osd", 2024-05-11T08:06:42.946 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow profile osd", 2024-05-11T08:06:42.946 INFO:teuthology.orchestra.run.smithi167.stdout: "osd": "allow *" 2024-05-11T08:06:42.946 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T08:06:42.946 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T08:06:42.946 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T08:06:42.946 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "osd.5", 2024-05-11T08:06:42.946 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQC/Jj9mEzhAMhAAjSzkBk4ucVR1bGhT1BAE4w==", 2024-05-11T08:06:42.946 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T08:06:42.946 INFO:teuthology.orchestra.run.smithi167.stdout: "mgr": "allow profile osd", 2024-05-11T08:06:42.946 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow profile osd", 2024-05-11T08:06:42.946 INFO:teuthology.orchestra.run.smithi167.stdout: "osd": "allow *" 2024-05-11T08:06:42.946 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T08:06:42.947 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T08:06:42.947 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T08:06:42.947 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "osd.6", 2024-05-11T08:06:42.947 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQDCJj9mWSICMRAACdekwLQOZxUVv9YUTvCDwg==", 2024-05-11T08:06:42.947 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T08:06:42.947 INFO:teuthology.orchestra.run.smithi167.stdout: "mgr": "allow profile osd", 2024-05-11T08:06:42.947 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow profile osd", 2024-05-11T08:06:42.947 INFO:teuthology.orchestra.run.smithi167.stdout: "osd": "allow *" 2024-05-11T08:06:42.947 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T08:06:42.947 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T08:06:42.947 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T08:06:42.948 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "osd.7", 2024-05-11T08:06:42.948 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQDFJj9mNNBZLxAAddJmZ9qfi77p4Uh/x7NA8g==", 2024-05-11T08:06:42.948 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T08:06:42.948 INFO:teuthology.orchestra.run.smithi167.stdout: "mgr": "allow profile osd", 2024-05-11T08:06:42.948 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow profile osd", 2024-05-11T08:06:42.948 INFO:teuthology.orchestra.run.smithi167.stdout: "osd": "allow *" 2024-05-11T08:06:42.948 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T08:06:42.948 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T08:06:42.948 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T08:06:42.948 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "client.0", 2024-05-11T08:06:42.948 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQCqJj9m/0ccFBAAu878v2GdClfu/qvQ/KOnvQ==", 2024-05-11T08:06:42.948 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T08:06:42.949 INFO:teuthology.orchestra.run.smithi167.stdout: "mds": "allow", 2024-05-11T08:06:42.949 INFO:teuthology.orchestra.run.smithi167.stdout: "mgr": "allow r", 2024-05-11T08:06:42.949 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow rw", 2024-05-11T08:06:42.949 INFO:teuthology.orchestra.run.smithi167.stdout: "osd": "allow rwx" 2024-05-11T08:06:42.949 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T08:06:42.949 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T08:06:42.949 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T08:06:42.949 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "client.1", 2024-05-11T08:06:42.949 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQCqJj9mWa3dHBAA4lrVGY9iSXZFb9WoZLVOVA==", 2024-05-11T08:06:42.949 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T08:06:42.949 INFO:teuthology.orchestra.run.smithi167.stdout: "mds": "allow", 2024-05-11T08:06:42.949 INFO:teuthology.orchestra.run.smithi167.stdout: "mgr": "allow r", 2024-05-11T08:06:42.950 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow rw", 2024-05-11T08:06:42.950 INFO:teuthology.orchestra.run.smithi167.stdout: "osd": "allow rwx" 2024-05-11T08:06:42.950 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T08:06:42.950 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T08:06:42.950 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T08:06:42.950 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "client.2", 2024-05-11T08:06:42.950 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQCqJj9mnwmyGRAA7lQ7Y8GJ5xnNwt+SF4D2LA==", 2024-05-11T08:06:42.950 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T08:06:42.950 INFO:teuthology.orchestra.run.smithi167.stdout: "mds": "allow", 2024-05-11T08:06:42.950 INFO:teuthology.orchestra.run.smithi167.stdout: "mgr": "allow r", 2024-05-11T08:06:42.950 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow rw", 2024-05-11T08:06:42.950 INFO:teuthology.orchestra.run.smithi167.stdout: "osd": "allow rwx" 2024-05-11T08:06:42.950 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T08:06:42.951 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T08:06:42.951 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T08:06:42.951 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "client.3", 2024-05-11T08:06:42.951 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQCqJj9mof5uIhAAsL1gMVkOcN9CMyrQc6brWg==", 2024-05-11T08:06:42.951 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T08:06:42.951 INFO:teuthology.orchestra.run.smithi167.stdout: "mds": "allow", 2024-05-11T08:06:42.951 INFO:teuthology.orchestra.run.smithi167.stdout: "mgr": "allow r", 2024-05-11T08:06:42.951 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow rw", 2024-05-11T08:06:42.951 INFO:teuthology.orchestra.run.smithi167.stdout: "osd": "allow rwx" 2024-05-11T08:06:42.951 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T08:06:42.951 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T08:06:42.951 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T08:06:42.951 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "client.admin", 2024-05-11T08:06:42.951 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQCpJj9mRJYXHRAAiaZiUQgXMb+BlqnxMM6mKw==", 2024-05-11T08:06:42.952 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T08:06:42.952 INFO:teuthology.orchestra.run.smithi167.stdout: "mds": "allow *", 2024-05-11T08:06:42.952 INFO:teuthology.orchestra.run.smithi167.stdout: "mgr": "allow *", 2024-05-11T08:06:42.952 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow *", 2024-05-11T08:06:42.952 INFO:teuthology.orchestra.run.smithi167.stdout: "osd": "allow *" 2024-05-11T08:06:42.952 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T08:06:42.952 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T08:06:42.952 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T08:06:42.952 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "client.bootstrap-mds", 2024-05-11T08:06:42.952 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQDLJj9m4DCHJBAAticJoCQwPjwXRqNcP7MONQ==", 2024-05-11T08:06:42.952 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T08:06:42.952 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow profile bootstrap-mds" 2024-05-11T08:06:42.952 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T08:06:42.952 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T08:06:42.953 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T08:06:42.953 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "client.bootstrap-mgr", 2024-05-11T08:06:42.953 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQDLJj9mGzmHJBAAvE8s2+DrFOzkZzD3xdjTlw==", 2024-05-11T08:06:42.953 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T08:06:42.953 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow profile bootstrap-mgr" 2024-05-11T08:06:42.953 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T08:06:42.953 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T08:06:42.953 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T08:06:42.953 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "client.bootstrap-osd", 2024-05-11T08:06:42.953 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQDLJj9m8D+HJBAAKxwn5J9x6EPVwimsctIt8g==", 2024-05-11T08:06:42.953 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T08:06:42.953 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow profile bootstrap-osd" 2024-05-11T08:06:42.953 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T08:06:42.953 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T08:06:42.953 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T08:06:42.954 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "client.bootstrap-rbd", 2024-05-11T08:06:42.954 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQDLJj9mikaHJBAASISu5y8uOfhFl6d/8hvfzA==", 2024-05-11T08:06:42.954 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T08:06:42.954 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow profile bootstrap-rbd" 2024-05-11T08:06:42.954 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T08:06:42.954 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T08:06:42.954 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T08:06:42.954 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-05-11T08:06:42.954 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQDLJj9mQU2HJBAAFrxNpfoEDj4I7BUL2zjeVQ==", 2024-05-11T08:06:42.954 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T08:06:42.954 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-05-11T08:06:42.954 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T08:06:42.954 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T08:06:42.955 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T08:06:42.955 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "client.bootstrap-rgw", 2024-05-11T08:06:42.955 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQDLJj9mkFSHJBAALi5k/kIrR7uyX/aLH7DgMg==", 2024-05-11T08:06:42.955 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T08:06:42.955 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow profile bootstrap-rgw" 2024-05-11T08:06:42.955 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T08:06:42.955 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T08:06:42.955 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T08:06:42.955 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "mgr.x", 2024-05-11T08:06:42.955 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQCpJj9mqi54NRAAd33DWaNHSGFFWs57F2Ft7Q==", 2024-05-11T08:06:42.955 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T08:06:42.955 INFO:teuthology.orchestra.run.smithi167.stdout: "mds": "allow *", 2024-05-11T08:06:42.955 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow profile mgr", 2024-05-11T08:06:42.955 INFO:teuthology.orchestra.run.smithi167.stdout: "osd": "allow *" 2024-05-11T08:06:42.956 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T08:06:42.956 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T08:06:42.956 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T08:06:42.956 INFO:teuthology.orchestra.run.smithi167.stdout: "entity": "mgr.y", 2024-05-11T08:06:42.956 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "AQCpJj9mHQi/MRAAa1ks1sdtI2VqeUQElx7gyw==", 2024-05-11T08:06:42.956 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": { 2024-05-11T08:06:42.956 INFO:teuthology.orchestra.run.smithi167.stdout: "mds": "allow *", 2024-05-11T08:06:42.956 INFO:teuthology.orchestra.run.smithi167.stdout: "mon": "allow profile mgr", 2024-05-11T08:06:42.956 INFO:teuthology.orchestra.run.smithi167.stdout: "osd": "allow *" 2024-05-11T08:06:42.956 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T08:06:42.956 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T08:06:42.956 INFO:teuthology.orchestra.run.smithi167.stdout: ] 2024-05-11T08:06:42.956 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-05-11T08:06:42.958 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:42.942+0000 7f5cce540640 1 -- 172.21.15.167:0/2169794562 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f5cb40762d0 msgr2=0x7f5cb4078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:42.958 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:42.942+0000 7f5cce540640 1 --2- 172.21.15.167:0/2169794562 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f5cb40762d0 0x7f5cb4078790 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f5cb8000f60 tx=0x7f5cb8009040 comp rx=0 tx=0).stop 2024-05-11T08:06:42.958 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:42.942+0000 7f5cce540640 1 -- 172.21.15.167:0/2169794562 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5cc80d00e0 msgr2=0x7f5cc81f9380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:42.958 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:42.942+0000 7f5cce540640 1 --2- 172.21.15.167:0/2169794562 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5cc80d00e0 0x7f5cc81f9380 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f5cc000c6b0 tx=0x7f5cc000cb70 comp rx=0 tx=0).stop 2024-05-11T08:06:42.958 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:42.942+0000 7f5cce540640 1 -- 172.21.15.167:0/2169794562 shutdown_connections 2024-05-11T08:06:42.958 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:42.942+0000 7f5cce540640 1 --2- 172.21.15.167:0/2169794562 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f5cb40762d0 0x7f5cb4078790 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:42.958 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:42.942+0000 7f5cce540640 1 --2- 172.21.15.167:0/2169794562 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5cc80d00e0 0x7f5cc81f9380 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:42.958 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:42.942+0000 7f5cce540640 1 --2- 172.21.15.167:0/2169794562 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5cc80b8a40 0x7f5cc81f8e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:42.958 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:42.942+0000 7f5cce540640 1 --2- 172.21.15.167:0/2169794562 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5cc80b5e90 0x7f5cc80ca300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:42.958 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:42.942+0000 7f5cce540640 1 -- 172.21.15.167:0/2169794562 >> 172.21.15.167:0/2169794562 conn(0x7f5cc8072730 msgr2=0x7f5cc81592b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:42.959 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:42.942+0000 7f5cce540640 1 -- 172.21.15.167:0/2169794562 shutdown_connections 2024-05-11T08:06:42.959 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:42.942+0000 7f5cce540640 1 -- 172.21.15.167:0/2169794562 wait complete. 2024-05-11T08:06:42.959 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-05-11T08:06:43.061 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.061+0000 7f5bfa483640 1 Processor -- start 2024-05-11T08:06:43.061 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.061+0000 7f5bfa483640 1 -- start start 2024-05-11T08:06:43.062 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.062+0000 7f5bfa483640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f5bf40cd210 0x7f5bf40cb750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:43.062 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.062+0000 7f5bfa483640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5bf40cd6a0 con 0x7f5bf40adf10 2024-05-11T08:06:43.062 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.062+0000 7f5bfa483640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5bf406d810 con 0x7f5bf40aed60 2024-05-11T08:06:43.062 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.062+0000 7f5bfa483640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f5bf406ddd0 con 0x7f5bf40cd210 2024-05-11T08:06:43.062 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.062+0000 7f5bf8a20640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f5bf40cd210 0x7f5bf40cb750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:43.063 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.062+0000 7f5bf3fff640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f5bf40adf10 0x7f5bf41639c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:49684/0 (socket says 172.21.15.167:49684) 2024-05-11T08:06:43.063 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.062+0000 7f5bf3fff640 1 -- 172.21.15.167:0/1598269447 learned_addr learned my addr 172.21.15.167:0/1598269447 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T08:06:43.063 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.062+0000 7f5bf8a20640 1 -- 172.21.15.167:0/1598269447 >> v1:172.21.15.204:6789/0 conn(0x7f5bf40aed60 legacy=0x7f5bf40ae2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:43.063 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.062+0000 7f5bf8a20640 1 -- 172.21.15.167:0/1598269447 >> v1:172.21.15.167:6789/0 conn(0x7f5bf40adf10 legacy=0x7f5bf41639c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:43.063 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.062+0000 7f5bf8a20640 1 -- 172.21.15.167:0/1598269447 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bf406daf0 con 0x7f5bf40cd210 2024-05-11T08:06:43.063 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.062+0000 7f5bf2ffd640 1 -- 172.21.15.167:0/1598269447 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 664225616 0 0) 0x7f5be4001110 con 0x7f5bf40adf10 2024-05-11T08:06:43.064 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.063+0000 7f5bf8a20640 1 --2- 172.21.15.167:0/1598269447 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5bf40cd210 0x7f5bf40cb750 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f5bec009d90 tx=0x7f5bec02a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca886e9f5c0ef8d8 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:43.064 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.063+0000 7f5bf2ffd640 1 -- 172.21.15.167:0/1598269447 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bec031600 con 0x7f5bf40cd210 2024-05-11T08:06:43.064 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.063+0000 7f5bf2ffd640 1 -- 172.21.15.167:0/1598269447 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bec0317a0 con 0x7f5bf40cd210 2024-05-11T08:06:43.065 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.063+0000 7f5bf2ffd640 1 -- 172.21.15.167:0/1598269447 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bec031b70 con 0x7f5bf40cd210 2024-05-11T08:06:43.065 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.063+0000 7f5bfa483640 1 -- 172.21.15.167:0/1598269447 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5bf40cd210 msgr2=0x7f5bf40cb750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:43.065 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.063+0000 7f5bfa483640 1 --2- 172.21.15.167:0/1598269447 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5bf40cd210 0x7f5bf40cb750 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f5bec009d90 tx=0x7f5bec02a5d0 comp rx=0 tx=0).stop 2024-05-11T08:06:43.065 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.063+0000 7f5bfa483640 1 -- 172.21.15.167:0/1598269447 shutdown_connections 2024-05-11T08:06:43.065 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.063+0000 7f5bfa483640 1 --2- 172.21.15.167:0/1598269447 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5bf40cd210 0x7f5bf40cb750 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:43.065 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.063+0000 7f5bfa483640 1 -- 172.21.15.167:0/1598269447 >> 172.21.15.167:0/1598269447 conn(0x7f5bf40d1100 msgr2=0x7f5bf40d1510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:43.065 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.063+0000 7f5bfa483640 1 -- 172.21.15.167:0/1598269447 shutdown_connections 2024-05-11T08:06:43.065 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.064+0000 7f5bfa483640 1 -- 172.21.15.167:0/1598269447 wait complete. 2024-05-11T08:06:43.065 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.065+0000 7f5bfa483640 1 Processor -- start 2024-05-11T08:06:43.067 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.065+0000 7f5bfa483640 1 -- start start 2024-05-11T08:06:43.067 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.065+0000 7f5bfa483640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5bf40adf10 0x7f5bf40736a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:43.068 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.066+0000 7f5bfa483640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5bf40aed60 0x7f5bf40cac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:43.068 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.066+0000 7f5bfa483640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5bf40cd210 0x7f5bf4159c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:43.068 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.066+0000 7f5bfa483640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5bf40d0cc0 con 0x7f5bf40cd210 2024-05-11T08:06:43.068 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.066+0000 7f5bfa483640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5bf41634a0 con 0x7f5bf40aed60 2024-05-11T08:06:43.068 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.066+0000 7f5bfa483640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f5bf40cbc90 con 0x7f5bf40adf10 2024-05-11T08:06:43.068 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.066+0000 7f5bf8a20640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5bf40cd210 0x7f5bf4159c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:43.068 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.066+0000 7f5bf8a20640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5bf40cd210 0x7f5bf4159c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:35014/0 (socket says 172.21.15.167:35014) 2024-05-11T08:06:43.068 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.066+0000 7f5bf8a20640 1 -- 172.21.15.167:0/2406970002 learned_addr learned my addr 172.21.15.167:0/2406970002 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:43.068 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.066+0000 7f5bf3fff640 1 --2- 172.21.15.167:0/2406970002 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5bf40adf10 0x7f5bf40736a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:43.068 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.066+0000 7f5bf37fe640 1 --2- 172.21.15.167:0/2406970002 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5bf40aed60 0x7f5bf40cac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:43.068 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.066+0000 7f5bf8a20640 1 -- 172.21.15.167:0/2406970002 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5bf40adf10 msgr2=0x7f5bf40736a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:43.068 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.066+0000 7f5bf8a20640 1 --2- 172.21.15.167:0/2406970002 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5bf40adf10 0x7f5bf40736a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:43.069 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.066+0000 7f5bf8a20640 1 -- 172.21.15.167:0/2406970002 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5bf40aed60 msgr2=0x7f5bf40cac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:43.069 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.066+0000 7f5bf8a20640 1 --2- 172.21.15.167:0/2406970002 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5bf40aed60 0x7f5bf40cac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:43.069 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.066+0000 7f5bf8a20640 1 -- 172.21.15.167:0/2406970002 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bf4073d30 con 0x7f5bf40cd210 2024-05-11T08:06:43.069 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.066+0000 7f5bf37fe640 1 --2- 172.21.15.167:0/2406970002 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5bf40aed60 0x7f5bf40cac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T08:06:43.069 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.066+0000 7f5bf8a20640 1 --2- 172.21.15.167:0/2406970002 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5bf40cd210 0x7f5bf4159c50 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f5bec009d90 tx=0x7f5bec004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:43.069 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.066+0000 7f5bf0ff9640 1 -- 172.21.15.167:0/2406970002 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bec03f070 con 0x7f5bf40cd210 2024-05-11T08:06:43.069 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.066+0000 7f5bf0ff9640 1 -- 172.21.15.167:0/2406970002 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bec030b60 con 0x7f5bf40cd210 2024-05-11T08:06:43.069 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.066+0000 7f5bfa483640 1 -- 172.21.15.167:0/2406970002 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bf4074010 con 0x7f5bf40cd210 2024-05-11T08:06:43.069 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.066+0000 7f5bf0ff9640 1 -- 172.21.15.167:0/2406970002 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bec043410 con 0x7f5bf40cd210 2024-05-11T08:06:43.069 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.066+0000 7f5bfa483640 1 -- 172.21.15.167:0/2406970002 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bf415fd60 con 0x7f5bf40cd210 2024-05-11T08:06:43.069 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.067+0000 7f5bfa483640 1 -- 172.21.15.167:0/2406970002 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5bf40694b0 con 0x7f5bf40cd210 2024-05-11T08:06:43.070 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.068+0000 7f5bf0ff9640 1 -- 172.21.15.167:0/2406970002 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f5bec03b410 con 0x7f5bf40cd210 2024-05-11T08:06:43.070 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.068+0000 7f5bf0ff9640 1 --2- 172.21.15.167:0/2406970002 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f5be00762d0 0x7f5be0078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:43.070 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.069+0000 7f5bf0ff9640 1 -- 172.21.15.167:0/2406970002 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5249+0+0 (secure 0 0 0) 0x7f5bec0b8710 con 0x7f5bf40cd210 2024-05-11T08:06:43.070 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.069+0000 7f5bf3fff640 1 --2- 172.21.15.167:0/2406970002 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f5be00762d0 0x7f5be0078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:43.070 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.069+0000 7f5bf3fff640 1 --2- 172.21.15.167:0/2406970002 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f5be00762d0 0x7f5be0078790 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5be4007d70 tx=0x7f5be4007d00 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:43.073 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.073+0000 7f5bf0ff9640 1 -- 172.21.15.167:0/2406970002 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f5bec082380 con 0x7f5bf40cd210 2024-05-11T08:06:43.323 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.323+0000 7f5bfa483640 1 -- 172.21.15.167:0/2406970002 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) -- 0x7f5bf40cd8f0 con 0x7f5bf40cd210 2024-05-11T08:06:43.325 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.325+0000 7f5bf0ff9640 1 -- 172.21.15.167:0/2406970002 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v3) ==== 90+0+0 (secure 0 0 0) 0x7f5bec081c80 con 0x7f5bf40cd210 2024-05-11T08:06:43.327 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.327+0000 7f5bfa483640 1 -- 172.21.15.167:0/2406970002 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f5be00762d0 msgr2=0x7f5be0078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:43.328 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.327+0000 7f5bfa483640 1 --2- 172.21.15.167:0/2406970002 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f5be00762d0 0x7f5be0078790 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5be4007d70 tx=0x7f5be4007d00 comp rx=0 tx=0).stop 2024-05-11T08:06:43.328 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.327+0000 7f5bfa483640 1 -- 172.21.15.167:0/2406970002 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5bf40cd210 msgr2=0x7f5bf4159c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:43.328 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.327+0000 7f5bfa483640 1 --2- 172.21.15.167:0/2406970002 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5bf40cd210 0x7f5bf4159c50 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f5bec009d90 tx=0x7f5bec004a40 comp rx=0 tx=0).stop 2024-05-11T08:06:43.328 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.327+0000 7f5bfa483640 1 -- 172.21.15.167:0/2406970002 shutdown_connections 2024-05-11T08:06:43.328 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.327+0000 7f5bfa483640 1 --2- 172.21.15.167:0/2406970002 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f5be00762d0 0x7f5be0078790 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:43.328 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.327+0000 7f5bfa483640 1 --2- 172.21.15.167:0/2406970002 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5bf40cd210 0x7f5bf4159c50 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:43.328 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.327+0000 7f5bfa483640 1 --2- 172.21.15.167:0/2406970002 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5bf40aed60 0x7f5bf40cac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:43.328 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.327+0000 7f5bfa483640 1 --2- 172.21.15.167:0/2406970002 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5bf40adf10 0x7f5bf40736a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:43.328 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.327+0000 7f5bfa483640 1 -- 172.21.15.167:0/2406970002 >> 172.21.15.167:0/2406970002 conn(0x7f5bf40d1100 msgr2=0x7f5bf415bb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:43.328 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.327+0000 7f5bfa483640 1 -- 172.21.15.167:0/2406970002 shutdown_connections 2024-05-11T08:06:43.328 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.327+0000 7f5bfa483640 1 -- 172.21.15.167:0/2406970002 wait complete. 2024-05-11T08:06:43.339 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-05-11T08:06:43.440 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.440+0000 7f8756459640 1 Processor -- start 2024-05-11T08:06:43.441 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.440+0000 7f8756459640 1 -- start start 2024-05-11T08:06:43.442 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.441+0000 7f8756459640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f8750161380 0x7f8750074260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:43.442 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.441+0000 7f8756459640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f87500b3d50 0x7f8750072950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:43.442 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.441+0000 7f8756459640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87500b65e0 con 0x7f875015ac80 2024-05-11T08:06:43.442 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.441+0000 7f8756459640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f875006e3c0 con 0x7f87500b3d50 2024-05-11T08:06:43.442 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.441+0000 7f8756459640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f875015b060 con 0x7f8750161380 2024-05-11T08:06:43.442 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.441+0000 7f87549f6640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f87500b3d50 0x7f8750072950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:43.442 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.441+0000 7f874ffff640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f875015ac80 0x7f8750160c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:49688/0 (socket says 172.21.15.167:49688) 2024-05-11T08:06:43.442 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.441+0000 7f874ffff640 1 -- 172.21.15.167:0/3749366044 learned_addr learned my addr 172.21.15.167:0/3749366044 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T08:06:43.443 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.442+0000 7f874f7fe640 1 --2- 172.21.15.167:0/3749366044 >> v2:172.21.15.204:3300/0 conn(0x7f8750161380 0x7f8750074260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:43.443 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.442+0000 7f87549f6640 1 -- 172.21.15.167:0/3749366044 >> v2:172.21.15.204:3300/0 conn(0x7f8750161380 msgr2=0x7f8750074260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:43.443 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.442+0000 7f87549f6640 1 --2- 172.21.15.167:0/3749366044 >> v2:172.21.15.204:3300/0 conn(0x7f8750161380 0x7f8750074260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:43.443 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.442+0000 7f87549f6640 1 -- 172.21.15.167:0/3749366044 >> v1:172.21.15.167:6789/0 conn(0x7f875015ac80 legacy=0x7f8750160c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:43.443 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.442+0000 7f87549f6640 1 -- 172.21.15.167:0/3749366044 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87500b68c0 con 0x7f87500b3d50 2024-05-11T08:06:43.444 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.442+0000 7f87549f6640 1 --2- 172.21.15.167:0/3749366044 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f87500b3d50 0x7f8750072950 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f8750074f20 tx=0x7f874802a570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31ac3c587eab7155 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:43.444 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.443+0000 7f874effd640 1 -- 172.21.15.167:0/3749366044 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8748007740 con 0x7f87500b3d50 2024-05-11T08:06:43.444 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.443+0000 7f874effd640 1 -- 172.21.15.167:0/3749366044 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87480078e0 con 0x7f87500b3d50 2024-05-11T08:06:43.445 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.443+0000 7f874effd640 1 -- 172.21.15.167:0/3749366044 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8748007cb0 con 0x7f87500b3d50 2024-05-11T08:06:43.445 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.443+0000 7f8756459640 1 -- 172.21.15.167:0/3749366044 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f87500b3d50 msgr2=0x7f8750072950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:43.445 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.443+0000 7f8756459640 1 --2- 172.21.15.167:0/3749366044 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f87500b3d50 0x7f8750072950 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f8750074f20 tx=0x7f874802a570 comp rx=0 tx=0).stop 2024-05-11T08:06:43.445 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.443+0000 7f8756459640 1 -- 172.21.15.167:0/3749366044 shutdown_connections 2024-05-11T08:06:43.445 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.443+0000 7f8756459640 1 --2- 172.21.15.167:0/3749366044 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f87500b3d50 0x7f8750072950 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:43.445 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.443+0000 7f8756459640 1 --2- 172.21.15.167:0/3749366044 >> v2:172.21.15.204:3300/0 conn(0x7f8750161380 0x7f8750074260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:43.445 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.443+0000 7f8756459640 1 -- 172.21.15.167:0/3749366044 >> 172.21.15.167:0/3749366044 conn(0x7f875006c450 msgr2=0x7f875006c860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:43.445 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.443+0000 7f8756459640 1 -- 172.21.15.167:0/3749366044 shutdown_connections 2024-05-11T08:06:43.445 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.443+0000 7f8756459640 1 -- 172.21.15.167:0/3749366044 wait complete. 2024-05-11T08:06:43.445 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.444+0000 7f8756459640 1 Processor -- start 2024-05-11T08:06:43.445 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.445+0000 7f8756459640 1 -- start start 2024-05-11T08:06:43.447 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.445+0000 7f8756459640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f87500b3d50 0x7f875013a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:43.447 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.445+0000 7f8756459640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f875015ac80 0x7f87500ac470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:43.447 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.445+0000 7f8756459640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8750161380 0x7f875006cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:43.447 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.445+0000 7f8756459640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f87500b4160 con 0x7f875015ac80 2024-05-11T08:06:43.448 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.445+0000 7f8756459640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f87500b44f0 con 0x7f87500b3d50 2024-05-11T08:06:43.448 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.445+0000 7f8756459640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f8750072e90 con 0x7f8750161380 2024-05-11T08:06:43.448 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.445+0000 7f874f7fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f875015ac80 0x7f87500ac470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:43.448 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.445+0000 7f874f7fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f875015ac80 0x7f87500ac470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:35028/0 (socket says 172.21.15.167:35028) 2024-05-11T08:06:43.448 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.445+0000 7f874f7fe640 1 -- 172.21.15.167:0/1463114635 learned_addr learned my addr 172.21.15.167:0/1463114635 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:43.448 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.446+0000 7f874ffff640 1 --2- 172.21.15.167:0/1463114635 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f87500b3d50 0x7f875013a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:43.448 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.446+0000 7f87549f6640 1 --2- 172.21.15.167:0/1463114635 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8750161380 0x7f875006cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:43.448 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.446+0000 7f874f7fe640 1 -- 172.21.15.167:0/1463114635 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8750161380 msgr2=0x7f875006cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:43.448 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.446+0000 7f874f7fe640 1 --2- 172.21.15.167:0/1463114635 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8750161380 0x7f875006cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:43.448 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.446+0000 7f874f7fe640 1 -- 172.21.15.167:0/1463114635 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f87500b3d50 msgr2=0x7f875013a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:43.448 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.446+0000 7f874f7fe640 1 --2- 172.21.15.167:0/1463114635 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f87500b3d50 0x7f875013a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:43.448 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.446+0000 7f874f7fe640 1 -- 172.21.15.167:0/1463114635 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87500acef0 con 0x7f875015ac80 2024-05-11T08:06:43.448 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.446+0000 7f874ffff640 1 --2- 172.21.15.167:0/1463114635 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f87500b3d50 0x7f875013a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T08:06:43.448 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.446+0000 7f874f7fe640 1 --2- 172.21.15.167:0/1463114635 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f875015ac80 0x7f87500ac470 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f874400d5f0 tx=0x7f874400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:43.449 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.446+0000 7f874cff9640 1 -- 172.21.15.167:0/1463114635 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87440125f0 con 0x7f875015ac80 2024-05-11T08:06:43.449 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.446+0000 7f874cff9640 1 -- 172.21.15.167:0/1463114635 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8744019070 con 0x7f875015ac80 2024-05-11T08:06:43.449 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.446+0000 7f874cff9640 1 -- 172.21.15.167:0/1463114635 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87440129d0 con 0x7f875015ac80 2024-05-11T08:06:43.449 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.446+0000 7f8756459640 1 -- 172.21.15.167:0/1463114635 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87500ad1d0 con 0x7f875015ac80 2024-05-11T08:06:43.449 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.446+0000 7f8756459640 1 -- 172.21.15.167:0/1463114635 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87500ad790 con 0x7f875015ac80 2024-05-11T08:06:43.449 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.447+0000 7f8756459640 1 -- 172.21.15.167:0/1463114635 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87500694b0 con 0x7f875015ac80 2024-05-11T08:06:43.450 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.448+0000 7f874cff9640 1 -- 172.21.15.167:0/1463114635 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f8744012b70 con 0x7f875015ac80 2024-05-11T08:06:43.450 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.448+0000 7f874cff9640 1 --2- 172.21.15.167:0/1463114635 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f873c0762d0 0x7f873c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:43.450 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.448+0000 7f874cff9640 1 -- 172.21.15.167:0/1463114635 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5249+0+0 (secure 0 0 0) 0x7f87440977f0 con 0x7f875015ac80 2024-05-11T08:06:43.450 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.448+0000 7f874ffff640 1 --2- 172.21.15.167:0/1463114635 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f873c0762d0 0x7f873c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:43.450 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.449+0000 7f874ffff640 1 --2- 172.21.15.167:0/1463114635 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f873c0762d0 0x7f873c078790 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f8740007fd0 tx=0x7f8740007d40 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:43.452 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.453+0000 7f874cff9640 1 -- 172.21.15.167:0/1463114635 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f8744061310 con 0x7f875015ac80 2024-05-11T08:06:43.704 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.704+0000 7f8756459640 1 -- 172.21.15.167:0/1463114635 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) -- 0x7f875015eb30 con 0x7f875015ac80 2024-05-11T08:06:43.706 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.706+0000 7f874cff9640 1 -- 172.21.15.167:0/1463114635 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v4) ==== 90+0+0 (secure 0 0 0) 0x7f8744060c10 con 0x7f875015ac80 2024-05-11T08:06:43.708 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.708+0000 7f8756459640 1 -- 172.21.15.167:0/1463114635 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f873c0762d0 msgr2=0x7f873c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:43.708 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.708+0000 7f8756459640 1 --2- 172.21.15.167:0/1463114635 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f873c0762d0 0x7f873c078790 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f8740007fd0 tx=0x7f8740007d40 comp rx=0 tx=0).stop 2024-05-11T08:06:43.708 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.708+0000 7f8756459640 1 -- 172.21.15.167:0/1463114635 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f875015ac80 msgr2=0x7f87500ac470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:43.708 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.708+0000 7f8756459640 1 --2- 172.21.15.167:0/1463114635 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f875015ac80 0x7f87500ac470 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f874400d5f0 tx=0x7f874400dab0 comp rx=0 tx=0).stop 2024-05-11T08:06:43.708 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.708+0000 7f8756459640 1 -- 172.21.15.167:0/1463114635 shutdown_connections 2024-05-11T08:06:43.708 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.708+0000 7f8756459640 1 --2- 172.21.15.167:0/1463114635 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f873c0762d0 0x7f873c078790 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:43.708 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.708+0000 7f8756459640 1 --2- 172.21.15.167:0/1463114635 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8750161380 0x7f875006cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:43.708 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.708+0000 7f8756459640 1 --2- 172.21.15.167:0/1463114635 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f875015ac80 0x7f87500ac470 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:43.709 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.708+0000 7f8756459640 1 --2- 172.21.15.167:0/1463114635 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f87500b3d50 0x7f875013a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:43.709 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.708+0000 7f8756459640 1 -- 172.21.15.167:0/1463114635 >> 172.21.15.167:0/1463114635 conn(0x7f875006c450 msgr2=0x7f875015f050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:43.709 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.708+0000 7f8756459640 1 -- 172.21.15.167:0/1463114635 shutdown_connections 2024-05-11T08:06:43.709 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.708+0000 7f8756459640 1 -- 172.21.15.167:0/1463114635 wait complete. 2024-05-11T08:06:43.719 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-05-11T08:06:43.820 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.820+0000 7f2a71262640 1 Processor -- start 2024-05-11T08:06:43.820 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.820+0000 7f2a71262640 1 -- start start 2024-05-11T08:06:43.821 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.821+0000 7f2a71262640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2a6c15f330 0x7f2a6c0d1130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:43.821 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.821+0000 7f2a71262640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f2a6c0d1670 0x7f2a6c0b82c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:43.821 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.821+0000 7f2a71262640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a6c06c6a0 con 0x7f2a6c0b8800 2024-05-11T08:06:43.821 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.821+0000 7f2a71262640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f2a6c06c450 con 0x7f2a6c0d1670 2024-05-11T08:06:43.822 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.821+0000 7f2a71262640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f2a6c06d2b0 con 0x7f2a6c15f330 2024-05-11T08:06:43.822 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.821+0000 7f2a627fc640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f2a6c0d1670 0x7f2a6c0b82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:43.822 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.821+0000 7f2a6b7fe640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f2a6c0b8800 0x7f2a6c0d3c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:49700/0 (socket says 172.21.15.167:49700) 2024-05-11T08:06:43.822 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.821+0000 7f2a6b7fe640 1 -- 172.21.15.167:0/1259478786 learned_addr learned my addr 172.21.15.167:0/1259478786 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T08:06:43.822 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.821+0000 7f2a6affd640 1 --2- 172.21.15.167:0/1259478786 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2a6c15f330 0x7f2a6c0d1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:43.822 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.821+0000 7f2a627fc640 1 -- 172.21.15.167:0/1259478786 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2a6c15f330 msgr2=0x7f2a6c0d1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:43.822 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.821+0000 7f2a627fc640 1 --2- 172.21.15.167:0/1259478786 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2a6c15f330 0x7f2a6c0d1130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:43.822 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.821+0000 7f2a627fc640 1 -- 172.21.15.167:0/1259478786 >> v1:172.21.15.167:6789/0 conn(0x7f2a6c0b8800 legacy=0x7f2a6c0d3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:43.822 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.821+0000 7f2a627fc640 1 -- 172.21.15.167:0/1259478786 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a6c06c980 con 0x7f2a6c0d1670 2024-05-11T08:06:43.822 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.821+0000 7f2a6a7fc640 1 -- 172.21.15.167:0/1259478786 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 456960556 0 0) 0x7f2a6c06c6a0 con 0x7f2a6c0b8800 2024-05-11T08:06:43.824 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.822+0000 7f2a627fc640 1 --2- 172.21.15.167:0/1259478786 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2a6c0d1670 0x7f2a6c0b82c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f2a540097a0 tx=0x7f2a5402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3a8122c0fe51de7 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:43.824 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.822+0000 7f2a6a7fc640 1 -- 172.21.15.167:0/1259478786 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a54037070 con 0x7f2a6c0d1670 2024-05-11T08:06:43.824 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.822+0000 7f2a6a7fc640 1 -- 172.21.15.167:0/1259478786 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a54032440 con 0x7f2a6c0d1670 2024-05-11T08:06:43.824 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.822+0000 7f2a6a7fc640 1 -- 172.21.15.167:0/1259478786 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a54032810 con 0x7f2a6c0d1670 2024-05-11T08:06:43.824 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.822+0000 7f2a71262640 1 -- 172.21.15.167:0/1259478786 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2a6c0d1670 msgr2=0x7f2a6c0b82c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:43.824 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.822+0000 7f2a71262640 1 --2- 172.21.15.167:0/1259478786 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2a6c0d1670 0x7f2a6c0b82c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f2a540097a0 tx=0x7f2a5402a180 comp rx=0 tx=0).stop 2024-05-11T08:06:43.824 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.822+0000 7f2a71262640 1 -- 172.21.15.167:0/1259478786 shutdown_connections 2024-05-11T08:06:43.824 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.822+0000 7f2a71262640 1 --2- 172.21.15.167:0/1259478786 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2a6c0d1670 0x7f2a6c0b82c0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:43.824 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.822+0000 7f2a71262640 1 --2- 172.21.15.167:0/1259478786 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2a6c15f330 0x7f2a6c0d1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:43.824 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.822+0000 7f2a71262640 1 -- 172.21.15.167:0/1259478786 >> 172.21.15.167:0/1259478786 conn(0x7f2a6c0b6b80 msgr2=0x7f2a6c0b6f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:43.824 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.823+0000 7f2a71262640 1 -- 172.21.15.167:0/1259478786 shutdown_connections 2024-05-11T08:06:43.824 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.823+0000 7f2a71262640 1 -- 172.21.15.167:0/1259478786 wait complete. 2024-05-11T08:06:43.824 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.824+0000 7f2a71262640 1 Processor -- start 2024-05-11T08:06:43.824 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.824+0000 7f2a71262640 1 -- start start 2024-05-11T08:06:43.826 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.825+0000 7f2a71262640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2a6c0b8800 0x7f2a6c15fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:43.826 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.825+0000 7f2a71262640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2a6c15f330 0x7f2a6c14d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:43.827 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.825+0000 7f2a71262640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2a6c14dcc0 0x7f2a6c0b35c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:43.827 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.825+0000 7f2a71262640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2a6c15f710 con 0x7f2a6c14dcc0 2024-05-11T08:06:43.827 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.825+0000 7f2a71262640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f2a6c06cf40 con 0x7f2a6c15f330 2024-05-11T08:06:43.827 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.825+0000 7f2a71262640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f2a6c0d06b0 con 0x7f2a6c0b8800 2024-05-11T08:06:43.827 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.825+0000 7f2a6b7fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2a6c14dcc0 0x7f2a6c0b35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:43.827 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.825+0000 7f2a6b7fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2a6c14dcc0 0x7f2a6c0b35c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:35042/0 (socket says 172.21.15.167:35042) 2024-05-11T08:06:43.827 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.825+0000 7f2a6b7fe640 1 -- 172.21.15.167:0/724316361 learned_addr learned my addr 172.21.15.167:0/724316361 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:43.827 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.825+0000 7f2a6affd640 1 --2- 172.21.15.167:0/724316361 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2a6c0b8800 0x7f2a6c15fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:43.827 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.825+0000 7f2a6b7fe640 1 -- 172.21.15.167:0/724316361 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2a6c0b8800 msgr2=0x7f2a6c15fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:43.827 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.825+0000 7f2a6b7fe640 1 --2- 172.21.15.167:0/724316361 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2a6c0b8800 0x7f2a6c15fdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:43.827 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.825+0000 7f2a6b7fe640 1 -- 172.21.15.167:0/724316361 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2a6c15f330 msgr2=0x7f2a6c14d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:43.827 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.825+0000 7f2a627fc640 1 --2- 172.21.15.167:0/724316361 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2a6c15f330 0x7f2a6c14d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:43.827 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.825+0000 7f2a6b7fe640 1 --2- 172.21.15.167:0/724316361 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2a6c15f330 0x7f2a6c14d780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:43.827 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.825+0000 7f2a6b7fe640 1 -- 172.21.15.167:0/724316361 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a6c14e470 con 0x7f2a6c14dcc0 2024-05-11T08:06:43.827 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.825+0000 7f2a6b7fe640 1 --2- 172.21.15.167:0/724316361 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2a6c14dcc0 0x7f2a6c0b35c0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f2a6400c7d0 tx=0x7f2a6400cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:43.827 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.825+0000 7f2a63fff640 1 -- 172.21.15.167:0/724316361 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a64018070 con 0x7f2a6c14dcc0 2024-05-11T08:06:43.828 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.825+0000 7f2a63fff640 1 -- 172.21.15.167:0/724316361 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a6400a5e0 con 0x7f2a6c14dcc0 2024-05-11T08:06:43.828 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.825+0000 7f2a71262640 1 -- 172.21.15.167:0/724316361 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a6c14b870 con 0x7f2a6c14dcc0 2024-05-11T08:06:43.828 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.825+0000 7f2a63fff640 1 -- 172.21.15.167:0/724316361 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a6400aa10 con 0x7f2a6c14dcc0 2024-05-11T08:06:43.828 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.825+0000 7f2a71262640 1 -- 172.21.15.167:0/724316361 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a6c14be30 con 0x7f2a6c14dcc0 2024-05-11T08:06:43.828 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.826+0000 7f2a71262640 1 -- 172.21.15.167:0/724316361 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a6c0c8980 con 0x7f2a6c14dcc0 2024-05-11T08:06:43.828 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.827+0000 7f2a63fff640 1 -- 172.21.15.167:0/724316361 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f2a6401b070 con 0x7f2a6c14dcc0 2024-05-11T08:06:43.829 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.828+0000 7f2a63fff640 1 --2- 172.21.15.167:0/724316361 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f2a4c076300 0x7f2a4c0787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:43.829 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.828+0000 7f2a63fff640 1 -- 172.21.15.167:0/724316361 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5249+0+0 (secure 0 0 0) 0x7f2a64096cb0 con 0x7f2a6c14dcc0 2024-05-11T08:06:43.829 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.828+0000 7f2a6affd640 1 --2- 172.21.15.167:0/724316361 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f2a4c076300 0x7f2a4c0787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:43.829 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.828+0000 7f2a6affd640 1 --2- 172.21.15.167:0/724316361 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f2a4c076300 0x7f2a4c0787c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c006fd0 tx=0x7f2a5c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:43.831 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:43.831+0000 7f2a63fff640 1 -- 172.21.15.167:0/724316361 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f2a6c0c8980 con 0x7f2a6c14dcc0 2024-05-11T08:06:44.085 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.085+0000 7f2a71262640 1 -- 172.21.15.167:0/724316361 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) -- 0x7f2a6c0694b0 con 0x7f2a6c14dcc0 2024-05-11T08:06:44.086 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.087+0000 7f2a63fff640 1 -- 172.21.15.167:0/724316361 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v5) ==== 90+0+0 (secure 0 0 0) 0x7f2a640608b0 con 0x7f2a6c14dcc0 2024-05-11T08:06:44.089 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.089+0000 7f2a71262640 1 -- 172.21.15.167:0/724316361 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f2a4c076300 msgr2=0x7f2a4c0787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:44.089 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.089+0000 7f2a71262640 1 --2- 172.21.15.167:0/724316361 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f2a4c076300 0x7f2a4c0787c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c006fd0 tx=0x7f2a5c008040 comp rx=0 tx=0).stop 2024-05-11T08:06:44.089 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.089+0000 7f2a71262640 1 -- 172.21.15.167:0/724316361 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2a6c14dcc0 msgr2=0x7f2a6c0b35c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:44.090 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.089+0000 7f2a71262640 1 --2- 172.21.15.167:0/724316361 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2a6c14dcc0 0x7f2a6c0b35c0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f2a6400c7d0 tx=0x7f2a6400cc90 comp rx=0 tx=0).stop 2024-05-11T08:06:44.090 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.089+0000 7f2a71262640 1 -- 172.21.15.167:0/724316361 shutdown_connections 2024-05-11T08:06:44.090 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.089+0000 7f2a71262640 1 --2- 172.21.15.167:0/724316361 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f2a4c076300 0x7f2a4c0787c0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:44.090 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.089+0000 7f2a71262640 1 --2- 172.21.15.167:0/724316361 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2a6c14dcc0 0x7f2a6c0b35c0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:44.090 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.089+0000 7f2a71262640 1 --2- 172.21.15.167:0/724316361 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2a6c15f330 0x7f2a6c14d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:44.090 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.089+0000 7f2a71262640 1 --2- 172.21.15.167:0/724316361 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2a6c0b8800 0x7f2a6c15fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:44.090 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.089+0000 7f2a71262640 1 -- 172.21.15.167:0/724316361 >> 172.21.15.167:0/724316361 conn(0x7f2a6c0b6b80 msgr2=0x7f2a6c1610c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:44.090 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.089+0000 7f2a71262640 1 -- 172.21.15.167:0/724316361 shutdown_connections 2024-05-11T08:06:44.090 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.089+0000 7f2a71262640 1 -- 172.21.15.167:0/724316361 wait complete. 2024-05-11T08:06:44.100 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-05-11T08:06:44.201 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.201+0000 7f94476f1640 1 Processor -- start 2024-05-11T08:06:44.201 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.202+0000 7f94476f1640 1 -- start start 2024-05-11T08:06:44.203 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.202+0000 7f94476f1640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f94400ce580 0x7f944006df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:44.203 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.202+0000 7f94476f1640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f94400b6810 0x7f94400b6c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:44.203 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.202+0000 7f94476f1640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f944015cdc0 con 0x7f94400ce1a0 2024-05-11T08:06:44.203 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.202+0000 7f94476f1640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f94400b60a0 con 0x7f94400ce580 2024-05-11T08:06:44.203 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.202+0000 7f94476f1640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f94400b6380 con 0x7f94400b6810 2024-05-11T08:06:44.203 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.203+0000 7f9444c8c640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f94400ce580 0x7f944006df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:44.203 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.203+0000 7f944548d640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f94400ce1a0 0x7f944015c540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:49716/0 (socket says 172.21.15.167:49716) 2024-05-11T08:06:44.203 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.203+0000 7f944548d640 1 -- 172.21.15.167:0/3696609654 learned_addr learned my addr 172.21.15.167:0/3696609654 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T08:06:44.204 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.203+0000 7f9445c8e640 1 --2- 172.21.15.167:0/3696609654 >> v2:172.21.15.204:3300/0 conn(0x7f94400b6810 0x7f94400b6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:44.204 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.203+0000 7f9444c8c640 1 -- 172.21.15.167:0/3696609654 >> v2:172.21.15.204:3300/0 conn(0x7f94400b6810 msgr2=0x7f94400b6c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:44.204 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.203+0000 7f9444c8c640 1 --2- 172.21.15.167:0/3696609654 >> v2:172.21.15.204:3300/0 conn(0x7f94400b6810 0x7f94400b6c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:44.204 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.203+0000 7f9444c8c640 1 -- 172.21.15.167:0/3696609654 >> v1:172.21.15.167:6789/0 conn(0x7f94400ce1a0 legacy=0x7f944015c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:44.204 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.203+0000 7f9444c8c640 1 -- 172.21.15.167:0/3696609654 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f944015d0a0 con 0x7f94400ce580 2024-05-11T08:06:44.204 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.203+0000 7f9445c8e640 1 --2- 172.21.15.167:0/3696609654 >> v2:172.21.15.204:3300/0 conn(0x7f94400b6810 0x7f94400b6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T08:06:44.205 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.204+0000 7f9444c8c640 1 --2- 172.21.15.167:0/3696609654 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f94400ce580 0x7f944006df10 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f94400cecc0 tx=0x7f943002c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c534420ebf04be6 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:44.205 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.204+0000 7f9437fff640 1 -- 172.21.15.167:0/3696609654 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9430032770 con 0x7f94400ce580 2024-05-11T08:06:44.205 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.204+0000 7f9437fff640 1 -- 172.21.15.167:0/3696609654 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9430038070 con 0x7f94400ce580 2024-05-11T08:06:44.205 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.204+0000 7f9437fff640 1 -- 172.21.15.167:0/3696609654 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9430032b30 con 0x7f94400ce580 2024-05-11T08:06:44.205 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.204+0000 7f94476f1640 1 -- 172.21.15.167:0/3696609654 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f94400ce580 msgr2=0x7f944006df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:44.205 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.204+0000 7f94476f1640 1 --2- 172.21.15.167:0/3696609654 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f94400ce580 0x7f944006df10 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f94400cecc0 tx=0x7f943002c870 comp rx=0 tx=0).stop 2024-05-11T08:06:44.205 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.204+0000 7f94476f1640 1 -- 172.21.15.167:0/3696609654 shutdown_connections 2024-05-11T08:06:44.205 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.204+0000 7f94476f1640 1 --2- 172.21.15.167:0/3696609654 >> v2:172.21.15.204:3300/0 conn(0x7f94400b6810 0x7f94400b6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:44.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.204+0000 7f94476f1640 1 --2- 172.21.15.167:0/3696609654 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f94400ce580 0x7f944006df10 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:44.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.204+0000 7f94476f1640 1 -- 172.21.15.167:0/3696609654 >> 172.21.15.167:0/3696609654 conn(0x7f94400732a0 msgr2=0x7f94400736b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:44.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.204+0000 7f94476f1640 1 -- 172.21.15.167:0/3696609654 shutdown_connections 2024-05-11T08:06:44.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.204+0000 7f94476f1640 1 -- 172.21.15.167:0/3696609654 wait complete. 2024-05-11T08:06:44.208 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.206+0000 7f94476f1640 1 Processor -- start 2024-05-11T08:06:44.208 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.206+0000 7f94476f1640 1 -- start start 2024-05-11T08:06:44.208 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.206+0000 7f94476f1640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f94400b6810 0x7f9440150f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:44.208 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.206+0000 7f94476f1640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f94400ce1a0 0x7f94400b2d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:44.208 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.206+0000 7f94476f1640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f94400ce580 0x7f94400b3270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:44.209 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.206+0000 7f94476f1640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f94400b71c0 con 0x7f94400ce1a0 2024-05-11T08:06:44.209 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.206+0000 7f94476f1640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f94400ce9c0 con 0x7f94400ce580 2024-05-11T08:06:44.209 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.206+0000 7f94476f1640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f94400b7340 con 0x7f94400b6810 2024-05-11T08:06:44.209 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.206+0000 7f9444c8c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f94400ce1a0 0x7f94400b2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:44.209 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.206+0000 7f9444c8c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f94400ce1a0 0x7f94400b2d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:35062/0 (socket says 172.21.15.167:35062) 2024-05-11T08:06:44.209 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.206+0000 7f9444c8c640 1 -- 172.21.15.167:0/3377296935 learned_addr learned my addr 172.21.15.167:0/3377296935 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:44.209 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.207+0000 7f9445c8e640 1 --2- 172.21.15.167:0/3377296935 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f94400ce580 0x7f94400b3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:44.209 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.207+0000 7f944548d640 1 --2- 172.21.15.167:0/3377296935 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f94400b6810 0x7f9440150f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:44.209 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.207+0000 7f9444c8c640 1 -- 172.21.15.167:0/3377296935 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f94400b6810 msgr2=0x7f9440150f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:44.209 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.207+0000 7f9444c8c640 1 --2- 172.21.15.167:0/3377296935 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f94400b6810 0x7f9440150f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:44.209 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.207+0000 7f9444c8c640 1 -- 172.21.15.167:0/3377296935 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f94400ce580 msgr2=0x7f94400b3270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:44.209 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.207+0000 7f9444c8c640 1 --2- 172.21.15.167:0/3377296935 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f94400ce580 0x7f94400b3270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:44.209 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.207+0000 7f9444c8c640 1 -- 172.21.15.167:0/3377296935 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94400703b0 con 0x7f94400ce1a0 2024-05-11T08:06:44.209 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.207+0000 7f9444c8c640 1 --2- 172.21.15.167:0/3377296935 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f94400ce1a0 0x7f94400b2d30 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f943002ce60 tx=0x7f94300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:44.210 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.207+0000 7f9435ffb640 1 -- 172.21.15.167:0/3377296935 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f943004a070 con 0x7f94400ce1a0 2024-05-11T08:06:44.210 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.207+0000 7f9435ffb640 1 -- 172.21.15.167:0/3377296935 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9430033610 con 0x7f94400ce1a0 2024-05-11T08:06:44.210 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.207+0000 7f9435ffb640 1 -- 172.21.15.167:0/3377296935 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9430033a00 con 0x7f94400ce1a0 2024-05-11T08:06:44.210 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.207+0000 7f94476f1640 1 -- 172.21.15.167:0/3377296935 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9440070690 con 0x7f94400ce1a0 2024-05-11T08:06:44.210 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.207+0000 7f94476f1640 1 -- 172.21.15.167:0/3377296935 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9440070c50 con 0x7f94400ce1a0 2024-05-11T08:06:44.210 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.208+0000 7f94476f1640 1 -- 172.21.15.167:0/3377296935 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9440069bc0 con 0x7f94400ce1a0 2024-05-11T08:06:44.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.209+0000 7f9435ffb640 1 -- 172.21.15.167:0/3377296935 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f943004d070 con 0x7f94400ce1a0 2024-05-11T08:06:44.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.209+0000 7f9435ffb640 1 --2- 172.21.15.167:0/3377296935 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f94240762d0 0x7f9424078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:44.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.209+0000 7f9435ffb640 1 -- 172.21.15.167:0/3377296935 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5249+0+0 (secure 0 0 0) 0x7f94300b9cf0 con 0x7f94400ce1a0 2024-05-11T08:06:44.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.210+0000 7f944548d640 1 --2- 172.21.15.167:0/3377296935 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f94240762d0 0x7f9424078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:44.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.210+0000 7f944548d640 1 --2- 172.21.15.167:0/3377296935 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f94240762d0 0x7f9424078790 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f943c0012e0 tx=0x7f943c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:44.214 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.214+0000 7f9435ffb640 1 -- 172.21.15.167:0/3377296935 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f94300be070 con 0x7f94400ce1a0 2024-05-11T08:06:44.461 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.461+0000 7f94476f1640 1 -- 172.21.15.167:0/3377296935 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) -- 0x7f94400cefc0 con 0x7f94400ce1a0 2024-05-11T08:06:44.462 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.463+0000 7f9435ffb640 1 -- 172.21.15.167:0/3377296935 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v6) ==== 90+0+0 (secure 0 0 0) 0x7f9430083810 con 0x7f94400ce1a0 2024-05-11T08:06:44.465 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.465+0000 7f94476f1640 1 -- 172.21.15.167:0/3377296935 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f94240762d0 msgr2=0x7f9424078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:44.465 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.465+0000 7f94476f1640 1 --2- 172.21.15.167:0/3377296935 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f94240762d0 0x7f9424078790 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f943c0012e0 tx=0x7f943c009040 comp rx=0 tx=0).stop 2024-05-11T08:06:44.465 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.465+0000 7f94476f1640 1 -- 172.21.15.167:0/3377296935 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f94400ce1a0 msgr2=0x7f94400b2d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:44.465 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.465+0000 7f94476f1640 1 --2- 172.21.15.167:0/3377296935 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f94400ce1a0 0x7f94400b2d30 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f943002ce60 tx=0x7f94300046c0 comp rx=0 tx=0).stop 2024-05-11T08:06:44.465 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.465+0000 7f94476f1640 1 -- 172.21.15.167:0/3377296935 shutdown_connections 2024-05-11T08:06:44.466 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.465+0000 7f94476f1640 1 --2- 172.21.15.167:0/3377296935 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f94240762d0 0x7f9424078790 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:44.466 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.465+0000 7f94476f1640 1 --2- 172.21.15.167:0/3377296935 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f94400ce580 0x7f94400b3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:44.466 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.465+0000 7f94476f1640 1 --2- 172.21.15.167:0/3377296935 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f94400ce1a0 0x7f94400b2d30 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:44.466 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.465+0000 7f94476f1640 1 --2- 172.21.15.167:0/3377296935 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f94400b6810 0x7f9440150f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:44.466 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.465+0000 7f94476f1640 1 -- 172.21.15.167:0/3377296935 >> 172.21.15.167:0/3377296935 conn(0x7f94400732a0 msgr2=0x7f94400b5820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:44.466 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.465+0000 7f94476f1640 1 -- 172.21.15.167:0/3377296935 shutdown_connections 2024-05-11T08:06:44.466 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.465+0000 7f94476f1640 1 -- 172.21.15.167:0/3377296935 wait complete. 2024-05-11T08:06:44.476 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-05-11T08:06:44.577 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.577+0000 7fb2ac6bb640 1 Processor -- start 2024-05-11T08:06:44.577 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.577+0000 7fb2ac6bb640 1 -- start start 2024-05-11T08:06:44.581 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.578+0000 7fb2ac6bb640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb2a4073270 0x7fb2a415d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:44.581 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.578+0000 7fb2ac6bb640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2a415c6d0 con 0x7fb2a4073650 2024-05-11T08:06:44.581 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.578+0000 7fb2ac6bb640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2a40700b0 con 0x7fb2a40ce8f0 2024-05-11T08:06:44.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.578+0000 7fb2ac6bb640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fb2a406ead0 con 0x7fb2a4073270 2024-05-11T08:06:44.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.579+0000 7fb2a9c56640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7fb2a4073650 0x7fb2a406e3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:55936/0 (socket says 172.21.15.167:55936) 2024-05-11T08:06:44.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.579+0000 7fb2a9c56640 1 -- 172.21.15.167:0/4069831793 learned_addr learned my addr 172.21.15.167:0/4069831793 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T08:06:44.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.579+0000 7fb2aa457640 1 --2- 172.21.15.167:0/4069831793 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb2a4073270 0x7fb2a415d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:44.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.579+0000 7fb2a9455640 1 -- 172.21.15.167:0/4069831793 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3257412672 0 0) 0x7fb2a415c6d0 con 0x7fb2a4073650 2024-05-11T08:06:44.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.579+0000 7fb2a9455640 1 -- 172.21.15.167:0/4069831793 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb290003520 con 0x7fb2a4073650 2024-05-11T08:06:44.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.579+0000 7fb2a9455640 1 -- 172.21.15.167:0/4069831793 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3011906897 0 0) 0x7fb2a40700b0 con 0x7fb2a40ce8f0 2024-05-11T08:06:44.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.579+0000 7fb2a9455640 1 -- 172.21.15.167:0/4069831793 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb2a415c6d0 con 0x7fb2a40ce8f0 2024-05-11T08:06:44.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.579+0000 7fb2a9455640 1 -- 172.21.15.167:0/4069831793 <== mon.0 v1:172.21.15.167:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 828419142 0 0) 0x7fb290003520 con 0x7fb2a4073650 2024-05-11T08:06:44.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.579+0000 7fb2a9455640 1 -- 172.21.15.167:0/4069831793 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb2a4073270 msgr2=0x7fb2a415d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:44.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.579+0000 7fb2a9455640 1 --2- 172.21.15.167:0/4069831793 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb2a4073270 0x7fb2a415d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:44.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.579+0000 7fb2a9455640 1 -- 172.21.15.167:0/4069831793 >> v1:172.21.15.204:6789/0 conn(0x7fb2a40ce8f0 legacy=0x7fb2a40ced80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:44.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.579+0000 7fb2a9455640 1 -- 172.21.15.167:0/4069831793 --> v1:172.21.15.167:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2a415c970 con 0x7fb2a4073650 2024-05-11T08:06:44.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.579+0000 7fb2a9455640 1 -- 172.21.15.167:0/4069831793 <== mon.0 v1:172.21.15.167:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3602277846 0 0) 0x7fb2940016f0 con 0x7fb2a4073650 2024-05-11T08:06:44.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.579+0000 7fb2a9455640 1 -- 172.21.15.167:0/4069831793 >> v1:172.21.15.167:6789/0 conn(0x7fb2a4073650 legacy=0x7fb2a406e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:44.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.579+0000 7fb2a9455640 1 --2- 172.21.15.167:0/4069831793 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb290003c10 0x7fb290004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:44.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.579+0000 7fb2a9455640 1 --2- 172.21.15.167:0/4069831793 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb290004800 0x7fb290004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:44.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.579+0000 7fb2a9455640 1 --2- 172.21.15.167:0/4069831793 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb2900051c0 0x7fb29000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:44.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.579+0000 7fb2a9455640 1 -- 172.21.15.167:0/4069831793 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb2a406ead0 con 0x7fb290004800 2024-05-11T08:06:44.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.579+0000 7fb2a9455640 1 -- 172.21.15.167:0/4069831793 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb29000c090 con 0x7fb290003c10 2024-05-11T08:06:44.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.579+0000 7fb2a9455640 1 -- 172.21.15.167:0/4069831793 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fb29000c290 con 0x7fb2900051c0 2024-05-11T08:06:44.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.579+0000 7fb2aa457640 1 --2- 172.21.15.167:0/4069831793 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb2a4073270 0x7fb2a415d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T08:06:44.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.579+0000 7fb2a9c56640 1 --2- 172.21.15.167:0/4069831793 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb290004800 0x7fb290004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:44.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.579+0000 7fb2aac58640 1 --2- 172.21.15.167:0/4069831793 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb2900051c0 0x7fb29000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:44.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.579+0000 7fb2aa457640 1 --2- 172.21.15.167:0/4069831793 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb290003c10 0x7fb290004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:44.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.580+0000 7fb2a9c56640 1 -- 172.21.15.167:0/4069831793 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb2900051c0 msgr2=0x7fb29000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:44.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.580+0000 7fb2a9c56640 1 --2- 172.21.15.167:0/4069831793 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb2900051c0 0x7fb29000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:44.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.580+0000 7fb2a9c56640 1 -- 172.21.15.167:0/4069831793 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb290003c10 msgr2=0x7fb290004010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:44.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.580+0000 7fb2a9c56640 1 --2- 172.21.15.167:0/4069831793 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb290003c10 0x7fb290004010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:44.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.580+0000 7fb2a9c56640 1 -- 172.21.15.167:0/4069831793 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb290003520 con 0x7fb290004800 2024-05-11T08:06:44.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.580+0000 7fb2aac58640 1 --2- 172.21.15.167:0/4069831793 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb2900051c0 0x7fb29000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T08:06:44.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.580+0000 7fb2aa457640 1 --2- 172.21.15.167:0/4069831793 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb290003c10 0x7fb290004010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T08:06:44.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.580+0000 7fb2a9c56640 1 --2- 172.21.15.167:0/4069831793 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb290004800 0x7fb290004c80 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fb29400c190 tx=0x7fb290005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6675440c9fb9c274 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:44.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.580+0000 7fb2a9455640 1 -- 172.21.15.167:0/4069831793 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb294007cc0 con 0x7fb290004800 2024-05-11T08:06:44.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.580+0000 7fb2a9455640 1 -- 172.21.15.167:0/4069831793 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb294005ad0 con 0x7fb290004800 2024-05-11T08:06:44.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.580+0000 7fb2a9455640 1 -- 172.21.15.167:0/4069831793 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb294007070 con 0x7fb290004800 2024-05-11T08:06:44.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.581+0000 7fb2ac6bb640 1 -- 172.21.15.167:0/4069831793 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb290004800 msgr2=0x7fb290004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:44.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.581+0000 7fb2ac6bb640 1 --2- 172.21.15.167:0/4069831793 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb290004800 0x7fb290004c80 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fb29400c190 tx=0x7fb290005830 comp rx=0 tx=0).stop 2024-05-11T08:06:44.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.581+0000 7fb2aac58640 1 -- 172.21.15.167:0/4069831793 reap_dead start 2024-05-11T08:06:44.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.581+0000 7fb2ac6bb640 1 -- 172.21.15.167:0/4069831793 shutdown_connections 2024-05-11T08:06:44.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.581+0000 7fb2ac6bb640 1 -- 172.21.15.167:0/4069831793 >> 172.21.15.167:0/4069831793 conn(0x7fb2a406c450 msgr2=0x7fb2a406c860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:44.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.581+0000 7fb2ac6bb640 1 -- 172.21.15.167:0/4069831793 shutdown_connections 2024-05-11T08:06:44.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.581+0000 7fb2ac6bb640 1 -- 172.21.15.167:0/4069831793 wait complete. 2024-05-11T08:06:44.587 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.582+0000 7fb2ac6bb640 1 Processor -- start 2024-05-11T08:06:44.587 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.582+0000 7fb2ac6bb640 1 -- start start 2024-05-11T08:06:44.587 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.583+0000 7fb2ac6bb640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb290004800 0x7fb2a41f9ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:44.587 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.583+0000 7fb2ac6bb640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb2a40b18b0 0x7fb2a41f5b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:44.587 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.583+0000 7fb2ac6bb640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb2a40b1d30 0x7fb2a41e5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:44.587 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.583+0000 7fb2ac6bb640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb2a415cce0 con 0x7fb290004800 2024-05-11T08:06:44.587 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.583+0000 7fb2ac6bb640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb2a406fd60 con 0x7fb2a40b18b0 2024-05-11T08:06:44.587 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.583+0000 7fb2ac6bb640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fb2a4073ad0 con 0x7fb2a40b1d30 2024-05-11T08:06:44.587 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.583+0000 7fb2aa457640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb290004800 0x7fb2a41f9ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:44.587 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.583+0000 7fb2aa457640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb290004800 0x7fb2a41f9ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:43852/0 (socket says 172.21.15.167:43852) 2024-05-11T08:06:44.587 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.583+0000 7fb2aa457640 1 -- 172.21.15.167:0/4214520009 learned_addr learned my addr 172.21.15.167:0/4214520009 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:44.587 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.583+0000 7fb2aa457640 1 -- 172.21.15.167:0/4214520009 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb2a40b1d30 msgr2=0x7fb2a41e5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:44.587 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.584+0000 7fb2aa457640 1 --2- 172.21.15.167:0/4214520009 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb2a40b1d30 0x7fb2a41e5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:44.587 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.584+0000 7fb2aa457640 1 -- 172.21.15.167:0/4214520009 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb2a40b18b0 msgr2=0x7fb2a41f5b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:44.588 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.584+0000 7fb2aa457640 1 --2- 172.21.15.167:0/4214520009 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb2a40b18b0 0x7fb2a41f5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:44.588 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.584+0000 7fb2aa457640 1 -- 172.21.15.167:0/4214520009 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2a41f6620 con 0x7fb290004800 2024-05-11T08:06:44.588 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.584+0000 7fb2aa457640 1 --2- 172.21.15.167:0/4214520009 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb290004800 0x7fb2a41f9ba0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fb2a0011830 tx=0x7fb2a0011cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:44.588 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.584+0000 7fb29affd640 1 -- 172.21.15.167:0/4214520009 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2a000f9c0 con 0x7fb290004800 2024-05-11T08:06:44.588 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.584+0000 7fb29affd640 1 -- 172.21.15.167:0/4214520009 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2a000fb60 con 0x7fb290004800 2024-05-11T08:06:44.588 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.584+0000 7fb29affd640 1 -- 172.21.15.167:0/4214520009 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2a0015460 con 0x7fb290004800 2024-05-11T08:06:44.588 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.584+0000 7fb2ac6bb640 1 -- 172.21.15.167:0/4214520009 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2a41f6900 con 0x7fb290004800 2024-05-11T08:06:44.588 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.584+0000 7fb2ac6bb640 1 -- 172.21.15.167:0/4214520009 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2a40afe90 con 0x7fb290004800 2024-05-11T08:06:44.588 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.585+0000 7fb2ac6bb640 1 -- 172.21.15.167:0/4214520009 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2a40694b0 con 0x7fb290004800 2024-05-11T08:06:44.588 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.585+0000 7fb29affd640 1 -- 172.21.15.167:0/4214520009 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fb2a0015810 con 0x7fb290004800 2024-05-11T08:06:44.588 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.586+0000 7fb29affd640 1 --2- 172.21.15.167:0/4214520009 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fb288076300 0x7fb2880787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:44.588 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.586+0000 7fb29affd640 1 -- 172.21.15.167:0/4214520009 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5249+0+0 (secure 0 0 0) 0x7fb2a0097fb0 con 0x7fb290004800 2024-05-11T08:06:44.588 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.586+0000 7fb2a9c56640 1 --2- 172.21.15.167:0/4214520009 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fb288076300 0x7fb2880787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:44.588 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.586+0000 7fb2a9c56640 1 --2- 172.21.15.167:0/4214520009 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fb288076300 0x7fb2880787c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb29400c190 tx=0x7fb294009030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:44.590 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.590+0000 7fb29affd640 1 -- 172.21.15.167:0/4214520009 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fb2a0019030 con 0x7fb290004800 2024-05-11T08:06:44.839 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.839+0000 7fb2ac6bb640 1 -- 172.21.15.167:0/4214520009 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) -- 0x7fb2a40d0a60 con 0x7fb290004800 2024-05-11T08:06:44.841 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.841+0000 7fb29affd640 1 -- 172.21.15.167:0/4214520009 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v7) ==== 97+0+0 (secure 0 0 0) 0x7fb2a0061ad0 con 0x7fb290004800 2024-05-11T08:06:44.843 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.843+0000 7fb2ac6bb640 1 -- 172.21.15.167:0/4214520009 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fb288076300 msgr2=0x7fb2880787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:44.844 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.843+0000 7fb2ac6bb640 1 --2- 172.21.15.167:0/4214520009 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fb288076300 0x7fb2880787c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb29400c190 tx=0x7fb294009030 comp rx=0 tx=0).stop 2024-05-11T08:06:44.844 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.843+0000 7fb2ac6bb640 1 -- 172.21.15.167:0/4214520009 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb290004800 msgr2=0x7fb2a41f9ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:44.844 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.843+0000 7fb2ac6bb640 1 --2- 172.21.15.167:0/4214520009 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb290004800 0x7fb2a41f9ba0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fb2a0011830 tx=0x7fb2a0011cf0 comp rx=0 tx=0).stop 2024-05-11T08:06:44.844 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.843+0000 7fb2ac6bb640 1 -- 172.21.15.167:0/4214520009 shutdown_connections 2024-05-11T08:06:44.844 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.843+0000 7fb2ac6bb640 1 --2- 172.21.15.167:0/4214520009 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fb288076300 0x7fb2880787c0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:44.844 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.843+0000 7fb2ac6bb640 1 --2- 172.21.15.167:0/4214520009 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb2a40b1d30 0x7fb2a41e5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:44.844 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.843+0000 7fb2ac6bb640 1 --2- 172.21.15.167:0/4214520009 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb2a40b18b0 0x7fb2a41f5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:44.844 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.843+0000 7fb2ac6bb640 1 --2- 172.21.15.167:0/4214520009 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb290004800 0x7fb2a41f9ba0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:44.844 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.843+0000 7fb2ac6bb640 1 -- 172.21.15.167:0/4214520009 >> 172.21.15.167:0/4214520009 conn(0x7fb2a406c450 msgr2=0x7fb2a40cfbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:44.844 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.844+0000 7fb2ac6bb640 1 -- 172.21.15.167:0/4214520009 shutdown_connections 2024-05-11T08:06:44.844 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.844+0000 7fb2ac6bb640 1 -- 172.21.15.167:0/4214520009 wait complete. 2024-05-11T08:06:44.855 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-05-11T08:06:44.957 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.956+0000 7ffb2d104640 1 Processor -- start 2024-05-11T08:06:44.957 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.956+0000 7ffb2d104640 1 -- start start 2024-05-11T08:06:44.958 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.957+0000 7ffb2d104640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7ffb28146910 0x7ffb280d1930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:44.958 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.957+0000 7ffb2d104640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ffb28071a40 0x7ffb280cd590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:44.958 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.957+0000 7ffb2d104640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb280d0bb0 con 0x7ffb28160b90 2024-05-11T08:06:44.958 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.957+0000 7ffb2d104640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7ffb280d0e50 con 0x7ffb28146910 2024-05-11T08:06:44.959 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.957+0000 7ffb2d104640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7ffb280d1130 con 0x7ffb28071a40 2024-05-11T08:06:44.959 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.958+0000 7ffb2759e640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7ffb28160b90 0x7ffb28070000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:55952/0 (socket says 172.21.15.167:55952) 2024-05-11T08:06:44.959 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.958+0000 7ffb2759e640 1 -- 172.21.15.167:0/1383436847 learned_addr learned my addr 172.21.15.167:0/1383436847 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T08:06:44.959 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.958+0000 7ffb2659c640 1 --2- 172.21.15.167:0/1383436847 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ffb28071a40 0x7ffb280cd590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:44.959 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.958+0000 7ffb26d9d640 1 --2- 172.21.15.167:0/1383436847 >> v2:172.21.15.204:3300/0 conn(0x7ffb28146910 0x7ffb280d1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:44.959 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.958+0000 7ffb25d9b640 1 -- 172.21.15.167:0/1383436847 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3392495026 0 0) 0x7ffb280d0bb0 con 0x7ffb28160b90 2024-05-11T08:06:44.959 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.958+0000 7ffb25d9b640 1 -- 172.21.15.167:0/1383436847 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffb14003520 con 0x7ffb28160b90 2024-05-11T08:06:44.959 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.958+0000 7ffb26d9d640 1 -- 172.21.15.167:0/1383436847 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ffb28071a40 msgr2=0x7ffb280cd590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:44.959 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.958+0000 7ffb26d9d640 1 --2- 172.21.15.167:0/1383436847 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ffb28071a40 0x7ffb280cd590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:44.959 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.958+0000 7ffb26d9d640 1 -- 172.21.15.167:0/1383436847 >> v1:172.21.15.167:6789/0 conn(0x7ffb28160b90 legacy=0x7ffb28070000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:44.959 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.958+0000 7ffb26d9d640 1 -- 172.21.15.167:0/1383436847 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb280d1410 con 0x7ffb28146910 2024-05-11T08:06:44.959 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.958+0000 7ffb2659c640 1 --2- 172.21.15.167:0/1383436847 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ffb28071a40 0x7ffb280cd590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T08:06:44.960 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.959+0000 7ffb26d9d640 1 --2- 172.21.15.167:0/1383436847 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffb28146910 0x7ffb280d1930 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ffb180097a0 tx=0x7ffb1802a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b8249453f293377 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:44.961 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.959+0000 7ffb25d9b640 1 -- 172.21.15.167:0/1383436847 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb18037070 con 0x7ffb28146910 2024-05-11T08:06:44.961 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.959+0000 7ffb25d9b640 1 -- 172.21.15.167:0/1383436847 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb18032440 con 0x7ffb28146910 2024-05-11T08:06:44.961 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.959+0000 7ffb25d9b640 1 -- 172.21.15.167:0/1383436847 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb180327f0 con 0x7ffb28146910 2024-05-11T08:06:44.961 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.959+0000 7ffb2d104640 1 -- 172.21.15.167:0/1383436847 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffb28146910 msgr2=0x7ffb280d1930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:44.961 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.959+0000 7ffb2d104640 1 --2- 172.21.15.167:0/1383436847 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffb28146910 0x7ffb280d1930 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ffb180097a0 tx=0x7ffb1802a180 comp rx=0 tx=0).stop 2024-05-11T08:06:44.961 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.959+0000 7ffb2d104640 1 -- 172.21.15.167:0/1383436847 shutdown_connections 2024-05-11T08:06:44.961 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.959+0000 7ffb2d104640 1 --2- 172.21.15.167:0/1383436847 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ffb28071a40 0x7ffb280cd590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:44.961 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.959+0000 7ffb2d104640 1 --2- 172.21.15.167:0/1383436847 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffb28146910 0x7ffb280d1930 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:44.961 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.960+0000 7ffb2d104640 1 -- 172.21.15.167:0/1383436847 >> 172.21.15.167:0/1383436847 conn(0x7ffb280b4c30 msgr2=0x7ffb280b5040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:44.961 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.960+0000 7ffb2d104640 1 -- 172.21.15.167:0/1383436847 shutdown_connections 2024-05-11T08:06:44.961 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.960+0000 7ffb2d104640 1 -- 172.21.15.167:0/1383436847 wait complete. 2024-05-11T08:06:44.963 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.961+0000 7ffb2d104640 1 Processor -- start 2024-05-11T08:06:44.964 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.961+0000 7ffb2d104640 1 -- start start 2024-05-11T08:06:44.964 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.962+0000 7ffb2d104640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffb28071a40 0x7ffb2815f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:44.964 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.962+0000 7ffb2d104640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ffb28146910 0x7ffb2806e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:44.964 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.962+0000 7ffb2d104640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffb28160b90 0x7ffb28150d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:44.964 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.962+0000 7ffb2d104640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ffb28146cf0 con 0x7ffb28160b90 2024-05-11T08:06:44.964 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.962+0000 7ffb2d104640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ffb280cdad0 con 0x7ffb28071a40 2024-05-11T08:06:44.964 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.962+0000 7ffb2d104640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7ffb28146e70 con 0x7ffb28146910 2024-05-11T08:06:44.964 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.962+0000 7ffb2759e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffb28160b90 0x7ffb28150d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:44.964 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.962+0000 7ffb2759e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffb28160b90 0x7ffb28150d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:43868/0 (socket says 172.21.15.167:43868) 2024-05-11T08:06:44.964 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.962+0000 7ffb2759e640 1 -- 172.21.15.167:0/2105143767 learned_addr learned my addr 172.21.15.167:0/2105143767 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:44.964 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.962+0000 7ffb26d9d640 1 --2- 172.21.15.167:0/2105143767 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffb28071a40 0x7ffb2815f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:44.964 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.962+0000 7ffb2659c640 1 --2- 172.21.15.167:0/2105143767 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ffb28146910 0x7ffb2806e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:44.964 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.962+0000 7ffb2759e640 1 -- 172.21.15.167:0/2105143767 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ffb28146910 msgr2=0x7ffb2806e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:44.964 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.962+0000 7ffb2759e640 1 --2- 172.21.15.167:0/2105143767 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ffb28146910 0x7ffb2806e7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:44.965 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.962+0000 7ffb2759e640 1 -- 172.21.15.167:0/2105143767 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffb28071a40 msgr2=0x7ffb2815f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:44.965 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.962+0000 7ffb2759e640 1 --2- 172.21.15.167:0/2105143767 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffb28071a40 0x7ffb2815f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:44.965 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.962+0000 7ffb2759e640 1 -- 172.21.15.167:0/2105143767 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb2806ef70 con 0x7ffb28160b90 2024-05-11T08:06:44.965 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.962+0000 7ffb2759e640 1 --2- 172.21.15.167:0/2105143767 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffb28160b90 0x7ffb28150d60 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7ffb2000cb40 tx=0x7ffb2000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:44.965 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.963+0000 7ffb1f7fe640 1 -- 172.21.15.167:0/2105143767 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb2000aca0 con 0x7ffb28160b90 2024-05-11T08:06:44.965 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.963+0000 7ffb1f7fe640 1 -- 172.21.15.167:0/2105143767 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb2000ae40 con 0x7ffb28160b90 2024-05-11T08:06:44.965 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.963+0000 7ffb2d104640 1 -- 172.21.15.167:0/2105143767 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb2806f250 con 0x7ffb28160b90 2024-05-11T08:06:44.965 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.963+0000 7ffb1f7fe640 1 -- 172.21.15.167:0/2105143767 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb20012600 con 0x7ffb28160b90 2024-05-11T08:06:44.965 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.963+0000 7ffb2d104640 1 -- 172.21.15.167:0/2105143767 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb2806f810 con 0x7ffb28160b90 2024-05-11T08:06:44.965 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.964+0000 7ffb2d104640 1 -- 172.21.15.167:0/2105143767 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb28069bc0 con 0x7ffb28160b90 2024-05-11T08:06:44.966 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.964+0000 7ffb1f7fe640 1 -- 172.21.15.167:0/2105143767 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7ffb200127a0 con 0x7ffb28160b90 2024-05-11T08:06:44.966 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.965+0000 7ffb1f7fe640 1 --2- 172.21.15.167:0/2105143767 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7ffb08076300 0x7ffb080787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:44.966 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.965+0000 7ffb26d9d640 1 --2- 172.21.15.167:0/2105143767 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7ffb08076300 0x7ffb080787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:44.966 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.965+0000 7ffb1f7fe640 1 -- 172.21.15.167:0/2105143767 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5249+0+0 (secure 0 0 0) 0x7ffb20097aa0 con 0x7ffb28160b90 2024-05-11T08:06:44.966 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.965+0000 7ffb26d9d640 1 --2- 172.21.15.167:0/2105143767 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7ffb08076300 0x7ffb080787c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ffb18034000 tx=0x7ffb1802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:44.969 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:44.969+0000 7ffb1f7fe640 1 -- 172.21.15.167:0/2105143767 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7ffb200615c0 con 0x7ffb28160b90 2024-05-11T08:06:45.226 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.226+0000 7ffb2d104640 1 -- 172.21.15.167:0/2105143767 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) -- 0x7ffb28068380 con 0x7ffb28160b90 2024-05-11T08:06:45.227 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.228+0000 7ffb1f7fe640 1 -- 172.21.15.167:0/2105143767 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v8) ==== 90+0+0 (secure 0 0 0) 0x7ffb20060ec0 con 0x7ffb28160b90 2024-05-11T08:06:45.231 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.230+0000 7ffb2d104640 1 -- 172.21.15.167:0/2105143767 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7ffb08076300 msgr2=0x7ffb080787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:45.231 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.230+0000 7ffb2d104640 1 --2- 172.21.15.167:0/2105143767 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7ffb08076300 0x7ffb080787c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ffb18034000 tx=0x7ffb1802e000 comp rx=0 tx=0).stop 2024-05-11T08:06:45.231 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.230+0000 7ffb2d104640 1 -- 172.21.15.167:0/2105143767 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffb28160b90 msgr2=0x7ffb28150d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:45.231 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.230+0000 7ffb2d104640 1 --2- 172.21.15.167:0/2105143767 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffb28160b90 0x7ffb28150d60 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7ffb2000cb40 tx=0x7ffb2000a060 comp rx=0 tx=0).stop 2024-05-11T08:06:45.231 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.230+0000 7ffb2d104640 1 -- 172.21.15.167:0/2105143767 shutdown_connections 2024-05-11T08:06:45.231 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.230+0000 7ffb2d104640 1 --2- 172.21.15.167:0/2105143767 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7ffb08076300 0x7ffb080787c0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:45.231 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.230+0000 7ffb2d104640 1 --2- 172.21.15.167:0/2105143767 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffb28160b90 0x7ffb28150d60 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:45.231 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.230+0000 7ffb2d104640 1 --2- 172.21.15.167:0/2105143767 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ffb28146910 0x7ffb2806e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:45.231 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.230+0000 7ffb2d104640 1 --2- 172.21.15.167:0/2105143767 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffb28071a40 0x7ffb2815f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:45.231 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.230+0000 7ffb2d104640 1 -- 172.21.15.167:0/2105143767 >> 172.21.15.167:0/2105143767 conn(0x7ffb280b4c30 msgr2=0x7ffb280b7940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:45.231 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.230+0000 7ffb2d104640 1 -- 172.21.15.167:0/2105143767 shutdown_connections 2024-05-11T08:06:45.231 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.230+0000 7ffb2d104640 1 -- 172.21.15.167:0/2105143767 wait complete. 2024-05-11T08:06:45.241 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-11T08:06:45.342 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.342+0000 7f0fe1151640 1 Processor -- start 2024-05-11T08:06:45.342 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.342+0000 7f0fe1151640 1 -- start start 2024-05-11T08:06:45.343 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.343+0000 7f0fe1151640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f0fdc13eeb0 0x7f0fdc06f480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:45.343 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.343+0000 7f0fe1151640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f0fdc13f290 0x7f0fdc06f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:45.343 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.343+0000 7f0fe1151640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0fdc15fb90 con 0x7f0fdc0b50a0 2024-05-11T08:06:45.344 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.343+0000 7f0fe1151640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f0fdc15d5b0 con 0x7f0fdc13eeb0 2024-05-11T08:06:45.344 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.343+0000 7f0fe1151640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f0fdc15fe70 con 0x7f0fdc13f290 2024-05-11T08:06:45.344 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.343+0000 7f0fdad9d640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f0fdc13eeb0 0x7f0fdc06f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:45.344 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.343+0000 7f0fdad9d640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f0fdc13eeb0 0x7f0fdc06f480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:43878/0 (socket says 172.21.15.167:43878) 2024-05-11T08:06:45.344 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.343+0000 7f0fdad9d640 1 -- 172.21.15.167:0/2989371594 learned_addr learned my addr 172.21.15.167:0/2989371594 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:45.344 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.343+0000 7f0fda59c640 1 --2- 172.21.15.167:0/2989371594 >> v2:172.21.15.204:3300/0 conn(0x7f0fdc13f290 0x7f0fdc06f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:45.344 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.343+0000 7f0fdad9d640 1 -- 172.21.15.167:0/2989371594 >> v2:172.21.15.204:3300/0 conn(0x7f0fdc13f290 msgr2=0x7f0fdc06f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:45.344 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.343+0000 7f0fdad9d640 1 --2- 172.21.15.167:0/2989371594 >> v2:172.21.15.204:3300/0 conn(0x7f0fdc13f290 0x7f0fdc06f9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:45.344 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.343+0000 7f0fdad9d640 1 -- 172.21.15.167:0/2989371594 >> v1:172.21.15.204:6789/0 conn(0x7f0fdc0b50a0 legacy=0x7f0fdc0735c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:45.344 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.343+0000 7f0fdad9d640 1 -- 172.21.15.167:0/2989371594 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fdc160150 con 0x7f0fdc13eeb0 2024-05-11T08:06:45.345 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.344+0000 7f0fdad9d640 1 --2- 172.21.15.167:0/2989371594 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0fdc13eeb0 0x7f0fdc06f480 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc074520 tx=0x7f0fcc02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d2f2cd2339422f1 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:45.345 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.344+0000 7f0fd9d9b640 1 -- 172.21.15.167:0/2989371594 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fcc037070 con 0x7f0fdc13eeb0 2024-05-11T08:06:45.345 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.344+0000 7f0fd9d9b640 1 -- 172.21.15.167:0/2989371594 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fcc032440 con 0x7f0fdc13eeb0 2024-05-11T08:06:45.345 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.344+0000 7f0fd9d9b640 1 -- 172.21.15.167:0/2989371594 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fcc032810 con 0x7f0fdc13eeb0 2024-05-11T08:06:45.346 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.344+0000 7f0fe1151640 1 -- 172.21.15.167:0/2989371594 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0fdc13eeb0 msgr2=0x7f0fdc06f480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:45.346 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.344+0000 7f0fe1151640 1 --2- 172.21.15.167:0/2989371594 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0fdc13eeb0 0x7f0fdc06f480 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc074520 tx=0x7f0fcc02a140 comp rx=0 tx=0).stop 2024-05-11T08:06:45.346 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.344+0000 7f0fe1151640 1 -- 172.21.15.167:0/2989371594 shutdown_connections 2024-05-11T08:06:45.346 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.344+0000 7f0fe1151640 1 --2- 172.21.15.167:0/2989371594 >> v2:172.21.15.204:3300/0 conn(0x7f0fdc13f290 0x7f0fdc06f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:45.346 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.344+0000 7f0fe1151640 1 --2- 172.21.15.167:0/2989371594 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0fdc13eeb0 0x7f0fdc06f480 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:45.346 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.344+0000 7f0fe1151640 1 -- 172.21.15.167:0/2989371594 >> 172.21.15.167:0/2989371594 conn(0x7f0fdc0cd8f0 msgr2=0x7f0fdc0cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:45.346 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.345+0000 7f0fe1151640 1 -- 172.21.15.167:0/2989371594 shutdown_connections 2024-05-11T08:06:45.346 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.345+0000 7f0fe1151640 1 -- 172.21.15.167:0/2989371594 wait complete. 2024-05-11T08:06:45.348 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.346+0000 7f0fe1151640 1 Processor -- start 2024-05-11T08:06:45.348 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.346+0000 7f0fe1151640 1 -- start start 2024-05-11T08:06:45.348 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.347+0000 7f0fe1151640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0fdc0b50a0 0x7f0fdc13dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:45.348 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.347+0000 7f0fe1151640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0fdc13eeb0 0x7f0fdc128c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:45.348 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.347+0000 7f0fe1151640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0fdc13f290 0x7f0fdc06e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:45.348 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.347+0000 7f0fe1151640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0fdc073cd0 con 0x7f0fdc0b50a0 2024-05-11T08:06:45.348 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.347+0000 7f0fe1151640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0fdc160710 con 0x7f0fdc13eeb0 2024-05-11T08:06:45.348 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.347+0000 7f0fe1151640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f0fdc073e50 con 0x7f0fdc13f290 2024-05-11T08:06:45.349 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.347+0000 7f0fdad9d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0fdc0b50a0 0x7f0fdc13dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:45.349 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.347+0000 7f0fdad9d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0fdc0b50a0 0x7f0fdc13dc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:43888/0 (socket says 172.21.15.167:43888) 2024-05-11T08:06:45.349 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.347+0000 7f0fdad9d640 1 -- 172.21.15.167:0/2530238444 learned_addr learned my addr 172.21.15.167:0/2530238444 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:45.349 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.347+0000 7f0fda59c640 1 --2- 172.21.15.167:0/2530238444 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0fdc13eeb0 0x7f0fdc128c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:45.349 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.347+0000 7f0fdb59e640 1 --2- 172.21.15.167:0/2530238444 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0fdc13f290 0x7f0fdc06e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:45.349 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.347+0000 7f0fdad9d640 1 -- 172.21.15.167:0/2530238444 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0fdc13f290 msgr2=0x7f0fdc06e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:45.349 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.347+0000 7f0fdad9d640 1 --2- 172.21.15.167:0/2530238444 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0fdc13f290 0x7f0fdc06e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:45.349 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.347+0000 7f0fdad9d640 1 -- 172.21.15.167:0/2530238444 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0fdc13eeb0 msgr2=0x7f0fdc128c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:45.349 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.347+0000 7f0fdad9d640 1 --2- 172.21.15.167:0/2530238444 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0fdc13eeb0 0x7f0fdc128c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:45.349 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.347+0000 7f0fdad9d640 1 -- 172.21.15.167:0/2530238444 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fdc129ba0 con 0x7f0fdc0b50a0 2024-05-11T08:06:45.349 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.347+0000 7f0fdad9d640 1 --2- 172.21.15.167:0/2530238444 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0fdc0b50a0 0x7f0fdc13dc40 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f0fcc000c00 tx=0x7f0fcc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:45.349 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.347+0000 7f0fc37fe640 1 -- 172.21.15.167:0/2530238444 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fcc041020 con 0x7f0fdc0b50a0 2024-05-11T08:06:45.349 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.348+0000 7f0fc37fe640 1 -- 172.21.15.167:0/2530238444 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fcc0306c0 con 0x7f0fdc0b50a0 2024-05-11T08:06:45.349 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.348+0000 7f0fc37fe640 1 -- 172.21.15.167:0/2530238444 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fcc037040 con 0x7f0fdc0b50a0 2024-05-11T08:06:45.349 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.348+0000 7f0fe1151640 1 -- 172.21.15.167:0/2530238444 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fdc1295e0 con 0x7f0fdc0b50a0 2024-05-11T08:06:45.350 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.348+0000 7f0fe1151640 1 -- 172.21.15.167:0/2530238444 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fdc1540f0 con 0x7f0fdc0b50a0 2024-05-11T08:06:45.350 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.349+0000 7f0fe1151640 1 -- 172.21.15.167:0/2530238444 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0fdc069590 con 0x7f0fdc0b50a0 2024-05-11T08:06:45.350 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.349+0000 7f0fc37fe640 1 -- 172.21.15.167:0/2530238444 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f0fcc044020 con 0x7f0fdc0b50a0 2024-05-11T08:06:45.351 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.350+0000 7f0fc37fe640 1 --2- 172.21.15.167:0/2530238444 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f0fc80762d0 0x7f0fc8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:45.351 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.350+0000 7f0fc37fe640 1 -- 172.21.15.167:0/2530238444 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5249+0+0 (secure 0 0 0) 0x7f0fcc0b7bb0 con 0x7f0fdc0b50a0 2024-05-11T08:06:45.351 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.350+0000 7f0fda59c640 1 --2- 172.21.15.167:0/2530238444 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f0fc80762d0 0x7f0fc8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:45.351 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.350+0000 7f0fda59c640 1 --2- 172.21.15.167:0/2530238444 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f0fc80762d0 0x7f0fc8078790 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0fd0006fd0 tx=0x7f0fd0008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:45.354 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.354+0000 7f0fc37fe640 1 -- 172.21.15.167:0/2530238444 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f0fcc04b030 con 0x7f0fdc0b50a0 2024-05-11T08:06:45.540 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.540+0000 7f0fe1151640 1 -- 172.21.15.167:0/2530238444 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f0fdc06df40 con 0x7f0fdc0b50a0 2024-05-11T08:06:45.540 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.540+0000 7f0fc37fe640 1 -- 172.21.15.167:0/2530238444 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v37) ==== 84+0+65 (secure 0 0 0) 0x7f0fcc081760 con 0x7f0fdc0b50a0 2024-05-11T08:06:45.541 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T08:06:45.541 INFO:teuthology.orchestra.run.smithi167.stdout:[ 2024-05-11T08:06:45.541 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T08:06:45.541 INFO:teuthology.orchestra.run.smithi167.stdout: "poolnum": 1, 2024-05-11T08:06:45.541 INFO:teuthology.orchestra.run.smithi167.stdout: "poolname": ".mgr" 2024-05-11T08:06:45.541 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T08:06:45.541 INFO:teuthology.orchestra.run.smithi167.stdout:] 2024-05-11T08:06:45.543 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.542+0000 7f0fe1151640 1 -- 172.21.15.167:0/2530238444 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f0fc80762d0 msgr2=0x7f0fc8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:45.543 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.542+0000 7f0fe1151640 1 --2- 172.21.15.167:0/2530238444 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f0fc80762d0 0x7f0fc8078790 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0fd0006fd0 tx=0x7f0fd0008040 comp rx=0 tx=0).stop 2024-05-11T08:06:45.543 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.542+0000 7f0fe1151640 1 -- 172.21.15.167:0/2530238444 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0fdc0b50a0 msgr2=0x7f0fdc13dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:45.543 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.542+0000 7f0fe1151640 1 --2- 172.21.15.167:0/2530238444 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0fdc0b50a0 0x7f0fdc13dc40 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f0fcc000c00 tx=0x7f0fcc0046c0 comp rx=0 tx=0).stop 2024-05-11T08:06:45.543 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.543+0000 7f0fe1151640 1 -- 172.21.15.167:0/2530238444 shutdown_connections 2024-05-11T08:06:45.543 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.543+0000 7f0fe1151640 1 --2- 172.21.15.167:0/2530238444 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f0fc80762d0 0x7f0fc8078790 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:45.543 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.543+0000 7f0fe1151640 1 --2- 172.21.15.167:0/2530238444 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0fdc13f290 0x7f0fdc06e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:45.543 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.543+0000 7f0fe1151640 1 --2- 172.21.15.167:0/2530238444 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0fdc13eeb0 0x7f0fdc128c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:45.543 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.543+0000 7f0fe1151640 1 --2- 172.21.15.167:0/2530238444 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0fdc0b50a0 0x7f0fdc13dc40 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:45.543 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.543+0000 7f0fe1151640 1 -- 172.21.15.167:0/2530238444 >> 172.21.15.167:0/2530238444 conn(0x7f0fdc0cd8f0 msgr2=0x7f0fdc0cd110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:45.543 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.543+0000 7f0fe1151640 1 -- 172.21.15.167:0/2530238444 shutdown_connections 2024-05-11T08:06:45.543 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.543+0000 7f0fe1151640 1 -- 172.21.15.167:0/2530238444 wait complete. 2024-05-11T08:06:45.554 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-11T08:06:45.554 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-11T08:06:45.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.658+0000 7f4b1e0a0640 1 Processor -- start 2024-05-11T08:06:45.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.658+0000 7f4b1e0a0640 1 -- start start 2024-05-11T08:06:45.659 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.658+0000 7f4b1e0a0640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f4b180d44b0 0x7f4b1806cb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:45.659 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.658+0000 7f4b1e0a0640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4b180b4460 0x7f4b1806d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:45.659 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.658+0000 7f4b1e0a0640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b18163420 con 0x7f4b181628c0 2024-05-11T08:06:45.659 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.658+0000 7f4b1e0a0640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f4b18164030 con 0x7f4b180d44b0 2024-05-11T08:06:45.659 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.658+0000 7f4b1e0a0640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f4b18163700 con 0x7f4b180b4460 2024-05-11T08:06:45.659 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.659+0000 7f4b0ffff640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f4b180d44b0 0x7f4b1806cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:45.659 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.659+0000 7f4b0ffff640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f4b180d44b0 0x7f4b1806cb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:43892/0 (socket says 172.21.15.167:43892) 2024-05-11T08:06:45.660 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.659+0000 7f4b0ffff640 1 -- 172.21.15.167:0/1081484204 learned_addr learned my addr 172.21.15.167:0/1081484204 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:45.660 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.659+0000 7f4b17fff640 1 --2- 172.21.15.167:0/1081484204 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4b180b4460 0x7f4b1806d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:45.660 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.659+0000 7f4b0ffff640 1 -- 172.21.15.167:0/1081484204 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4b180b4460 msgr2=0x7f4b1806d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:45.660 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.659+0000 7f4b0ffff640 1 --2- 172.21.15.167:0/1081484204 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4b180b4460 0x7f4b1806d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:45.660 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.659+0000 7f4b0ffff640 1 -- 172.21.15.167:0/1081484204 >> v1:172.21.15.204:6789/0 conn(0x7f4b181628c0 legacy=0x7f4b180d3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:45.660 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.659+0000 7f4b0ffff640 1 -- 172.21.15.167:0/1081484204 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b181639e0 con 0x7f4b180d44b0 2024-05-11T08:06:45.660 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.659+0000 7f4b17fff640 1 --2- 172.21.15.167:0/1081484204 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4b180b4460 0x7f4b1806d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T08:06:45.661 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.660+0000 7f4b0ffff640 1 --2- 172.21.15.167:0/1081484204 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4b180d44b0 0x7f4b1806cb60 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f4b00009870 tx=0x7f4b0002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9baaabb255346451 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:45.661 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.660+0000 7f4b16ffd640 1 -- 172.21.15.167:0/1081484204 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b00037070 con 0x7f4b180d44b0 2024-05-11T08:06:45.661 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.660+0000 7f4b16ffd640 1 -- 172.21.15.167:0/1081484204 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b00032440 con 0x7f4b180d44b0 2024-05-11T08:06:45.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.660+0000 7f4b16ffd640 1 -- 172.21.15.167:0/1081484204 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b00032810 con 0x7f4b180d44b0 2024-05-11T08:06:45.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.660+0000 7f4b1e0a0640 1 -- 172.21.15.167:0/1081484204 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4b180d44b0 msgr2=0x7f4b1806cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:45.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.660+0000 7f4b1e0a0640 1 --2- 172.21.15.167:0/1081484204 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4b180d44b0 0x7f4b1806cb60 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f4b00009870 tx=0x7f4b0002a1c0 comp rx=0 tx=0).stop 2024-05-11T08:06:45.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.660+0000 7f4b1e0a0640 1 -- 172.21.15.167:0/1081484204 shutdown_connections 2024-05-11T08:06:45.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.660+0000 7f4b1e0a0640 1 --2- 172.21.15.167:0/1081484204 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4b180b4460 0x7f4b1806d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:45.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.660+0000 7f4b1e0a0640 1 --2- 172.21.15.167:0/1081484204 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4b180d44b0 0x7f4b1806cb60 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:45.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.660+0000 7f4b1e0a0640 1 -- 172.21.15.167:0/1081484204 >> 172.21.15.167:0/1081484204 conn(0x7f4b180b6cb0 msgr2=0x7f4b180b7090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:45.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.660+0000 7f4b1e0a0640 1 -- 172.21.15.167:0/1081484204 shutdown_connections 2024-05-11T08:06:45.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.660+0000 7f4b1e0a0640 1 -- 172.21.15.167:0/1081484204 wait complete. 2024-05-11T08:06:45.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.662+0000 7f4b1e0a0640 1 Processor -- start 2024-05-11T08:06:45.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.662+0000 7f4b1e0a0640 1 -- start start 2024-05-11T08:06:45.664 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.662+0000 7f4b1e0a0640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4b180b4460 0x7f4b180cae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:45.664 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.662+0000 7f4b1e0a0640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4b180d44b0 0x7f4b180c8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:45.664 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.662+0000 7f4b1e0a0640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4b181628c0 0x7f4b180c9230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:45.664 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.662+0000 7f4b1e0a0640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4b18162ca0 con 0x7f4b180d44b0 2024-05-11T08:06:45.664 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.662+0000 7f4b1e0a0640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4b181630d0 con 0x7f4b181628c0 2024-05-11T08:06:45.664 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.662+0000 7f4b1e0a0640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f4b18162e20 con 0x7f4b180b4460 2024-05-11T08:06:45.664 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.662+0000 7f4b0ffff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4b180d44b0 0x7f4b180c8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:45.665 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.662+0000 7f4b0ffff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4b180d44b0 0x7f4b180c8cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:43902/0 (socket says 172.21.15.167:43902) 2024-05-11T08:06:45.665 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.662+0000 7f4b0ffff640 1 -- 172.21.15.167:0/264187282 learned_addr learned my addr 172.21.15.167:0/264187282 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:45.665 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.663+0000 7f4b17fff640 1 --2- 172.21.15.167:0/264187282 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4b181628c0 0x7f4b180c9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:45.665 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.663+0000 7f4b177fe640 1 --2- 172.21.15.167:0/264187282 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4b180b4460 0x7f4b180cae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:45.665 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.663+0000 7f4b0ffff640 1 -- 172.21.15.167:0/264187282 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4b180b4460 msgr2=0x7f4b180cae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:45.665 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.663+0000 7f4b0ffff640 1 --2- 172.21.15.167:0/264187282 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4b180b4460 0x7f4b180cae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:45.665 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.663+0000 7f4b0ffff640 1 -- 172.21.15.167:0/264187282 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4b181628c0 msgr2=0x7f4b180c9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:45.665 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.663+0000 7f4b0ffff640 1 --2- 172.21.15.167:0/264187282 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4b181628c0 0x7f4b180c9230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:45.665 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.663+0000 7f4b0ffff640 1 -- 172.21.15.167:0/264187282 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b180c9840 con 0x7f4b180d44b0 2024-05-11T08:06:45.665 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.663+0000 7f4b0ffff640 1 --2- 172.21.15.167:0/264187282 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4b180d44b0 0x7f4b180c8cf0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f4b00009870 tx=0x7f4b000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:45.665 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.663+0000 7f4b14ff9640 1 -- 172.21.15.167:0/264187282 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b00037040 con 0x7f4b180d44b0 2024-05-11T08:06:45.665 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.663+0000 7f4b14ff9640 1 -- 172.21.15.167:0/264187282 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b00042070 con 0x7f4b180d44b0 2024-05-11T08:06:45.665 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.663+0000 7f4b1e0a0640 1 -- 172.21.15.167:0/264187282 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b180c9b20 con 0x7f4b180d44b0 2024-05-11T08:06:45.665 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.663+0000 7f4b14ff9640 1 -- 172.21.15.167:0/264187282 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b00030bb0 con 0x7f4b180d44b0 2024-05-11T08:06:45.665 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.663+0000 7f4b1e0a0640 1 -- 172.21.15.167:0/264187282 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b180ca0e0 con 0x7f4b180d44b0 2024-05-11T08:06:45.665 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.664+0000 7f4b1e0a0640 1 -- 172.21.15.167:0/264187282 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b18069590 con 0x7f4b180d44b0 2024-05-11T08:06:45.666 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.665+0000 7f4b14ff9640 1 -- 172.21.15.167:0/264187282 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f4b0002ad10 con 0x7f4b180d44b0 2024-05-11T08:06:45.666 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.665+0000 7f4b14ff9640 1 --2- 172.21.15.167:0/264187282 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f4af80762d0 0x7f4af8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:45.666 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.665+0000 7f4b14ff9640 1 -- 172.21.15.167:0/264187282 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5249+0+0 (secure 0 0 0) 0x7f4b000b7c80 con 0x7f4b180d44b0 2024-05-11T08:06:45.666 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.665+0000 7f4b177fe640 1 --2- 172.21.15.167:0/264187282 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f4af80762d0 0x7f4af8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:45.667 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.666+0000 7f4b177fe640 1 --2- 172.21.15.167:0/264187282 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f4af80762d0 0x7f4af8078790 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f4b08004ea0 tx=0x7f4b08009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:45.669 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.670+0000 7f4b14ff9640 1 -- 172.21.15.167:0/264187282 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f4b00081830 con 0x7f4b180d44b0 2024-05-11T08:06:45.852 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:45.852+0000 7f4b1e0a0640 1 -- 172.21.15.167:0/264187282 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f4b180b74f0 con 0x7f4b180d44b0 2024-05-11T08:06:46.572 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:46.572+0000 7f4b14ff9640 1 -- 172.21.15.167:0/264187282 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v38) ==== 138+0+0 (secure 0 0 0) 0x7f4b00081130 con 0x7f4b180d44b0 2024-05-11T08:06:46.572 INFO:teuthology.orchestra.run.smithi167.stderr:pool 'cephfs_metadata' created 2024-05-11T08:06:46.575 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:46.574+0000 7f4b1e0a0640 1 -- 172.21.15.167:0/264187282 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f4af80762d0 msgr2=0x7f4af8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:46.575 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:46.574+0000 7f4b1e0a0640 1 --2- 172.21.15.167:0/264187282 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f4af80762d0 0x7f4af8078790 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f4b08004ea0 tx=0x7f4b08009040 comp rx=0 tx=0).stop 2024-05-11T08:06:46.575 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:46.574+0000 7f4b1e0a0640 1 -- 172.21.15.167:0/264187282 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4b180d44b0 msgr2=0x7f4b180c8cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:46.575 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:46.574+0000 7f4b1e0a0640 1 --2- 172.21.15.167:0/264187282 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4b180d44b0 0x7f4b180c8cf0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f4b00009870 tx=0x7f4b000046c0 comp rx=0 tx=0).stop 2024-05-11T08:06:46.575 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:46.575+0000 7f4b1e0a0640 1 -- 172.21.15.167:0/264187282 shutdown_connections 2024-05-11T08:06:46.575 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:46.575+0000 7f4b1e0a0640 1 --2- 172.21.15.167:0/264187282 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f4af80762d0 0x7f4af8078790 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:46.575 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:46.575+0000 7f4b1e0a0640 1 --2- 172.21.15.167:0/264187282 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4b181628c0 0x7f4b180c9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:46.575 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:46.575+0000 7f4b1e0a0640 1 --2- 172.21.15.167:0/264187282 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4b180d44b0 0x7f4b180c8cf0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:46.575 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:46.575+0000 7f4b1e0a0640 1 --2- 172.21.15.167:0/264187282 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4b180b4460 0x7f4b180cae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:46.575 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:46.575+0000 7f4b1e0a0640 1 -- 172.21.15.167:0/264187282 >> 172.21.15.167:0/264187282 conn(0x7f4b180b6cb0 msgr2=0x7f4b180ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:46.575 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:46.575+0000 7f4b1e0a0640 1 -- 172.21.15.167:0/264187282 shutdown_connections 2024-05-11T08:06:46.576 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:46.575+0000 7f4b1e0a0640 1 -- 172.21.15.167:0/264187282 wait complete. 2024-05-11T08:06:46.590 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-11T08:06:46.699 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:46.699+0000 7f421d140640 1 Processor -- start 2024-05-11T08:06:46.699 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:46.699+0000 7f421d140640 1 -- start start 2024-05-11T08:06:46.700 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:46.700+0000 7f421d140640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f421815cc80 0x7f42181638f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:46.700 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:46.700+0000 7f421d140640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f4218071800 0x7f42180b5470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:46.701 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:46.700+0000 7f421d140640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42180aa6d0 con 0x7f421815d060 2024-05-11T08:06:46.701 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:46.700+0000 7f421d140640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f42180b4f90 con 0x7f4218071800 2024-05-11T08:06:46.701 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:46.700+0000 7f421d140640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f421815fe40 con 0x7f421815cc80 2024-05-11T08:06:46.701 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:46.700+0000 7f421759e640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f4218071800 0x7f42180b5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:46.701 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:46.700+0000 7f4216d9d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f421815cc80 0x7f42181638f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:46.701 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:46.700+0000 7f420ffff640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f421815d060 0x7f42180710f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.167:42154/0 (socket says 172.21.15.167:42154) 2024-05-11T08:06:46.701 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:46.700+0000 7f420ffff640 1 -- 172.21.15.167:0/2263407641 learned_addr learned my addr 172.21.15.167:0/2263407641 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T08:06:46.701 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:46.700+0000 7f421759e640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f4218071800 0x7f42180b5470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.167:60716/0 (socket says 172.21.15.167:60716) 2024-05-11T08:06:46.701 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:46.700+0000 7f421759e640 1 -- 172.21.15.167:0/2263407641 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f421815cc80 msgr2=0x7f42181638f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:46.701 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:46.700+0000 7f421759e640 1 --2- 172.21.15.167:0/2263407641 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f421815cc80 0x7f42181638f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:46.701 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:46.700+0000 7f421759e640 1 -- 172.21.15.167:0/2263407641 >> v1:172.21.15.204:6789/0 conn(0x7f421815d060 legacy=0x7f42180710f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:46.701 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:46.701+0000 7f421759e640 1 -- 172.21.15.167:0/2263407641 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4218160120 con 0x7f4218071800 2024-05-11T08:06:46.702 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:46.701+0000 7f421759e640 1 --2- 172.21.15.167:0/2263407641 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4218071800 0x7f42180b5470 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f4210009d90 tx=0x7f421002a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f855a2e09b1dc742 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:46.703 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:46.701+0000 7f421659c640 1 -- 172.21.15.167:0/2263407641 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4210007740 con 0x7f4218071800 2024-05-11T08:06:46.703 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:46.701+0000 7f421659c640 1 -- 172.21.15.167:0/2263407641 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42100078e0 con 0x7f4218071800 2024-05-11T08:06:46.703 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:46.701+0000 7f421659c640 1 -- 172.21.15.167:0/2263407641 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4210007cb0 con 0x7f4218071800 2024-05-11T08:06:46.703 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:46.702+0000 7f421d140640 1 -- 172.21.15.167:0/2263407641 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4218071800 msgr2=0x7f42180b5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:46.703 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:46.702+0000 7f421d140640 1 --2- 172.21.15.167:0/2263407641 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4218071800 0x7f42180b5470 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f4210009d90 tx=0x7f421002a660 comp rx=0 tx=0).stop 2024-05-11T08:06:46.703 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:46.702+0000 7f421d140640 1 -- 172.21.15.167:0/2263407641 shutdown_connections 2024-05-11T08:06:46.703 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:46.702+0000 7f421d140640 1 --2- 172.21.15.167:0/2263407641 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4218071800 0x7f42180b5470 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:46.703 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:46.702+0000 7f421d140640 1 --2- 172.21.15.167:0/2263407641 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f421815cc80 0x7f42181638f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:46.703 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:46.702+0000 7f421d140640 1 -- 172.21.15.167:0/2263407641 >> 172.21.15.167:0/2263407641 conn(0x7f42180ce9e0 msgr2=0x7f42180cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:46.703 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:46.702+0000 7f421d140640 1 -- 172.21.15.167:0/2263407641 shutdown_connections 2024-05-11T08:06:46.703 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:46.702+0000 7f421d140640 1 -- 172.21.15.167:0/2263407641 wait complete. 2024-05-11T08:06:46.705 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:46.703+0000 7f421d140640 1 Processor -- start 2024-05-11T08:06:46.706 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:46.703+0000 7f421d140640 1 -- start start 2024-05-11T08:06:46.706 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:46.704+0000 7f421d140640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4218071800 0x7f42180b10b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:46.706 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:46.704+0000 7f421d140640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f421815cc80 0x7f421813a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:46.706 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:46.704+0000 7f421d140640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f421815d060 0x7f42180ac710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:46.706 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:46.704+0000 7f421d140640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f421815f8f0 con 0x7f421815cc80 2024-05-11T08:06:46.706 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:46.704+0000 7f421d140640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f42181606e0 con 0x7f4218071800 2024-05-11T08:06:46.706 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:46.704+0000 7f421d140640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f421815fa70 con 0x7f421815d060 2024-05-11T08:06:46.706 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:46.704+0000 7f420ffff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f421815cc80 0x7f421813a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:46.706 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:46.704+0000 7f420ffff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f421815cc80 0x7f421813a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:43910/0 (socket says 172.21.15.167:43910) 2024-05-11T08:06:46.706 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:46.704+0000 7f420ffff640 1 -- 172.21.15.167:0/3370881753 learned_addr learned my addr 172.21.15.167:0/3370881753 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:46.706 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:46.704+0000 7f4216d9d640 1 --2- 172.21.15.167:0/3370881753 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4218071800 0x7f42180b10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:46.706 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:46.704+0000 7f421759e640 1 --2- 172.21.15.167:0/3370881753 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f421815d060 0x7f42180ac710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:46.706 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:46.705+0000 7f420ffff640 1 -- 172.21.15.167:0/3370881753 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f421815d060 msgr2=0x7f42180ac710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:46.707 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:46.705+0000 7f420ffff640 1 --2- 172.21.15.167:0/3370881753 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f421815d060 0x7f42180ac710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:46.707 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:46.705+0000 7f420ffff640 1 -- 172.21.15.167:0/3370881753 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4218071800 msgr2=0x7f42180b10b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:46.707 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:46.705+0000 7f420ffff640 1 --2- 172.21.15.167:0/3370881753 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4218071800 0x7f42180b10b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:46.707 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:46.705+0000 7f420ffff640 1 -- 172.21.15.167:0/3370881753 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42180acf40 con 0x7f421815cc80 2024-05-11T08:06:46.707 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:46.705+0000 7f420ffff640 1 --2- 172.21.15.167:0/3370881753 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f421815cc80 0x7f421813a750 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f4200001eb0 tx=0x7f420000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:46.707 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:46.705+0000 7f420f7fe640 1 -- 172.21.15.167:0/3370881753 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4200017070 con 0x7f421815cc80 2024-05-11T08:06:46.707 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:46.705+0000 7f420f7fe640 1 -- 172.21.15.167:0/3370881753 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4200005d00 con 0x7f421815cc80 2024-05-11T08:06:46.707 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:46.705+0000 7f421d140640 1 -- 172.21.15.167:0/3370881753 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42180ad220 con 0x7f421815cc80 2024-05-11T08:06:46.707 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:46.705+0000 7f420f7fe640 1 -- 172.21.15.167:0/3370881753 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4200007310 con 0x7f421815cc80 2024-05-11T08:06:46.707 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:46.705+0000 7f421d140640 1 -- 172.21.15.167:0/3370881753 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42180ad7e0 con 0x7f421815cc80 2024-05-11T08:06:46.708 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:46.706+0000 7f421d140640 1 -- 172.21.15.167:0/3370881753 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4218069630 con 0x7f421815cc80 2024-05-11T08:06:46.708 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:46.707+0000 7f420f7fe640 1 -- 172.21.15.167:0/3370881753 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f420001a070 con 0x7f421815cc80 2024-05-11T08:06:46.708 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:46.707+0000 7f420f7fe640 1 --2- 172.21.15.167:0/3370881753 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f41f8076300 0x7f41f80787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:46.708 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:46.707+0000 7f420f7fe640 1 -- 172.21.15.167:0/3370881753 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5619+0+0 (secure 0 0 0) 0x7f42000968d0 con 0x7f421815cc80 2024-05-11T08:06:46.708 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:46.707+0000 7f4216d9d640 1 --2- 172.21.15.167:0/3370881753 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f41f8076300 0x7f41f80787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:46.708 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:46.708+0000 7f4216d9d640 1 --2- 172.21.15.167:0/3370881753 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f41f8076300 0x7f41f80787c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4208009b40 tx=0x7f4208009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:46.712 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:46.712+0000 7f420f7fe640 1 -- 172.21.15.167:0/3370881753 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f420005fb90 con 0x7f421815cc80 2024-05-11T08:06:46.894 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:46.894+0000 7f421d140640 1 -- 172.21.15.167:0/3370881753 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f4218148150 con 0x7f421815cc80 2024-05-11T08:06:47.575 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:47.575+0000 7f420f7fe640 1 -- 172.21.15.167:0/3370881753 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v39) ==== 170+0+0 (secure 0 0 0) 0x7f4200063e90 con 0x7f421815cc80 2024-05-11T08:06:47.575 INFO:teuthology.orchestra.run.smithi167.stderr:pool 'cephfs_data' created 2024-05-11T08:06:47.578 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:47.578+0000 7f421d140640 1 -- 172.21.15.167:0/3370881753 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f41f8076300 msgr2=0x7f41f80787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:47.578 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:47.578+0000 7f421d140640 1 --2- 172.21.15.167:0/3370881753 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f41f8076300 0x7f41f80787c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4208009b40 tx=0x7f4208009190 comp rx=0 tx=0).stop 2024-05-11T08:06:47.578 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:47.578+0000 7f421d140640 1 -- 172.21.15.167:0/3370881753 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f421815cc80 msgr2=0x7f421813a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:47.578 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:47.578+0000 7f421d140640 1 --2- 172.21.15.167:0/3370881753 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f421815cc80 0x7f421813a750 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f4200001eb0 tx=0x7f420000c890 comp rx=0 tx=0).stop 2024-05-11T08:06:47.579 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:47.578+0000 7f421d140640 1 -- 172.21.15.167:0/3370881753 shutdown_connections 2024-05-11T08:06:47.579 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:47.578+0000 7f421d140640 1 --2- 172.21.15.167:0/3370881753 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f41f8076300 0x7f41f80787c0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:47.579 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:47.578+0000 7f421d140640 1 --2- 172.21.15.167:0/3370881753 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f421815d060 0x7f42180ac710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:47.579 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:47.578+0000 7f421d140640 1 --2- 172.21.15.167:0/3370881753 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f421815cc80 0x7f421813a750 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:47.579 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:47.578+0000 7f421d140640 1 --2- 172.21.15.167:0/3370881753 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4218071800 0x7f42180b10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:47.579 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:47.578+0000 7f421d140640 1 -- 172.21.15.167:0/3370881753 >> 172.21.15.167:0/3370881753 conn(0x7f42180ce9e0 msgr2=0x7f42180b5590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:47.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:47.581+0000 7f421d140640 1 -- 172.21.15.167:0/3370881753 shutdown_connections 2024-05-11T08:06:47.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:47.582+0000 7f421d140640 1 -- 172.21.15.167:0/3370881753 wait complete. 2024-05-11T08:06:47.607 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-11T08:06:47.714 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:47.714+0000 7f641d3f0640 1 Processor -- start 2024-05-11T08:06:47.715 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:47.714+0000 7f641d3f0640 1 -- start start 2024-05-11T08:06:47.716 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:47.715+0000 7f641d3f0640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f641806d200 0x7f641806d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:47.716 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:47.715+0000 7f641d3f0640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f64180d2db0 0x7f64180d3240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:47.716 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:47.715+0000 7f641d3f0640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6418159d50 con 0x7f6418159970 2024-05-11T08:06:47.716 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:47.715+0000 7f641d3f0640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f64180d04b0 con 0x7f64180d2db0 2024-05-11T08:06:47.716 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:47.715+0000 7f641d3f0640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f64180d0790 con 0x7f641806d200 2024-05-11T08:06:47.716 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:47.715+0000 7f6416ffd640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f6418159970 0x7f64180d1270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:55960/0 (socket says 172.21.15.167:55960) 2024-05-11T08:06:47.716 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:47.715+0000 7f6416ffd640 1 -- 172.21.15.167:0/3820544146 learned_addr learned my addr 172.21.15.167:0/3820544146 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T08:06:47.716 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:47.715+0000 7f64177fe640 1 --2- 172.21.15.167:0/3820544146 >> v2:172.21.15.204:3300/0 conn(0x7f64180d2db0 0x7f64180d3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:47.716 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:47.715+0000 7f64167fc640 1 --2- 172.21.15.167:0/3820544146 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f641806d200 0x7f641806d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:47.716 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:47.716+0000 7f6415ffb640 1 -- 172.21.15.167:0/3820544146 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1374464035 0 0) 0x7f6418159d50 con 0x7f6418159970 2024-05-11T08:06:47.717 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:47.716+0000 7f6415ffb640 1 -- 172.21.15.167:0/3820544146 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6404003520 con 0x7f6418159970 2024-05-11T08:06:47.717 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:47.716+0000 7f64177fe640 1 -- 172.21.15.167:0/3820544146 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f641806d200 msgr2=0x7f641806d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:47.717 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:47.716+0000 7f64177fe640 1 --2- 172.21.15.167:0/3820544146 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f641806d200 0x7f641806d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:47.717 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:47.716+0000 7f64177fe640 1 -- 172.21.15.167:0/3820544146 >> v1:172.21.15.167:6789/0 conn(0x7f6418159970 legacy=0x7f64180d1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:47.717 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:47.716+0000 7f64177fe640 1 -- 172.21.15.167:0/3820544146 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64180d0a70 con 0x7f64180d2db0 2024-05-11T08:06:47.718 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:47.716+0000 7f64177fe640 1 --2- 172.21.15.167:0/3820544146 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f64180d2db0 0x7f64180d3240 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f64080097d0 tx=0x7f640802a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd1c9d684ed4c8e1 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:47.718 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:47.717+0000 7f6415ffb640 1 -- 172.21.15.167:0/3820544146 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6408037070 con 0x7f64180d2db0 2024-05-11T08:06:47.718 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:47.717+0000 7f6415ffb640 1 -- 172.21.15.167:0/3820544146 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6408032440 con 0x7f64180d2db0 2024-05-11T08:06:47.718 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:47.717+0000 7f6415ffb640 1 -- 172.21.15.167:0/3820544146 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64080327f0 con 0x7f64180d2db0 2024-05-11T08:06:47.718 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:47.717+0000 7f641d3f0640 1 -- 172.21.15.167:0/3820544146 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f64180d2db0 msgr2=0x7f64180d3240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:47.718 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:47.717+0000 7f641d3f0640 1 --2- 172.21.15.167:0/3820544146 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f64180d2db0 0x7f64180d3240 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f64080097d0 tx=0x7f640802a1c0 comp rx=0 tx=0).stop 2024-05-11T08:06:47.718 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:47.717+0000 7f641d3f0640 1 -- 172.21.15.167:0/3820544146 shutdown_connections 2024-05-11T08:06:47.718 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:47.717+0000 7f641d3f0640 1 --2- 172.21.15.167:0/3820544146 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f64180d2db0 0x7f64180d3240 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:47.718 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:47.717+0000 7f641d3f0640 1 --2- 172.21.15.167:0/3820544146 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f641806d200 0x7f641806d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:47.719 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:47.717+0000 7f641d3f0640 1 -- 172.21.15.167:0/3820544146 >> 172.21.15.167:0/3820544146 conn(0x7f64180d51e0 msgr2=0x7f64180d55f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:47.719 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:47.717+0000 7f641d3f0640 1 -- 172.21.15.167:0/3820544146 shutdown_connections 2024-05-11T08:06:47.719 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:47.717+0000 7f641d3f0640 1 -- 172.21.15.167:0/3820544146 wait complete. 2024-05-11T08:06:47.720 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:47.718+0000 7f641d3f0640 1 Processor -- start 2024-05-11T08:06:47.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:47.719+0000 7f641d3f0640 1 -- start start 2024-05-11T08:06:47.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:47.719+0000 7f641d3f0640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f641806d200 0x7f64180cc2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:47.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:47.719+0000 7f641d3f0640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f64180d2db0 0x7f641815ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:47.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:47.719+0000 7f641d3f0640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6418159970 0x7f6418155260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:47.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:47.719+0000 7f641d3f0640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f64180d1980 con 0x7f6418159970 2024-05-11T08:06:47.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:47.719+0000 7f641d3f0640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6418163d00 con 0x7f641806d200 2024-05-11T08:06:47.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:47.719+0000 7f641d3f0640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f6418162f10 con 0x7f64180d2db0 2024-05-11T08:06:47.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:47.719+0000 7f64177fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6418159970 0x7f6418155260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:47.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:47.719+0000 7f64177fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6418159970 0x7f6418155260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:43926/0 (socket says 172.21.15.167:43926) 2024-05-11T08:06:47.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:47.719+0000 7f64177fe640 1 -- 172.21.15.167:0/801875566 learned_addr learned my addr 172.21.15.167:0/801875566 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:47.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:47.720+0000 7f6416ffd640 1 --2- 172.21.15.167:0/801875566 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f641806d200 0x7f64180cc2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:47.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:47.720+0000 7f64167fc640 1 --2- 172.21.15.167:0/801875566 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f64180d2db0 0x7f641815ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:47.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:47.720+0000 7f64177fe640 1 -- 172.21.15.167:0/801875566 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f64180d2db0 msgr2=0x7f641815ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:47.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:47.720+0000 7f64177fe640 1 --2- 172.21.15.167:0/801875566 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f64180d2db0 0x7f641815ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:47.722 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:47.720+0000 7f64177fe640 1 -- 172.21.15.167:0/801875566 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f641806d200 msgr2=0x7f64180cc2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:47.722 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:47.720+0000 7f64177fe640 1 --2- 172.21.15.167:0/801875566 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f641806d200 0x7f64180cc2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:47.722 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:47.720+0000 7f64177fe640 1 -- 172.21.15.167:0/801875566 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64180cc940 con 0x7f6418159970 2024-05-11T08:06:47.722 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:47.720+0000 7f64177fe640 1 --2- 172.21.15.167:0/801875566 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6418159970 0x7f6418155260 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f64080098f0 tx=0x7f64080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:47.722 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:47.720+0000 7f640f7fe640 1 -- 172.21.15.167:0/801875566 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6408037050 con 0x7f6418159970 2024-05-11T08:06:47.722 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:47.720+0000 7f640f7fe640 1 -- 172.21.15.167:0/801875566 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6408038040 con 0x7f6418159970 2024-05-11T08:06:47.722 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:47.720+0000 7f641d3f0640 1 -- 172.21.15.167:0/801875566 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64180ccc20 con 0x7f6418159970 2024-05-11T08:06:47.722 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:47.720+0000 7f640f7fe640 1 -- 172.21.15.167:0/801875566 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6408004470 con 0x7f6418159970 2024-05-11T08:06:47.722 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:47.720+0000 7f641d3f0640 1 -- 172.21.15.167:0/801875566 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64180cd1e0 con 0x7f6418159970 2024-05-11T08:06:47.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:47.721+0000 7f641d3f0640 1 -- 172.21.15.167:0/801875566 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6418069520 con 0x7f6418159970 2024-05-11T08:06:47.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:47.722+0000 7f640f7fe640 1 -- 172.21.15.167:0/801875566 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f6408003610 con 0x7f6418159970 2024-05-11T08:06:47.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:47.722+0000 7f640f7fe640 1 --2- 172.21.15.167:0/801875566 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f63f80762d0 0x7f63f8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:47.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:47.722+0000 7f640f7fe640 1 -- 172.21.15.167:0/801875566 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6001+0+0 (secure 0 0 0) 0x7f64080b7f20 con 0x7f6418159970 2024-05-11T08:06:47.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:47.722+0000 7f6416ffd640 1 --2- 172.21.15.167:0/801875566 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f63f80762d0 0x7f63f8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:47.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:47.723+0000 7f6416ffd640 1 --2- 172.21.15.167:0/801875566 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f63f80762d0 0x7f63f8078790 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f6410005c00 tx=0x7f6410009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:47.727 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:47.727+0000 7f640f7fe640 1 -- 172.21.15.167:0/801875566 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f6408081110 con 0x7f6418159970 2024-05-11T08:06:47.942 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:47.942+0000 7f641d3f0640 1 -- 172.21.15.167:0/801875566 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f64180d00f0 con 0x7f6418159970 2024-05-11T08:06:48.602 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:48.601+0000 7f640f7fe640 1 -- 172.21.15.167:0/801875566 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 5 and data pool 6 v14) ==== 172+0+0 (secure 0 0 0) 0x7f6408085410 con 0x7f6418159970 2024-05-11T08:06:48.602 INFO:teuthology.orchestra.run.smithi167.stderr:new fs with metadata pool 5 and data pool 6 2024-05-11T08:06:48.607 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:48.606+0000 7f641d3f0640 1 -- 172.21.15.167:0/801875566 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f63f80762d0 msgr2=0x7f63f8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:48.607 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:48.606+0000 7f641d3f0640 1 --2- 172.21.15.167:0/801875566 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f63f80762d0 0x7f63f8078790 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f6410005c00 tx=0x7f6410009040 comp rx=0 tx=0).stop 2024-05-11T08:06:48.608 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:48.606+0000 7f641d3f0640 1 -- 172.21.15.167:0/801875566 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6418159970 msgr2=0x7f6418155260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:48.608 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:48.606+0000 7f641d3f0640 1 --2- 172.21.15.167:0/801875566 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6418159970 0x7f6418155260 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f64080098f0 tx=0x7f64080046c0 comp rx=0 tx=0).stop 2024-05-11T08:06:48.608 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:48.607+0000 7f641d3f0640 1 -- 172.21.15.167:0/801875566 shutdown_connections 2024-05-11T08:06:48.608 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:48.607+0000 7f641d3f0640 1 --2- 172.21.15.167:0/801875566 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f63f80762d0 0x7f63f8078790 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:48.608 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:48.607+0000 7f641d3f0640 1 --2- 172.21.15.167:0/801875566 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6418159970 0x7f6418155260 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:48.608 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:48.607+0000 7f641d3f0640 1 --2- 172.21.15.167:0/801875566 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f64180d2db0 0x7f641815ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:48.608 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:48.607+0000 7f641d3f0640 1 --2- 172.21.15.167:0/801875566 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f641806d200 0x7f64180cc2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:48.608 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:48.607+0000 7f641d3f0640 1 -- 172.21.15.167:0/801875566 >> 172.21.15.167:0/801875566 conn(0x7f64180d51e0 msgr2=0x7f64180cb9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:48.608 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:48.608+0000 7f641d3f0640 1 -- 172.21.15.167:0/801875566 shutdown_connections 2024-05-11T08:06:48.608 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:48.608+0000 7f641d3f0640 1 -- 172.21.15.167:0/801875566 wait complete. 2024-05-11T08:06:48.629 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-11T08:06:48.730 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:48.730+0000 7fca4d023640 1 Processor -- start 2024-05-11T08:06:48.730 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:48.730+0000 7fca4d023640 1 -- start start 2024-05-11T08:06:48.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:48.731+0000 7fca4d023640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fca48146b40 0x7fca4806e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:48.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:48.731+0000 7fca4d023640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fca4806eb50 0x7fca4806cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:48.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:48.731+0000 7fca4d023640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fca4815e520 con 0x7fca48146760 2024-05-11T08:06:48.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:48.731+0000 7fca4d023640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7fca480707a0 con 0x7fca4806eb50 2024-05-11T08:06:48.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:48.731+0000 7fca4d023640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fca4806efa0 con 0x7fca48146b40 2024-05-11T08:06:48.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:48.731+0000 7fca4759e640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fca4806eb50 0x7fca4806cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:48.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:48.731+0000 7fca46d9d640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7fca48146760 0x7fca480d13a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:55976/0 (socket says 172.21.15.167:55976) 2024-05-11T08:06:48.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:48.731+0000 7fca46d9d640 1 -- 172.21.15.167:0/2514263926 learned_addr learned my addr 172.21.15.167:0/2514263926 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T08:06:48.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:48.732+0000 7fca4759e640 1 -- 172.21.15.167:0/2514263926 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fca48146b40 msgr2=0x7fca4806e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:48.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:48.732+0000 7fca4759e640 1 --2- 172.21.15.167:0/2514263926 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fca48146b40 0x7fca4806e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:48.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:48.732+0000 7fca4759e640 1 -- 172.21.15.167:0/2514263926 >> v1:172.21.15.167:6789/0 conn(0x7fca48146760 legacy=0x7fca480d13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:48.733 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:48.732+0000 7fca4759e640 1 -- 172.21.15.167:0/2514263926 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca480709d0 con 0x7fca4806eb50 2024-05-11T08:06:48.733 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:48.732+0000 7fca45d9b640 1 -- 172.21.15.167:0/2514263926 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3664681968 0 0) 0x7fca4815e520 con 0x7fca48146760 2024-05-11T08:06:48.734 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:48.732+0000 7fca4759e640 1 --2- 172.21.15.167:0/2514263926 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca4806eb50 0x7fca4806cce0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fca380097a0 tx=0x7fca3802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=861b8cd3dd0944bd server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:48.734 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:48.732+0000 7fca45d9b640 1 -- 172.21.15.167:0/2514263926 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca38037070 con 0x7fca4806eb50 2024-05-11T08:06:48.734 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:48.732+0000 7fca45d9b640 1 -- 172.21.15.167:0/2514263926 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca38032440 con 0x7fca4806eb50 2024-05-11T08:06:48.734 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:48.732+0000 7fca45d9b640 1 -- 172.21.15.167:0/2514263926 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca38032810 con 0x7fca4806eb50 2024-05-11T08:06:48.734 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:48.733+0000 7fca4d023640 1 -- 172.21.15.167:0/2514263926 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca4806eb50 msgr2=0x7fca4806cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:48.734 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:48.733+0000 7fca4d023640 1 --2- 172.21.15.167:0/2514263926 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca4806eb50 0x7fca4806cce0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fca380097a0 tx=0x7fca3802a180 comp rx=0 tx=0).stop 2024-05-11T08:06:48.734 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:48.733+0000 7fca4d023640 1 -- 172.21.15.167:0/2514263926 shutdown_connections 2024-05-11T08:06:48.734 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:48.733+0000 7fca4d023640 1 --2- 172.21.15.167:0/2514263926 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca4806eb50 0x7fca4806cce0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:48.734 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:48.733+0000 7fca4d023640 1 --2- 172.21.15.167:0/2514263926 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fca48146b40 0x7fca4806e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:48.734 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:48.733+0000 7fca4d023640 1 -- 172.21.15.167:0/2514263926 >> 172.21.15.167:0/2514263926 conn(0x7fca480d1e40 msgr2=0x7fca480d2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:48.734 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:48.733+0000 7fca4d023640 1 -- 172.21.15.167:0/2514263926 shutdown_connections 2024-05-11T08:06:48.734 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:48.733+0000 7fca4d023640 1 -- 172.21.15.167:0/2514263926 wait complete. 2024-05-11T08:06:48.736 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:48.734+0000 7fca4d023640 1 Processor -- start 2024-05-11T08:06:48.736 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:48.735+0000 7fca4d023640 1 -- start start 2024-05-11T08:06:48.737 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:48.735+0000 7fca4d023640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fca4806eb50 0x7fca48156a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:48.737 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:48.735+0000 7fca4d023640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fca48146760 0x7fca480b61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:48.737 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:48.735+0000 7fca4d023640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca48146b40 0x7fca480cfbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:48.737 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:48.735+0000 7fca4d023640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fca48071550 con 0x7fca48146b40 2024-05-11T08:06:48.737 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:48.735+0000 7fca4d023640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fca48071270 con 0x7fca48146760 2024-05-11T08:06:48.737 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:48.735+0000 7fca4d023640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fca480d1ab0 con 0x7fca4806eb50 2024-05-11T08:06:48.737 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:48.735+0000 7fca4759e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca48146b40 0x7fca480cfbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:48.737 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:48.735+0000 7fca4759e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca48146b40 0x7fca480cfbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:43946/0 (socket says 172.21.15.167:43946) 2024-05-11T08:06:48.737 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:48.735+0000 7fca4759e640 1 -- 172.21.15.167:0/665944715 learned_addr learned my addr 172.21.15.167:0/665944715 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:48.737 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:48.736+0000 7fca4759e640 1 -- 172.21.15.167:0/665944715 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fca4806eb50 msgr2=0x7fca48156a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:48.737 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:48.736+0000 7fca4759e640 1 --2- 172.21.15.167:0/665944715 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fca4806eb50 0x7fca48156a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:48.737 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:48.736+0000 7fca4759e640 1 -- 172.21.15.167:0/665944715 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fca48146760 msgr2=0x7fca480b61f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:48.737 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:48.736+0000 7fca4759e640 1 --2- 172.21.15.167:0/665944715 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fca48146760 0x7fca480b61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:48.737 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:48.736+0000 7fca4759e640 1 -- 172.21.15.167:0/665944715 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca480b6f90 con 0x7fca48146b40 2024-05-11T08:06:48.737 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:48.736+0000 7fca4759e640 1 --2- 172.21.15.167:0/665944715 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca48146b40 0x7fca480cfbb0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fca380098c0 tx=0x7fca380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:48.738 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:48.736+0000 7fca3f7fe640 1 -- 172.21.15.167:0/665944715 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca38037040 con 0x7fca48146b40 2024-05-11T08:06:48.738 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:48.736+0000 7fca3f7fe640 1 -- 172.21.15.167:0/665944715 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca38042070 con 0x7fca48146b40 2024-05-11T08:06:48.738 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:48.736+0000 7fca3f7fe640 1 -- 172.21.15.167:0/665944715 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca38030b30 con 0x7fca48146b40 2024-05-11T08:06:48.738 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:48.736+0000 7fca4d023640 1 -- 172.21.15.167:0/665944715 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca480b6cb0 con 0x7fca48146b40 2024-05-11T08:06:48.738 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:48.736+0000 7fca4d023640 1 -- 172.21.15.167:0/665944715 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca480b5330 con 0x7fca48146b40 2024-05-11T08:06:48.738 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:48.737+0000 7fca4d023640 1 -- 172.21.15.167:0/665944715 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca48069bc0 con 0x7fca48146b40 2024-05-11T08:06:48.739 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:48.738+0000 7fca3f7fe640 1 -- 172.21.15.167:0/665944715 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fca38030cd0 con 0x7fca48146b40 2024-05-11T08:06:48.739 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:48.738+0000 7fca3f7fe640 1 --2- 172.21.15.167:0/665944715 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fca280762d0 0x7fca28078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:48.739 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:48.738+0000 7fca46d9d640 1 --2- 172.21.15.167:0/665944715 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fca280762d0 0x7fca28078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:48.739 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:48.738+0000 7fca3f7fe640 1 -- 172.21.15.167:0/665944715 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7fca380b83a0 con 0x7fca48146b40 2024-05-11T08:06:48.739 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:48.739+0000 7fca46d9d640 1 --2- 172.21.15.167:0/665944715 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fca280762d0 0x7fca28078790 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fca40001cd0 tx=0x7fca40007c50 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:48.743 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:48.743+0000 7fca3f7fe640 1 -- 172.21.15.167:0/665944715 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fca38081b70 con 0x7fca48146b40 2024-05-11T08:06:48.925 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:48.925+0000 7fca4d023640 1 -- 172.21.15.167:0/665944715 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fca480d0f50 con 0x7fca48146b40 2024-05-11T08:06:48.927 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:48.926+0000 7fca3f7fe640 1 -- 172.21.15.167:0/665944715 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) ==== 74+0+17003 (secure 0 0 0) 0x7fca38081470 con 0x7fca48146b40 2024-05-11T08:06:48.927 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T08:06:48.927 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":40,"fsid":"2034778d-8ff1-4823-aa44-ded32b518241","created":"2024-05-11T08:05:31.612421+0000","modified":"2024-05-11T08:06:47.943973+0000","last_up_change":"2024-05-11T08:05:42.557382+0000","last_in_change":"2024-05-11T08:05:33.757311+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-11T08:05:43.931263+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-05-11T08:06:45.857156+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-05-11T08:06:46.898591+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"587eaced-5ded-406d-884b-30c1caa8686b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":3863307356},{"type":"v1","addr":"172.21.15.167:6806","nonce":3863307356}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":3863307356},{"type":"v1","addr":"172.21.15.167:6810","nonce":3863307356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":3863307356},{"type":"v1","addr":"172.21.15.167:6815","nonce":3863307356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":3863307356},{"type":"v1","addr":"172.21.15.167:6813","nonce":3863307356}]},"public_addr":"172.21.15.167:6806/3863307356","cluster_addr":"172.21.15.167:6810/3863307356","heartbeat_back_addr":"172.21.15.167:6815/3863307356","heartbeat_front_addr":"172.21.15.167:6813/3863307356","state":["exists","up"]},{"osd":1,"uuid":"37c59104-2dd0-4cf9-85ea-56636f6b02a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":1585024048},{"type":"v1","addr":"172.21.15.167:6817","nonce":1585024048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1585024048},{"type":"v1","addr":"172.21.15.167:6819","nonce":1585024048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1585024048},{"type":"v1","addr":"172.21.15.167:6823","nonce":1585024048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":1585024048},{"type":"v1","addr":"172.21.15.167:6821","nonce":1585024048}]},"public_addr":"172.21.15.167:6817/1585024048","cluster_addr":"172.21.15.167:6819/1585024048","heartbeat_back_addr":"172.21.15.167:6823/1585024048","heartbeat_front_addr":"172.21.15.167:6821/1585024048","state":["exists","up"]},{"osd":2,"uuid":"2bb4f7f5-c698-451b-b4bb-78e716921291","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":3242166900},{"type":"v1","addr":"172.21.15.167:6801","nonce":3242166900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":3242166900},{"type":"v1","addr":"172.21.15.167:6803","nonce":3242166900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6809","nonce":3242166900},{"type":"v1","addr":"172.21.15.167:6811","nonce":3242166900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6805","nonce":3242166900},{"type":"v1","addr":"172.21.15.167:6807","nonce":3242166900}]},"public_addr":"172.21.15.167:6801/3242166900","cluster_addr":"172.21.15.167:6803/3242166900","heartbeat_back_addr":"172.21.15.167:6811/3242166900","heartbeat_front_addr":"172.21.15.167:6807/3242166900","state":["exists","up"]},{"osd":3,"uuid":"9dd26759-a338-49d2-8103-45d068b9c000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":453689293},{"type":"v1","addr":"172.21.15.167:6825","nonce":453689293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":453689293},{"type":"v1","addr":"172.21.15.167:6827","nonce":453689293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":453689293},{"type":"v1","addr":"172.21.15.167:6831","nonce":453689293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":453689293},{"type":"v1","addr":"172.21.15.167:6829","nonce":453689293}]},"public_addr":"172.21.15.167:6825/453689293","cluster_addr":"172.21.15.167:6827/453689293","heartbeat_back_addr":"172.21.15.167:6831/453689293","heartbeat_front_addr":"172.21.15.167:6829/453689293","state":["exists","up"]},{"osd":4,"uuid":"cb27250d-2a80-4c7b-8cef-3a7af05c8a16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1311831500},{"type":"v1","addr":"172.21.15.204:6801","nonce":1311831500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1311831500},{"type":"v1","addr":"172.21.15.204:6803","nonce":1311831500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1311831500},{"type":"v1","addr":"172.21.15.204:6807","nonce":1311831500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1311831500},{"type":"v1","addr":"172.21.15.204:6805","nonce":1311831500}]},"public_addr":"172.21.15.204:6801/1311831500","cluster_addr":"172.21.15.204:6803/1311831500","heartbeat_back_addr":"172.21.15.204:6807/1311831500","heartbeat_front_addr":"172.21.15.204:6805/1311831500","state":["exists","up"]},{"osd":5,"uuid":"92998541-ffec-430b-ab76-8c3cb7c2752d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2921847521},{"type":"v1","addr":"172.21.15.204:6825","nonce":2921847521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2921847521},{"type":"v1","addr":"172.21.15.204:6827","nonce":2921847521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2921847521},{"type":"v1","addr":"172.21.15.204:6831","nonce":2921847521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2921847521},{"type":"v1","addr":"172.21.15.204:6829","nonce":2921847521}]},"public_addr":"172.21.15.204:6825/2921847521","cluster_addr":"172.21.15.204:6827/2921847521","heartbeat_back_addr":"172.21.15.204:6831/2921847521","heartbeat_front_addr":"172.21.15.204:6829/2921847521","state":["exists","up"]},{"osd":6,"uuid":"9543479c-2ec5-42a8-8933-707bfed2e048","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":2889851077},{"type":"v1","addr":"172.21.15.204:6817","nonce":2889851077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":2889851077},{"type":"v1","addr":"172.21.15.204:6819","nonce":2889851077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":2889851077},{"type":"v1","addr":"172.21.15.204:6823","nonce":2889851077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":2889851077},{"type":"v1","addr":"172.21.15.204:6821","nonce":2889851077}]},"public_addr":"172.21.15.204:6817/2889851077","cluster_addr":"172.21.15.204:6819/2889851077","heartbeat_back_addr":"172.21.15.204:6823/2889851077","heartbeat_front_addr":"172.21.15.204:6821/2889851077","state":["exists","up"]},{"osd":7,"uuid":"c45a5647-8358-409e-af91-c05dda65f0b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":1949534184},{"type":"v1","addr":"172.21.15.204:6809","nonce":1949534184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":1949534184},{"type":"v1","addr":"172.21.15.204:6811","nonce":1949534184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1949534184},{"type":"v1","addr":"172.21.15.204:6815","nonce":1949534184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1949534184},{"type":"v1","addr":"172.21.15.204:6813","nonce":1949534184}]},"public_addr":"172.21.15.204:6809/1949534184","cluster_addr":"172.21.15.204:6811/1949534184","heartbeat_back_addr":"172.21.15.204:6815/1949534184","heartbeat_front_addr":"172.21.15.204:6813/1949534184","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.952253+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.940689+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.860682+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.929350+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.901627+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.957451+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.987249+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.951197+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-11T08:06:48.929 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:48.929+0000 7fca4d023640 1 -- 172.21.15.167:0/665944715 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fca280762d0 msgr2=0x7fca28078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:48.929 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:48.929+0000 7fca4d023640 1 --2- 172.21.15.167:0/665944715 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fca280762d0 0x7fca28078790 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fca40001cd0 tx=0x7fca40007c50 comp rx=0 tx=0).stop 2024-05-11T08:06:48.929 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:48.929+0000 7fca4d023640 1 -- 172.21.15.167:0/665944715 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca48146b40 msgr2=0x7fca480cfbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:48.929 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:48.929+0000 7fca4d023640 1 --2- 172.21.15.167:0/665944715 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca48146b40 0x7fca480cfbb0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fca380098c0 tx=0x7fca380046c0 comp rx=0 tx=0).stop 2024-05-11T08:06:48.929 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:48.929+0000 7fca4d023640 1 -- 172.21.15.167:0/665944715 shutdown_connections 2024-05-11T08:06:48.929 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:48.929+0000 7fca4d023640 1 --2- 172.21.15.167:0/665944715 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fca280762d0 0x7fca28078790 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:48.929 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:48.929+0000 7fca4d023640 1 --2- 172.21.15.167:0/665944715 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca48146b40 0x7fca480cfbb0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:48.929 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:48.929+0000 7fca4d023640 1 --2- 172.21.15.167:0/665944715 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fca48146760 0x7fca480b61f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:48.929 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:48.929+0000 7fca4d023640 1 --2- 172.21.15.167:0/665944715 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fca4806eb50 0x7fca48156a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:48.930 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:48.929+0000 7fca4d023640 1 -- 172.21.15.167:0/665944715 >> 172.21.15.167:0/665944715 conn(0x7fca480d1e40 msgr2=0x7fca480d2220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:48.930 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:48.929+0000 7fca4d023640 1 -- 172.21.15.167:0/665944715 shutdown_connections 2024-05-11T08:06:48.930 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:48.929+0000 7fca4d023640 1 -- 172.21.15.167:0/665944715 wait complete. 2024-05-11T08:06:48.940 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-11T08:06:49.042 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.041+0000 7f59b3790640 1 Processor -- start 2024-05-11T08:06:49.042 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.041+0000 7f59b3790640 1 -- start start 2024-05-11T08:06:49.045 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.042+0000 7f59b3790640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f59ac0710f0 0x7f59ac071520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:49.045 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.042+0000 7f59b3790640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59ac15ff20 con 0x7f59ac148760 2024-05-11T08:06:49.045 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.042+0000 7f59b3790640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59ac0d0d80 con 0x7f59ac148b40 2024-05-11T08:06:49.045 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.042+0000 7f59b3790640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f59ac0d1330 con 0x7f59ac0710f0 2024-05-11T08:06:49.045 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.042+0000 7f59b152c640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f59ac148760 0x7f59ac15f810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:55984/0 (socket says 172.21.15.167:55984) 2024-05-11T08:06:49.045 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.042+0000 7f59b152c640 1 -- 172.21.15.167:0/2007133221 learned_addr learned my addr 172.21.15.167:0/2007133221 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T08:06:49.045 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.043+0000 7f59b1d2d640 1 --2- 172.21.15.167:0/2007133221 >> v2:172.21.15.204:3300/0 conn(0x7f59ac0710f0 0x7f59ac071520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:49.045 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.043+0000 7f59abfff640 1 -- 172.21.15.167:0/2007133221 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4208542594 0 0) 0x7f59ac15ff20 con 0x7f59ac148760 2024-05-11T08:06:49.046 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.043+0000 7f59abfff640 1 -- 172.21.15.167:0/2007133221 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5994003520 con 0x7f59ac148760 2024-05-11T08:06:49.046 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.043+0000 7f59abfff640 1 -- 172.21.15.167:0/2007133221 <== mon.0 v1:172.21.15.167:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4191852116 0 0) 0x7f5994003520 con 0x7f59ac148760 2024-05-11T08:06:49.046 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.043+0000 7f59abfff640 1 -- 172.21.15.167:0/2007133221 >> v2:172.21.15.204:3300/0 conn(0x7f59ac0710f0 msgr2=0x7f59ac071520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:49.046 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.043+0000 7f59abfff640 1 --2- 172.21.15.167:0/2007133221 >> v2:172.21.15.204:3300/0 conn(0x7f59ac0710f0 0x7f59ac071520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:49.046 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.043+0000 7f59abfff640 1 -- 172.21.15.167:0/2007133221 >> v1:172.21.15.204:6789/0 conn(0x7f59ac148b40 legacy=0x7f59ac0d0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:49.046 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.043+0000 7f59abfff640 1 -- 172.21.15.167:0/2007133221 --> v1:172.21.15.167:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59ac0d1050 con 0x7f59ac148760 2024-05-11T08:06:49.046 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.043+0000 7f59abfff640 1 -- 172.21.15.167:0/2007133221 <== mon.0 v1:172.21.15.167:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3602277846 0 0) 0x7f599c001a70 con 0x7f59ac148760 2024-05-11T08:06:49.046 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.043+0000 7f59abfff640 1 -- 172.21.15.167:0/2007133221 >> v1:172.21.15.167:6789/0 conn(0x7f59ac148760 legacy=0x7f59ac15f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:49.046 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.043+0000 7f59abfff640 1 --2- 172.21.15.167:0/2007133221 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5994003a50 0x7f5994003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:49.046 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.043+0000 7f59abfff640 1 --2- 172.21.15.167:0/2007133221 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5994004800 0x7f5994004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:49.046 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.043+0000 7f59abfff640 1 --2- 172.21.15.167:0/2007133221 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f59940051c0 0x7f599400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:49.046 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.043+0000 7f59abfff640 1 -- 172.21.15.167:0/2007133221 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f59ac0d1330 con 0x7f5994003a50 2024-05-11T08:06:49.046 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.043+0000 7f59abfff640 1 -- 172.21.15.167:0/2007133221 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f599400c160 con 0x7f59940051c0 2024-05-11T08:06:49.046 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.043+0000 7f59abfff640 1 -- 172.21.15.167:0/2007133221 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f599400c360 con 0x7f5994004800 2024-05-11T08:06:49.046 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.043+0000 7f59b152c640 1 --2- 172.21.15.167:0/2007133221 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5994003a50 0x7f5994003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:49.047 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.043+0000 7f59b0d2b640 1 --2- 172.21.15.167:0/2007133221 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5994004800 0x7f5994004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:49.047 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.043+0000 7f59b1d2d640 1 --2- 172.21.15.167:0/2007133221 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f59940051c0 0x7f599400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:49.047 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.043+0000 7f59b152c640 1 -- 172.21.15.167:0/2007133221 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5994004800 msgr2=0x7f5994004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:49.047 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.043+0000 7f59b152c640 1 --2- 172.21.15.167:0/2007133221 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5994004800 0x7f5994004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:49.047 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.043+0000 7f59b152c640 1 -- 172.21.15.167:0/2007133221 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f59940051c0 msgr2=0x7f599400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:49.047 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.043+0000 7f59b152c640 1 --2- 172.21.15.167:0/2007133221 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f59940051c0 0x7f599400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:49.047 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.043+0000 7f59b1d2d640 1 -- 172.21.15.167:0/2007133221 reap_dead start 2024-05-11T08:06:49.047 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.043+0000 7f59b152c640 1 -- 172.21.15.167:0/2007133221 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5994003520 con 0x7f5994003a50 2024-05-11T08:06:49.047 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.043+0000 7f59b0d2b640 1 --2- 172.21.15.167:0/2007133221 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5994004800 0x7f5994004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T08:06:49.047 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.044+0000 7f59b152c640 1 --2- 172.21.15.167:0/2007133221 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5994003a50 0x7f5994003e70 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f599c00a830 tx=0x7f5994005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=237a74636af4f54e server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:49.047 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.044+0000 7f59abfff640 1 -- 172.21.15.167:0/2007133221 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f599c00a450 con 0x7f5994003a50 2024-05-11T08:06:49.047 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.044+0000 7f59abfff640 1 -- 172.21.15.167:0/2007133221 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f599c02f070 con 0x7f5994003a50 2024-05-11T08:06:49.047 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.044+0000 7f59abfff640 1 -- 172.21.15.167:0/2007133221 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f599c02bbc0 con 0x7f5994003a50 2024-05-11T08:06:49.047 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.045+0000 7f59b3790640 1 -- 172.21.15.167:0/2007133221 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5994003a50 msgr2=0x7f5994003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:49.047 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.045+0000 7f59b3790640 1 --2- 172.21.15.167:0/2007133221 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5994003a50 0x7f5994003e70 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f599c00a830 tx=0x7f5994005830 comp rx=0 tx=0).stop 2024-05-11T08:06:49.048 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.045+0000 7f59b3790640 1 -- 172.21.15.167:0/2007133221 shutdown_connections 2024-05-11T08:06:49.048 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.045+0000 7f59b3790640 1 --2- 172.21.15.167:0/2007133221 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5994003a50 0x7f5994003e70 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:49.048 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.045+0000 7f59b3790640 1 -- 172.21.15.167:0/2007133221 >> 172.21.15.167:0/2007133221 conn(0x7f59ac073330 msgr2=0x7f59ac073740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:49.048 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.045+0000 7f59b3790640 1 -- 172.21.15.167:0/2007133221 shutdown_connections 2024-05-11T08:06:49.048 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.045+0000 7f59b3790640 1 -- 172.21.15.167:0/2007133221 wait complete. 2024-05-11T08:06:49.050 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.046+0000 7f59b3790640 1 Processor -- start 2024-05-11T08:06:49.050 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.046+0000 7f59b3790640 1 -- start start 2024-05-11T08:06:49.050 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.047+0000 7f59b3790640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5994003a50 0x7f59ac15edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:49.051 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.047+0000 7f59b3790640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f59ac13d130 0x7f59ac13f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:49.051 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.047+0000 7f59b3790640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f59ac1448e0 0x7f59ac13fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:49.051 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.047+0000 7f59b3790640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f59ac0cef80 con 0x7f5994003a50 2024-05-11T08:06:49.051 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.047+0000 7f59b3790640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f59ac0ceca0 con 0x7f59ac13d130 2024-05-11T08:06:49.051 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.047+0000 7f59b3790640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f59ac0b6d80 con 0x7f59ac1448e0 2024-05-11T08:06:49.051 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.047+0000 7f59b152c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5994003a50 0x7f59ac15edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:49.051 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.047+0000 7f59b152c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5994003a50 0x7f59ac15edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:43952/0 (socket says 172.21.15.167:43952) 2024-05-11T08:06:49.051 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.047+0000 7f59b152c640 1 -- 172.21.15.167:0/1258439613 learned_addr learned my addr 172.21.15.167:0/1258439613 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:49.051 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.047+0000 7f59b152c640 1 -- 172.21.15.167:0/1258439613 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f59ac1448e0 msgr2=0x7f59ac13fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:49.051 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.047+0000 7f59b0d2b640 1 --2- 172.21.15.167:0/1258439613 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f59ac13d130 0x7f59ac13f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:49.051 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.047+0000 7f59b152c640 1 --2- 172.21.15.167:0/1258439613 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f59ac1448e0 0x7f59ac13fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:49.051 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.047+0000 7f59b152c640 1 -- 172.21.15.167:0/1258439613 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f59ac13d130 msgr2=0x7f59ac13f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:49.051 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.047+0000 7f59b152c640 1 --2- 172.21.15.167:0/1258439613 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f59ac13d130 0x7f59ac13f4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:49.051 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.047+0000 7f59b152c640 1 -- 172.21.15.167:0/1258439613 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59ac144ea0 con 0x7f5994003a50 2024-05-11T08:06:49.051 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.047+0000 7f59b152c640 1 --2- 172.21.15.167:0/1258439613 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5994003a50 0x7f59ac15edf0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f599c02b430 tx=0x7f59ac0730f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:49.052 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.047+0000 7f59a9ffb640 1 -- 172.21.15.167:0/1258439613 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f599c001fa0 con 0x7f5994003a50 2024-05-11T08:06:49.052 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.047+0000 7f59a9ffb640 1 -- 172.21.15.167:0/1258439613 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f599c02bb90 con 0x7f5994003a50 2024-05-11T08:06:49.052 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.047+0000 7f59b3790640 1 -- 172.21.15.167:0/1258439613 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59ac145180 con 0x7f5994003a50 2024-05-11T08:06:49.052 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.047+0000 7f59a9ffb640 1 -- 172.21.15.167:0/1258439613 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f599c034470 con 0x7f5994003a50 2024-05-11T08:06:49.052 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.048+0000 7f59b3790640 1 -- 172.21.15.167:0/1258439613 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59ac145740 con 0x7f5994003a50 2024-05-11T08:06:49.052 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.049+0000 7f59b3790640 1 -- 172.21.15.167:0/1258439613 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59ac069bc0 con 0x7f5994003a50 2024-05-11T08:06:49.052 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.049+0000 7f59a9ffb640 1 -- 172.21.15.167:0/1258439613 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f599c034610 con 0x7f5994003a50 2024-05-11T08:06:49.052 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.050+0000 7f59a9ffb640 1 --2- 172.21.15.167:0/1258439613 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f59980762d0 0x7f5998078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:49.052 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.050+0000 7f59a9ffb640 1 -- 172.21.15.167:0/1258439613 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f599c0b7ad0 con 0x7f5994003a50 2024-05-11T08:06:49.052 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.050+0000 7f59b0d2b640 1 --2- 172.21.15.167:0/1258439613 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f59980762d0 0x7f5998078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:49.052 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.050+0000 7f59b0d2b640 1 --2- 172.21.15.167:0/1258439613 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f59980762d0 0x7f5998078790 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f59a0007fd0 tx=0x7f59a0009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:49.054 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.054+0000 7f59a9ffb640 1 -- 172.21.15.167:0/1258439613 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f599c0812b0 con 0x7f5994003a50 2024-05-11T08:06:49.236 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.236+0000 7f59b3790640 1 -- 172.21.15.167:0/1258439613 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f59ac069bc0 con 0x7f5994003a50 2024-05-11T08:06:49.237 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.237+0000 7f59a9ffb640 1 -- 172.21.15.167:0/1258439613 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) ==== 74+0+17003 (secure 0 0 0) 0x7f599c080bb0 con 0x7f5994003a50 2024-05-11T08:06:49.238 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T08:06:49.238 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":40,"fsid":"2034778d-8ff1-4823-aa44-ded32b518241","created":"2024-05-11T08:05:31.612421+0000","modified":"2024-05-11T08:06:47.943973+0000","last_up_change":"2024-05-11T08:05:42.557382+0000","last_in_change":"2024-05-11T08:05:33.757311+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-11T08:05:43.931263+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-05-11T08:06:45.857156+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-05-11T08:06:46.898591+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"587eaced-5ded-406d-884b-30c1caa8686b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":3863307356},{"type":"v1","addr":"172.21.15.167:6806","nonce":3863307356}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":3863307356},{"type":"v1","addr":"172.21.15.167:6810","nonce":3863307356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":3863307356},{"type":"v1","addr":"172.21.15.167:6815","nonce":3863307356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":3863307356},{"type":"v1","addr":"172.21.15.167:6813","nonce":3863307356}]},"public_addr":"172.21.15.167:6806/3863307356","cluster_addr":"172.21.15.167:6810/3863307356","heartbeat_back_addr":"172.21.15.167:6815/3863307356","heartbeat_front_addr":"172.21.15.167:6813/3863307356","state":["exists","up"]},{"osd":1,"uuid":"37c59104-2dd0-4cf9-85ea-56636f6b02a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":1585024048},{"type":"v1","addr":"172.21.15.167:6817","nonce":1585024048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1585024048},{"type":"v1","addr":"172.21.15.167:6819","nonce":1585024048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1585024048},{"type":"v1","addr":"172.21.15.167:6823","nonce":1585024048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":1585024048},{"type":"v1","addr":"172.21.15.167:6821","nonce":1585024048}]},"public_addr":"172.21.15.167:6817/1585024048","cluster_addr":"172.21.15.167:6819/1585024048","heartbeat_back_addr":"172.21.15.167:6823/1585024048","heartbeat_front_addr":"172.21.15.167:6821/1585024048","state":["exists","up"]},{"osd":2,"uuid":"2bb4f7f5-c698-451b-b4bb-78e716921291","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":3242166900},{"type":"v1","addr":"172.21.15.167:6801","nonce":3242166900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":3242166900},{"type":"v1","addr":"172.21.15.167:6803","nonce":3242166900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6809","nonce":3242166900},{"type":"v1","addr":"172.21.15.167:6811","nonce":3242166900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6805","nonce":3242166900},{"type":"v1","addr":"172.21.15.167:6807","nonce":3242166900}]},"public_addr":"172.21.15.167:6801/3242166900","cluster_addr":"172.21.15.167:6803/3242166900","heartbeat_back_addr":"172.21.15.167:6811/3242166900","heartbeat_front_addr":"172.21.15.167:6807/3242166900","state":["exists","up"]},{"osd":3,"uuid":"9dd26759-a338-49d2-8103-45d068b9c000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":453689293},{"type":"v1","addr":"172.21.15.167:6825","nonce":453689293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":453689293},{"type":"v1","addr":"172.21.15.167:6827","nonce":453689293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":453689293},{"type":"v1","addr":"172.21.15.167:6831","nonce":453689293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":453689293},{"type":"v1","addr":"172.21.15.167:6829","nonce":453689293}]},"public_addr":"172.21.15.167:6825/453689293","cluster_addr":"172.21.15.167:6827/453689293","heartbeat_back_addr":"172.21.15.167:6831/453689293","heartbeat_front_addr":"172.21.15.167:6829/453689293","state":["exists","up"]},{"osd":4,"uuid":"cb27250d-2a80-4c7b-8cef-3a7af05c8a16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1311831500},{"type":"v1","addr":"172.21.15.204:6801","nonce":1311831500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1311831500},{"type":"v1","addr":"172.21.15.204:6803","nonce":1311831500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1311831500},{"type":"v1","addr":"172.21.15.204:6807","nonce":1311831500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1311831500},{"type":"v1","addr":"172.21.15.204:6805","nonce":1311831500}]},"public_addr":"172.21.15.204:6801/1311831500","cluster_addr":"172.21.15.204:6803/1311831500","heartbeat_back_addr":"172.21.15.204:6807/1311831500","heartbeat_front_addr":"172.21.15.204:6805/1311831500","state":["exists","up"]},{"osd":5,"uuid":"92998541-ffec-430b-ab76-8c3cb7c2752d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2921847521},{"type":"v1","addr":"172.21.15.204:6825","nonce":2921847521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2921847521},{"type":"v1","addr":"172.21.15.204:6827","nonce":2921847521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2921847521},{"type":"v1","addr":"172.21.15.204:6831","nonce":2921847521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2921847521},{"type":"v1","addr":"172.21.15.204:6829","nonce":2921847521}]},"public_addr":"172.21.15.204:6825/2921847521","cluster_addr":"172.21.15.204:6827/2921847521","heartbeat_back_addr":"172.21.15.204:6831/2921847521","heartbeat_front_addr":"172.21.15.204:6829/2921847521","state":["exists","up"]},{"osd":6,"uuid":"9543479c-2ec5-42a8-8933-707bfed2e048","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":2889851077},{"type":"v1","addr":"172.21.15.204:6817","nonce":2889851077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":2889851077},{"type":"v1","addr":"172.21.15.204:6819","nonce":2889851077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":2889851077},{"type":"v1","addr":"172.21.15.204:6823","nonce":2889851077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":2889851077},{"type":"v1","addr":"172.21.15.204:6821","nonce":2889851077}]},"public_addr":"172.21.15.204:6817/2889851077","cluster_addr":"172.21.15.204:6819/2889851077","heartbeat_back_addr":"172.21.15.204:6823/2889851077","heartbeat_front_addr":"172.21.15.204:6821/2889851077","state":["exists","up"]},{"osd":7,"uuid":"c45a5647-8358-409e-af91-c05dda65f0b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":1949534184},{"type":"v1","addr":"172.21.15.204:6809","nonce":1949534184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":1949534184},{"type":"v1","addr":"172.21.15.204:6811","nonce":1949534184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1949534184},{"type":"v1","addr":"172.21.15.204:6815","nonce":1949534184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1949534184},{"type":"v1","addr":"172.21.15.204:6813","nonce":1949534184}]},"public_addr":"172.21.15.204:6809/1949534184","cluster_addr":"172.21.15.204:6811/1949534184","heartbeat_back_addr":"172.21.15.204:6815/1949534184","heartbeat_front_addr":"172.21.15.204:6813/1949534184","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.952253+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.940689+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.860682+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.929350+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.901627+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.957451+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.987249+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.951197+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-11T08:06:49.240 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.239+0000 7f59b3790640 1 -- 172.21.15.167:0/1258439613 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f59980762d0 msgr2=0x7f5998078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:49.240 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.239+0000 7f59b3790640 1 --2- 172.21.15.167:0/1258439613 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f59980762d0 0x7f5998078790 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f59a0007fd0 tx=0x7f59a0009040 comp rx=0 tx=0).stop 2024-05-11T08:06:49.240 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.239+0000 7f59b3790640 1 -- 172.21.15.167:0/1258439613 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5994003a50 msgr2=0x7f59ac15edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:49.240 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.239+0000 7f59b3790640 1 --2- 172.21.15.167:0/1258439613 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5994003a50 0x7f59ac15edf0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f599c02b430 tx=0x7f59ac0730f0 comp rx=0 tx=0).stop 2024-05-11T08:06:49.240 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.239+0000 7f59b3790640 1 -- 172.21.15.167:0/1258439613 shutdown_connections 2024-05-11T08:06:49.240 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.239+0000 7f59b3790640 1 --2- 172.21.15.167:0/1258439613 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f59980762d0 0x7f5998078790 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:49.240 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.239+0000 7f59b3790640 1 --2- 172.21.15.167:0/1258439613 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f59ac1448e0 0x7f59ac13fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:49.240 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.239+0000 7f59b3790640 1 --2- 172.21.15.167:0/1258439613 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f59ac13d130 0x7f59ac13f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:49.240 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.239+0000 7f59b3790640 1 --2- 172.21.15.167:0/1258439613 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5994003a50 0x7f59ac15edf0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:49.240 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.239+0000 7f59b3790640 1 -- 172.21.15.167:0/1258439613 >> 172.21.15.167:0/1258439613 conn(0x7f59ac073330 msgr2=0x7f59ac0ab3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:49.240 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.239+0000 7f59b3790640 1 -- 172.21.15.167:0/1258439613 shutdown_connections 2024-05-11T08:06:49.240 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.239+0000 7f59b3790640 1 -- 172.21.15.167:0/1258439613 wait complete. 2024-05-11T08:06:49.251 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-11T08:06:49.355 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.355+0000 7efc55ed1640 1 Processor -- start 2024-05-11T08:06:49.355 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.355+0000 7efc55ed1640 1 -- start start 2024-05-11T08:06:49.356 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.356+0000 7efc55ed1640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7efc50074260 0x7efc500b3d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:49.356 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.356+0000 7efc55ed1640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc500b65e0 con 0x7efc50161380 2024-05-11T08:06:49.356 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.356+0000 7efc55ed1640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc500b68c0 con 0x7efc5015ac80 2024-05-11T08:06:49.356 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.356+0000 7efc55ed1640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7efc50074670 con 0x7efc50074260 2024-05-11T08:06:49.356 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.356+0000 7efc4ffff640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7efc50074260 0x7efc500b3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:49.356 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.356+0000 7efc4effd640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7efc50161380 0x7efc50072950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:55986/0 (socket says 172.21.15.167:55986) 2024-05-11T08:06:49.356 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.356+0000 7efc4effd640 1 -- 172.21.15.167:0/3384380515 learned_addr learned my addr 172.21.15.167:0/3384380515 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T08:06:49.357 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.356+0000 7efc4ffff640 1 -- 172.21.15.167:0/3384380515 >> v1:172.21.15.204:6789/0 conn(0x7efc5015ac80 legacy=0x7efc50160c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:49.357 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.356+0000 7efc4ffff640 1 -- 172.21.15.167:0/3384380515 >> v1:172.21.15.167:6789/0 conn(0x7efc50161380 legacy=0x7efc50072950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:49.357 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.356+0000 7efc4e7fc640 1 -- 172.21.15.167:0/3384380515 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1442630279 0 0) 0x7efc500b65e0 con 0x7efc50161380 2024-05-11T08:06:49.357 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.356+0000 7efc4ffff640 1 -- 172.21.15.167:0/3384380515 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc500b6ba0 con 0x7efc50074260 2024-05-11T08:06:49.357 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.357+0000 7efc4ffff640 1 --2- 172.21.15.167:0/3384380515 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efc50074260 0x7efc500b3d50 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7efc480097a0 tx=0x7efc4802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9685c655772b7b28 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:49.357 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.357+0000 7efc4e7fc640 1 -- 172.21.15.167:0/3384380515 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc48037070 con 0x7efc50074260 2024-05-11T08:06:49.357 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.357+0000 7efc4e7fc640 1 -- 172.21.15.167:0/3384380515 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc48032440 con 0x7efc50074260 2024-05-11T08:06:49.357 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.357+0000 7efc4e7fc640 1 -- 172.21.15.167:0/3384380515 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc48032810 con 0x7efc50074260 2024-05-11T08:06:49.358 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.358+0000 7efc55ed1640 1 -- 172.21.15.167:0/3384380515 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efc50074260 msgr2=0x7efc500b3d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:49.358 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.358+0000 7efc55ed1640 1 --2- 172.21.15.167:0/3384380515 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efc50074260 0x7efc500b3d50 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7efc480097a0 tx=0x7efc4802a180 comp rx=0 tx=0).stop 2024-05-11T08:06:49.358 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.358+0000 7efc55ed1640 1 -- 172.21.15.167:0/3384380515 shutdown_connections 2024-05-11T08:06:49.358 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.358+0000 7efc55ed1640 1 --2- 172.21.15.167:0/3384380515 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efc50074260 0x7efc500b3d50 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:49.358 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.358+0000 7efc55ed1640 1 -- 172.21.15.167:0/3384380515 >> 172.21.15.167:0/3384380515 conn(0x7efc5006c450 msgr2=0x7efc5006c860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:49.358 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.358+0000 7efc55ed1640 1 -- 172.21.15.167:0/3384380515 shutdown_connections 2024-05-11T08:06:49.358 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.358+0000 7efc55ed1640 1 -- 172.21.15.167:0/3384380515 wait complete. 2024-05-11T08:06:49.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.359+0000 7efc55ed1640 1 Processor -- start 2024-05-11T08:06:49.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.359+0000 7efc55ed1640 1 -- start start 2024-05-11T08:06:49.361 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.360+0000 7efc55ed1640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efc50074260 0x7efc50159360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:49.361 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.360+0000 7efc55ed1640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efc5015ac80 0x7efc500aacc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:49.361 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.360+0000 7efc55ed1640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efc50161380 0x7efc501345c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:49.361 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.360+0000 7efc55ed1640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7efc5015b060 con 0x7efc50074260 2024-05-11T08:06:49.361 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.360+0000 7efc55ed1640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7efc500b4290 con 0x7efc5015ac80 2024-05-11T08:06:49.361 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.360+0000 7efc55ed1640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7efc50073060 con 0x7efc50161380 2024-05-11T08:06:49.361 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.360+0000 7efc4f7fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efc50074260 0x7efc50159360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:49.361 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.360+0000 7efc4f7fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efc50074260 0x7efc50159360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:43960/0 (socket says 172.21.15.167:43960) 2024-05-11T08:06:49.361 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.360+0000 7efc4f7fe640 1 -- 172.21.15.167:0/2711391509 learned_addr learned my addr 172.21.15.167:0/2711391509 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:49.362 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.360+0000 7efc4f7fe640 1 -- 172.21.15.167:0/2711391509 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efc50161380 msgr2=0x7efc501345c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:49.362 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.360+0000 7efc4f7fe640 1 --2- 172.21.15.167:0/2711391509 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efc50161380 0x7efc501345c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:49.362 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.360+0000 7efc4f7fe640 1 -- 172.21.15.167:0/2711391509 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efc5015ac80 msgr2=0x7efc500aacc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:49.362 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.360+0000 7efc4f7fe640 1 --2- 172.21.15.167:0/2711391509 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efc5015ac80 0x7efc500aacc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:49.362 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.360+0000 7efc4f7fe640 1 -- 172.21.15.167:0/2711391509 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc500b03c0 con 0x7efc50074260 2024-05-11T08:06:49.362 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.360+0000 7efc4f7fe640 1 --2- 172.21.15.167:0/2711391509 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efc50074260 0x7efc50159360 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7efc4000c7a0 tx=0x7efc4000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:49.362 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.360+0000 7efc37fff640 1 -- 172.21.15.167:0/2711391509 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc40017070 con 0x7efc50074260 2024-05-11T08:06:49.362 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.361+0000 7efc37fff640 1 -- 172.21.15.167:0/2711391509 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc40003da0 con 0x7efc50074260 2024-05-11T08:06:49.362 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.361+0000 7efc55ed1640 1 -- 172.21.15.167:0/2711391509 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc500afb20 con 0x7efc50074260 2024-05-11T08:06:49.362 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.361+0000 7efc37fff640 1 -- 172.21.15.167:0/2711391509 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc400052b0 con 0x7efc50074260 2024-05-11T08:06:49.362 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.361+0000 7efc55ed1640 1 -- 172.21.15.167:0/2711391509 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc500b00e0 con 0x7efc50074260 2024-05-11T08:06:49.363 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.362+0000 7efc55ed1640 1 -- 172.21.15.167:0/2711391509 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc50069520 con 0x7efc50074260 2024-05-11T08:06:49.363 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.362+0000 7efc37fff640 1 -- 172.21.15.167:0/2711391509 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7efc4001a070 con 0x7efc50074260 2024-05-11T08:06:49.363 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.363+0000 7efc37fff640 1 --2- 172.21.15.167:0/2711391509 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7efc3c0762d0 0x7efc3c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:49.363 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.363+0000 7efc4effd640 1 --2- 172.21.15.167:0/2711391509 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7efc3c0762d0 0x7efc3c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:49.363 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.363+0000 7efc37fff640 1 -- 172.21.15.167:0/2711391509 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7efc40096e00 con 0x7efc50074260 2024-05-11T08:06:49.363 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.363+0000 7efc4effd640 1 --2- 172.21.15.167:0/2711391509 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7efc3c0762d0 0x7efc3c078790 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7efc44007fd0 tx=0x7efc44007960 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:49.367 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.367+0000 7efc37fff640 1 -- 172.21.15.167:0/2711391509 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7efc40060660 con 0x7efc50074260 2024-05-11T08:06:49.586 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.586+0000 7efc55ed1640 1 -- 172.21.15.167:0/2711391509 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7efc50070120 con 0x7efc50074260 2024-05-11T08:06:49.602 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.602+0000 7efc37fff640 1 -- 172.21.15.167:0/2711391509 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v15) ==== 118+0+0 (secure 0 0 0) 0x7efc4005ff60 con 0x7efc50074260 2024-05-11T08:06:49.605 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.605+0000 7efc55ed1640 1 -- 172.21.15.167:0/2711391509 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7efc3c0762d0 msgr2=0x7efc3c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:49.605 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.605+0000 7efc55ed1640 1 --2- 172.21.15.167:0/2711391509 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7efc3c0762d0 0x7efc3c078790 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7efc44007fd0 tx=0x7efc44007960 comp rx=0 tx=0).stop 2024-05-11T08:06:49.605 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.605+0000 7efc55ed1640 1 -- 172.21.15.167:0/2711391509 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efc50074260 msgr2=0x7efc50159360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:49.605 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.605+0000 7efc55ed1640 1 --2- 172.21.15.167:0/2711391509 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efc50074260 0x7efc50159360 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7efc4000c7a0 tx=0x7efc4000cc60 comp rx=0 tx=0).stop 2024-05-11T08:06:49.605 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.605+0000 7efc55ed1640 1 -- 172.21.15.167:0/2711391509 shutdown_connections 2024-05-11T08:06:49.605 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.605+0000 7efc55ed1640 1 --2- 172.21.15.167:0/2711391509 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7efc3c0762d0 0x7efc3c078790 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:49.606 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.605+0000 7efc55ed1640 1 --2- 172.21.15.167:0/2711391509 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efc50161380 0x7efc501345c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:49.606 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.605+0000 7efc55ed1640 1 --2- 172.21.15.167:0/2711391509 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efc5015ac80 0x7efc500aacc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:49.606 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.605+0000 7efc55ed1640 1 --2- 172.21.15.167:0/2711391509 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efc50074260 0x7efc50159360 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:49.606 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.605+0000 7efc55ed1640 1 -- 172.21.15.167:0/2711391509 >> 172.21.15.167:0/2711391509 conn(0x7efc5006c450 msgr2=0x7efc5006f2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:49.606 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.605+0000 7efc55ed1640 1 -- 172.21.15.167:0/2711391509 shutdown_connections 2024-05-11T08:06:49.606 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.605+0000 7efc55ed1640 1 -- 172.21.15.167:0/2711391509 wait complete. 2024-05-11T08:06:49.616 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-11T08:06:49.616 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-11T08:06:49.720 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.720+0000 7f935dfb6640 1 Processor -- start 2024-05-11T08:06:49.720 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.720+0000 7f935dfb6640 1 -- start start 2024-05-11T08:06:49.720 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.721+0000 7f935dfb6640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9358073270 0x7f935815d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:49.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.721+0000 7f935dfb6640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f9358073650 0x7f935806e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:49.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.721+0000 7f935dfb6640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f935815c6d0 con 0x7f93580ce8f0 2024-05-11T08:06:49.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.721+0000 7f935dfb6640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f9358160b00 con 0x7f9358073650 2024-05-11T08:06:49.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.721+0000 7f935dfb6640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f935815cce0 con 0x7f9358073270 2024-05-11T08:06:49.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.721+0000 7f93577fe640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9358073270 0x7f935815d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:49.722 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.721+0000 7f9356ffd640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f9358073650 0x7f935806e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:49.722 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.721+0000 7f93577fe640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9358073270 0x7f935815d170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.167:35338/0 (socket says 172.21.15.167:35338) 2024-05-11T08:06:49.722 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.721+0000 7f93577fe640 1 -- 172.21.15.167:0/50985986 learned_addr learned my addr 172.21.15.167:0/50985986 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:49.722 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.721+0000 7f9357fff640 1 --1- 172.21.15.167:0/50985986 >> v1:172.21.15.204:6789/0 conn(0x7f93580ce8f0 0x7f93580ced80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.167:42180/0 (socket says 172.21.15.167:42180) 2024-05-11T08:06:49.722 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.721+0000 7f93577fe640 1 -- 172.21.15.167:0/50985986 >> v2:172.21.15.204:3300/0 conn(0x7f9358073650 msgr2=0x7f935806e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:49.722 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.721+0000 7f93577fe640 1 --2- 172.21.15.167:0/50985986 >> v2:172.21.15.204:3300/0 conn(0x7f9358073650 0x7f935806e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:49.722 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.721+0000 7f93577fe640 1 -- 172.21.15.167:0/50985986 >> v1:172.21.15.204:6789/0 conn(0x7f93580ce8f0 legacy=0x7f93580ced80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:49.722 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.721+0000 7f93577fe640 1 -- 172.21.15.167:0/50985986 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93580700b0 con 0x7f9358073270 2024-05-11T08:06:49.722 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.721+0000 7f9356ffd640 1 --2- 172.21.15.167:0/50985986 >> v2:172.21.15.204:3300/0 conn(0x7f9358073650 0x7f935806e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T08:06:49.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.722+0000 7f93577fe640 1 --2- 172.21.15.167:0/50985986 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9358073270 0x7f935815d170 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f9348009870 tx=0x7f934802a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=29238afb94658cb2 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:49.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.722+0000 7f93567fc640 1 -- 172.21.15.167:0/50985986 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9348037070 con 0x7f9358073270 2024-05-11T08:06:49.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.722+0000 7f93567fc640 1 -- 172.21.15.167:0/50985986 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9348032440 con 0x7f9358073270 2024-05-11T08:06:49.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.722+0000 7f93567fc640 1 -- 172.21.15.167:0/50985986 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9348032810 con 0x7f9358073270 2024-05-11T08:06:49.724 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.723+0000 7f935dfb6640 1 -- 172.21.15.167:0/50985986 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9358073270 msgr2=0x7f935815d170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:49.724 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.723+0000 7f935dfb6640 1 --2- 172.21.15.167:0/50985986 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9358073270 0x7f935815d170 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f9348009870 tx=0x7f934802a270 comp rx=0 tx=0).stop 2024-05-11T08:06:49.724 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.723+0000 7f935dfb6640 1 -- 172.21.15.167:0/50985986 shutdown_connections 2024-05-11T08:06:49.724 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.723+0000 7f935dfb6640 1 --2- 172.21.15.167:0/50985986 >> v2:172.21.15.204:3300/0 conn(0x7f9358073650 0x7f935806e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:49.724 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.723+0000 7f935dfb6640 1 --2- 172.21.15.167:0/50985986 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9358073270 0x7f935815d170 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:49.724 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.723+0000 7f935dfb6640 1 -- 172.21.15.167:0/50985986 >> 172.21.15.167:0/50985986 conn(0x7f935806c450 msgr2=0x7f935806c860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:49.724 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.723+0000 7f935dfb6640 1 -- 172.21.15.167:0/50985986 shutdown_connections 2024-05-11T08:06:49.724 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.723+0000 7f935dfb6640 1 -- 172.21.15.167:0/50985986 wait complete. 2024-05-11T08:06:49.724 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.724+0000 7f935dfb6640 1 Processor -- start 2024-05-11T08:06:49.724 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.724+0000 7f935dfb6640 1 -- start start 2024-05-11T08:06:49.726 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.725+0000 7f935dfb6640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9358073270 0x7f9358154f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:49.727 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.725+0000 7f935dfb6640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9358073650 0x7f93580cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:49.727 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.725+0000 7f935dfb6640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f93580ce8f0 0x7f93581410a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:49.727 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.725+0000 7f935dfb6640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9358073ad0 con 0x7f9358073270 2024-05-11T08:06:49.727 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.725+0000 7f935dfb6640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f9358070390 con 0x7f9358073650 2024-05-11T08:06:49.727 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.725+0000 7f935dfb6640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f9358145bb0 con 0x7f93580ce8f0 2024-05-11T08:06:49.727 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.725+0000 7f93577fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9358073270 0x7f9358154f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:49.727 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.725+0000 7f93577fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9358073270 0x7f9358154f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:43968/0 (socket says 172.21.15.167:43968) 2024-05-11T08:06:49.727 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.725+0000 7f93577fe640 1 -- 172.21.15.167:0/3866516268 learned_addr learned my addr 172.21.15.167:0/3866516268 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:49.727 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.725+0000 7f9356ffd640 1 --2- 172.21.15.167:0/3866516268 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9358073650 0x7f93580cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:49.727 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.725+0000 7f9357fff640 1 --2- 172.21.15.167:0/3866516268 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f93580ce8f0 0x7f93581410a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:49.727 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.725+0000 7f93577fe640 1 -- 172.21.15.167:0/3866516268 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f93580ce8f0 msgr2=0x7f93581410a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:49.727 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.725+0000 7f93577fe640 1 --2- 172.21.15.167:0/3866516268 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f93580ce8f0 0x7f93581410a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:49.729 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.725+0000 7f93577fe640 1 -- 172.21.15.167:0/3866516268 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9358073650 msgr2=0x7f93580cfb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:49.729 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.725+0000 7f93577fe640 1 --2- 172.21.15.167:0/3866516268 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9358073650 0x7f93580cfb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:49.729 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.725+0000 7f93577fe640 1 -- 172.21.15.167:0/3866516268 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93581f9a70 con 0x7f9358073270 2024-05-11T08:06:49.729 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.725+0000 7f9357fff640 1 --2- 172.21.15.167:0/3866516268 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f93580ce8f0 0x7f93581410a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T08:06:49.729 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.726+0000 7f93577fe640 1 --2- 172.21.15.167:0/3866516268 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9358073270 0x7f9358154f40 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f9348009870 tx=0x7f93480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:49.729 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.726+0000 7f933ffff640 1 -- 172.21.15.167:0/3866516268 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9348041070 con 0x7f9358073270 2024-05-11T08:06:49.729 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.726+0000 7f933ffff640 1 -- 172.21.15.167:0/3866516268 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f934803c890 con 0x7f9358073270 2024-05-11T08:06:49.729 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.726+0000 7f935dfb6640 1 -- 172.21.15.167:0/3866516268 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93581f9d50 con 0x7f9358073270 2024-05-11T08:06:49.729 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.726+0000 7f933ffff640 1 -- 172.21.15.167:0/3866516268 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9348037040 con 0x7f9358073270 2024-05-11T08:06:49.729 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.726+0000 7f935dfb6640 1 -- 172.21.15.167:0/3866516268 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93581fa310 con 0x7f9358073270 2024-05-11T08:06:49.729 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.727+0000 7f935dfb6640 1 -- 172.21.15.167:0/3866516268 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f935806dbe0 con 0x7f9358073270 2024-05-11T08:06:49.730 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.727+0000 7f933ffff640 1 -- 172.21.15.167:0/3866516268 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f9348044070 con 0x7f9358073270 2024-05-11T08:06:49.730 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.728+0000 7f933ffff640 1 --2- 172.21.15.167:0/3866516268 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f9344076300 0x7f93440787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:49.730 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.728+0000 7f933ffff640 1 -- 172.21.15.167:0/3866516268 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6101+0+0 (secure 0 0 0) 0x7f93480b7dc0 con 0x7f9358073270 2024-05-11T08:06:49.730 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.728+0000 7f9356ffd640 1 --2- 172.21.15.167:0/3866516268 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f9344076300 0x7f93440787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:49.730 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.728+0000 7f9356ffd640 1 --2- 172.21.15.167:0/3866516268 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f9344076300 0x7f93440787c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f934c009560 tx=0x7f934c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:49.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.732+0000 7f933ffff640 1 -- 172.21.15.167:0/3866516268 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f935806dbe0 con 0x7f9358073270 2024-05-11T08:06:49.954 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:49.954+0000 7f935dfb6640 1 -- 172.21.15.167:0/3866516268 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f93580b3ef0 con 0x7f9358073270 2024-05-11T08:06:50.612 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:50.612+0000 7f933ffff640 1 -- 172.21.15.167:0/3866516268 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v16) ==== 122+0+0 (secure 0 0 0) 0x7f93580b3ef0 con 0x7f9358073270 2024-05-11T08:06:50.615 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:50.615+0000 7f935dfb6640 1 -- 172.21.15.167:0/3866516268 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f9344076300 msgr2=0x7f93440787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:50.615 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:50.615+0000 7f935dfb6640 1 --2- 172.21.15.167:0/3866516268 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f9344076300 0x7f93440787c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f934c009560 tx=0x7f934c008040 comp rx=0 tx=0).stop 2024-05-11T08:06:50.615 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:50.615+0000 7f935dfb6640 1 -- 172.21.15.167:0/3866516268 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9358073270 msgr2=0x7f9358154f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:50.615 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:50.615+0000 7f935dfb6640 1 --2- 172.21.15.167:0/3866516268 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9358073270 0x7f9358154f40 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f9348009870 tx=0x7f93480046c0 comp rx=0 tx=0).stop 2024-05-11T08:06:50.616 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:50.615+0000 7f935dfb6640 1 -- 172.21.15.167:0/3866516268 shutdown_connections 2024-05-11T08:06:50.616 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:50.615+0000 7f935dfb6640 1 --2- 172.21.15.167:0/3866516268 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f9344076300 0x7f93440787c0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:50.616 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:50.615+0000 7f935dfb6640 1 --2- 172.21.15.167:0/3866516268 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f93580ce8f0 0x7f93581410a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:50.616 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:50.615+0000 7f935dfb6640 1 --2- 172.21.15.167:0/3866516268 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9358073650 0x7f93580cfb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:50.616 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:50.615+0000 7f935dfb6640 1 --2- 172.21.15.167:0/3866516268 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9358073270 0x7f9358154f40 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:50.616 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:50.615+0000 7f935dfb6640 1 -- 172.21.15.167:0/3866516268 >> 172.21.15.167:0/3866516268 conn(0x7f935806c450 msgr2=0x7f93581385e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:50.616 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:50.615+0000 7f935dfb6640 1 -- 172.21.15.167:0/3866516268 shutdown_connections 2024-05-11T08:06:50.616 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:50.615+0000 7f935dfb6640 1 -- 172.21.15.167:0/3866516268 wait complete. 2024-05-11T08:06:50.627 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-11T08:06:50.628 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-11T08:06:50.730 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:50.730+0000 7fad1d67c640 1 Processor -- start 2024-05-11T08:06:50.730 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:50.730+0000 7fad1d67c640 1 -- start start 2024-05-11T08:06:50.731 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:50.731+0000 7fad1d67c640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fad181598d0 0x7fad180d2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:50.731 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:50.731+0000 7fad1d67c640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fad180d0640 0x7fad180d0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:50.731 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:50.731+0000 7fad1d67c640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad180d1190 con 0x7fad180cafb0 2024-05-11T08:06:50.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:50.731+0000 7fad1d67c640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7fad1806d500 con 0x7fad180d0640 2024-05-11T08:06:50.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:50.731+0000 7fad1d67c640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fad1806d7e0 con 0x7fad181598d0 2024-05-11T08:06:50.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:50.731+0000 7fad0e7fc640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fad180d0640 0x7fad180d0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:50.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:50.731+0000 7fad0e7fc640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fad180d0640 0x7fad180d0ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:43970/0 (socket says 172.21.15.167:43970) 2024-05-11T08:06:50.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:50.731+0000 7fad0e7fc640 1 -- 172.21.15.167:0/2081578905 learned_addr learned my addr 172.21.15.167:0/2081578905 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:50.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:50.731+0000 7fad16ffd640 1 --2- 172.21.15.167:0/2081578905 >> v2:172.21.15.204:3300/0 conn(0x7fad181598d0 0x7fad180d2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:50.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:50.731+0000 7fad0e7fc640 1 -- 172.21.15.167:0/2081578905 >> v2:172.21.15.204:3300/0 conn(0x7fad181598d0 msgr2=0x7fad180d2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:50.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:50.731+0000 7fad0e7fc640 1 --2- 172.21.15.167:0/2081578905 >> v2:172.21.15.204:3300/0 conn(0x7fad181598d0 0x7fad180d2cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:50.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:50.731+0000 7fad0e7fc640 1 -- 172.21.15.167:0/2081578905 >> v1:172.21.15.204:6789/0 conn(0x7fad180cafb0 legacy=0x7fad1806dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:50.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:50.731+0000 7fad0e7fc640 1 -- 172.21.15.167:0/2081578905 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad1806dac0 con 0x7fad180d0640 2024-05-11T08:06:50.733 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:50.732+0000 7fad0e7fc640 1 --2- 172.21.15.167:0/2081578905 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fad180d0640 0x7fad180d0ad0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fad1808d610 tx=0x7fad0002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1c79ebf49f34987 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:50.733 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:50.732+0000 7fad167fc640 1 -- 172.21.15.167:0/2081578905 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad00037070 con 0x7fad180d0640 2024-05-11T08:06:50.733 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:50.732+0000 7fad167fc640 1 -- 172.21.15.167:0/2081578905 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad00032440 con 0x7fad180d0640 2024-05-11T08:06:50.734 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:50.732+0000 7fad167fc640 1 -- 172.21.15.167:0/2081578905 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad00032810 con 0x7fad180d0640 2024-05-11T08:06:50.734 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:50.733+0000 7fad1d67c640 1 -- 172.21.15.167:0/2081578905 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fad180d0640 msgr2=0x7fad180d0ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:50.734 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:50.733+0000 7fad1d67c640 1 --2- 172.21.15.167:0/2081578905 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fad180d0640 0x7fad180d0ad0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fad1808d610 tx=0x7fad0002a270 comp rx=0 tx=0).stop 2024-05-11T08:06:50.734 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:50.733+0000 7fad1d67c640 1 -- 172.21.15.167:0/2081578905 shutdown_connections 2024-05-11T08:06:50.734 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:50.733+0000 7fad1d67c640 1 --2- 172.21.15.167:0/2081578905 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fad180d0640 0x7fad180d0ad0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:50.734 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:50.733+0000 7fad1d67c640 1 --2- 172.21.15.167:0/2081578905 >> v2:172.21.15.204:3300/0 conn(0x7fad181598d0 0x7fad180d2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:50.734 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:50.733+0000 7fad1d67c640 1 -- 172.21.15.167:0/2081578905 >> 172.21.15.167:0/2081578905 conn(0x7fad180d1ba0 msgr2=0x7fad180d1fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:50.734 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:50.733+0000 7fad1d67c640 1 -- 172.21.15.167:0/2081578905 shutdown_connections 2024-05-11T08:06:50.734 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:50.733+0000 7fad1d67c640 1 -- 172.21.15.167:0/2081578905 wait complete. 2024-05-11T08:06:50.736 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:50.734+0000 7fad1d67c640 1 Processor -- start 2024-05-11T08:06:50.736 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:50.734+0000 7fad1d67c640 1 -- start start 2024-05-11T08:06:50.737 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:50.735+0000 7fad1d67c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad180cafb0 0x7fad180b9400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:50.737 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:50.735+0000 7fad1d67c640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fad180d0640 0x7fad180ba250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:50.737 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:50.735+0000 7fad1d67c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fad181598d0 0x7fad180ba790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:50.737 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:50.735+0000 7fad1d67c640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fad1806d1b0 con 0x7fad181598d0 2024-05-11T08:06:50.737 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:50.735+0000 7fad1d67c640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fad18159cb0 con 0x7fad180cafb0 2024-05-11T08:06:50.737 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:50.735+0000 7fad1d67c640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fad180cb3e0 con 0x7fad180d0640 2024-05-11T08:06:50.737 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:50.735+0000 7fad177fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fad181598d0 0x7fad180ba790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:50.737 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:50.735+0000 7fad177fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fad181598d0 0x7fad180ba790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:43984/0 (socket says 172.21.15.167:43984) 2024-05-11T08:06:50.737 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:50.735+0000 7fad177fe640 1 -- 172.21.15.167:0/3978921331 learned_addr learned my addr 172.21.15.167:0/3978921331 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:50.737 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:50.735+0000 7fad16ffd640 1 --2- 172.21.15.167:0/3978921331 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad180cafb0 0x7fad180b9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:50.737 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:50.735+0000 7fad0e7fc640 1 --2- 172.21.15.167:0/3978921331 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fad180d0640 0x7fad180ba250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:50.737 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:50.735+0000 7fad177fe640 1 -- 172.21.15.167:0/3978921331 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fad180d0640 msgr2=0x7fad180ba250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:50.737 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:50.735+0000 7fad177fe640 1 --2- 172.21.15.167:0/3978921331 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fad180d0640 0x7fad180ba250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:50.738 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:50.735+0000 7fad177fe640 1 -- 172.21.15.167:0/3978921331 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad180cafb0 msgr2=0x7fad180b9400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:50.738 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:50.735+0000 7fad177fe640 1 --2- 172.21.15.167:0/3978921331 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad180cafb0 0x7fad180b9400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:50.738 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:50.735+0000 7fad177fe640 1 -- 172.21.15.167:0/3978921331 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad180b8080 con 0x7fad181598d0 2024-05-11T08:06:50.738 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:50.735+0000 7fad16ffd640 1 --2- 172.21.15.167:0/3978921331 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad180cafb0 0x7fad180b9400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T08:06:50.738 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:50.735+0000 7fad0e7fc640 1 --2- 172.21.15.167:0/3978921331 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fad180d0640 0x7fad180ba250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T08:06:50.738 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:50.735+0000 7fad177fe640 1 --2- 172.21.15.167:0/3978921331 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fad181598d0 0x7fad180ba790 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fad1000c7d0 tx=0x7fad1000cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:50.738 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:50.735+0000 7fad0ffff640 1 -- 172.21.15.167:0/3978921331 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad10003b70 con 0x7fad181598d0 2024-05-11T08:06:50.738 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:50.736+0000 7fad0ffff640 1 -- 172.21.15.167:0/3978921331 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad10003d10 con 0x7fad181598d0 2024-05-11T08:06:50.738 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:50.736+0000 7fad0ffff640 1 -- 172.21.15.167:0/3978921331 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad1000aa70 con 0x7fad181598d0 2024-05-11T08:06:50.738 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:50.736+0000 7fad1d67c640 1 -- 172.21.15.167:0/3978921331 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad180b8360 con 0x7fad181598d0 2024-05-11T08:06:50.738 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:50.736+0000 7fad1d67c640 1 -- 172.21.15.167:0/3978921331 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad180b8920 con 0x7fad181598d0 2024-05-11T08:06:50.738 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:50.737+0000 7fad1d67c640 1 -- 172.21.15.167:0/3978921331 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad18069590 con 0x7fad181598d0 2024-05-11T08:06:50.739 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:50.737+0000 7fad0ffff640 1 -- 172.21.15.167:0/3978921331 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fad1000ac10 con 0x7fad181598d0 2024-05-11T08:06:50.739 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:50.738+0000 7fad0ffff640 1 --2- 172.21.15.167:0/3978921331 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7facf80762d0 0x7facf8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:50.739 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:50.738+0000 7fad16ffd640 1 --2- 172.21.15.167:0/3978921331 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7facf80762d0 0x7facf8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:50.739 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:50.738+0000 7fad0ffff640 1 -- 172.21.15.167:0/3978921331 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6101+0+0 (secure 0 0 0) 0x7fad100976c0 con 0x7fad181598d0 2024-05-11T08:06:50.739 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:50.738+0000 7fad16ffd640 1 --2- 172.21.15.167:0/3978921331 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7facf80762d0 0x7facf8078790 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fad08009af0 tx=0x7fad08008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:50.742 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:50.742+0000 7fad0ffff640 1 -- 172.21.15.167:0/3978921331 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fad10060e90 con 0x7fad181598d0 2024-05-11T08:06:50.961 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:50.961+0000 7fad1d67c640 1 -- 172.21.15.167:0/3978921331 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fad180d4570 con 0x7fad181598d0 2024-05-11T08:06:50.962 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:50.962+0000 7fad0ffff640 1 -- 172.21.15.167:0/3978921331 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5190 (secure 0 0 0) 0x7fad10060790 con 0x7fad181598d0 2024-05-11T08:06:50.962 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T08:06:50.962 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4604,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2674983673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":2674983673},{"type":"v1","addr":"172.21.15.167:6837","nonce":2674983673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4607,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6834/1018548066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1018548066},{"type":"v1","addr":"172.21.15.204:6834","nonce":1018548066}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4729,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3369722249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3369722249},{"type":"v1","addr":"172.21.15.167:6836","nonce":3369722249}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-11T08:06:47.943939+0000","modified":"2024-05-11T08:06:49.955430+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4750},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4750":{"gid":4750,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.204:6835/469864006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6833","nonce":469864006},{"type":"v1","addr":"172.21.15.204:6835","nonce":469864006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4750,"qdb_cluster":[4750]},"id":2}]} 2024-05-11T08:06:50.963 INFO:teuthology.orchestra.run.smithi167.stderr:dumped fsmap epoch 16 2024-05-11T08:06:50.964 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:50.964+0000 7fad1d67c640 1 -- 172.21.15.167:0/3978921331 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7facf80762d0 msgr2=0x7facf8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:50.964 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:50.964+0000 7fad1d67c640 1 --2- 172.21.15.167:0/3978921331 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7facf80762d0 0x7facf8078790 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fad08009af0 tx=0x7fad08008040 comp rx=0 tx=0).stop 2024-05-11T08:06:50.964 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:50.964+0000 7fad1d67c640 1 -- 172.21.15.167:0/3978921331 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fad181598d0 msgr2=0x7fad180ba790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:50.964 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:50.964+0000 7fad1d67c640 1 --2- 172.21.15.167:0/3978921331 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fad181598d0 0x7fad180ba790 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fad1000c7d0 tx=0x7fad1000cc90 comp rx=0 tx=0).stop 2024-05-11T08:06:50.964 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:50.964+0000 7fad1d67c640 1 -- 172.21.15.167:0/3978921331 shutdown_connections 2024-05-11T08:06:50.964 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:50.964+0000 7fad1d67c640 1 --2- 172.21.15.167:0/3978921331 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7facf80762d0 0x7facf8078790 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:50.965 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:50.964+0000 7fad1d67c640 1 --2- 172.21.15.167:0/3978921331 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fad181598d0 0x7fad180ba790 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:50.965 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:50.964+0000 7fad1d67c640 1 --2- 172.21.15.167:0/3978921331 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fad180d0640 0x7fad180ba250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:50.965 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:50.964+0000 7fad1d67c640 1 --2- 172.21.15.167:0/3978921331 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad180cafb0 0x7fad180b9400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:50.965 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:50.964+0000 7fad1d67c640 1 -- 172.21.15.167:0/3978921331 >> 172.21.15.167:0/3978921331 conn(0x7fad180d1ba0 msgr2=0x7fad180d1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:50.965 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:50.964+0000 7fad1d67c640 1 -- 172.21.15.167:0/3978921331 shutdown_connections 2024-05-11T08:06:50.965 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:50.964+0000 7fad1d67c640 1 -- 172.21.15.167:0/3978921331 wait complete. 2024-05-11T08:06:50.975 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-11T08:06:51.075 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.075+0000 7fa49202a640 1 Processor -- start 2024-05-11T08:06:51.076 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.075+0000 7fa49202a640 1 -- start start 2024-05-11T08:06:51.077 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.076+0000 7fa49202a640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fa48c0cd610 0x7fa48c163be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:51.077 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.076+0000 7fa49202a640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fa48c0cd9f0 0x7fa48c160230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:51.077 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.076+0000 7fa49202a640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa48c0b6300 con 0x7fa48c160770 2024-05-11T08:06:51.078 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.076+0000 7fa49202a640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7fa48c0b65e0 con 0x7fa48c0cd610 2024-05-11T08:06:51.078 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.076+0000 7fa49202a640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fa48c163100 con 0x7fa48c0cd9f0 2024-05-11T08:06:51.078 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.076+0000 7fa48b7fe640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fa48c0cd610 0x7fa48c163be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:51.078 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.076+0000 7fa48b7fe640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fa48c0cd610 0x7fa48c163be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:43990/0 (socket says 172.21.15.167:43990) 2024-05-11T08:06:51.078 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.076+0000 7fa48b7fe640 1 -- 172.21.15.167:0/3103620113 learned_addr learned my addr 172.21.15.167:0/3103620113 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:51.078 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.076+0000 7fa48bfff640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7fa48c160770 0x7fa48c0d0ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:55988/0 (socket says 172.21.15.167:55988) 2024-05-11T08:06:51.078 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.077+0000 7fa48affd640 1 --2- 172.21.15.167:0/3103620113 >> v2:172.21.15.204:3300/0 conn(0x7fa48c0cd9f0 0x7fa48c160230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:51.078 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.077+0000 7fa48b7fe640 1 -- 172.21.15.167:0/3103620113 >> v2:172.21.15.204:3300/0 conn(0x7fa48c0cd9f0 msgr2=0x7fa48c160230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:51.078 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.077+0000 7fa48b7fe640 1 --2- 172.21.15.167:0/3103620113 >> v2:172.21.15.204:3300/0 conn(0x7fa48c0cd9f0 0x7fa48c160230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:51.078 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.077+0000 7fa48b7fe640 1 -- 172.21.15.167:0/3103620113 >> v1:172.21.15.167:6789/0 conn(0x7fa48c160770 legacy=0x7fa48c0d0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:51.078 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.077+0000 7fa48b7fe640 1 -- 172.21.15.167:0/3103620113 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa48c1633e0 con 0x7fa48c0cd610 2024-05-11T08:06:51.078 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.077+0000 7fa48a7fc640 1 -- 172.21.15.167:0/3103620113 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2778626369 0 0) 0x7fa48c0b6300 con 0x7fa48c160770 2024-05-11T08:06:51.078 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.077+0000 7fa48affd640 1 --2- 172.21.15.167:0/3103620113 >> v2:172.21.15.204:3300/0 conn(0x7fa48c0cd9f0 0x7fa48c160230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T08:06:51.078 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.077+0000 7fa48b7fe640 1 --2- 172.21.15.167:0/3103620113 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa48c0cd610 0x7fa48c163be0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fa48c0d05d0 tx=0x7fa47c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad31d68391300b5b server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:51.079 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.077+0000 7fa48a7fc640 1 -- 172.21.15.167:0/3103620113 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa47c037070 con 0x7fa48c0cd610 2024-05-11T08:06:51.079 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.077+0000 7fa48a7fc640 1 -- 172.21.15.167:0/3103620113 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa47c032440 con 0x7fa48c0cd610 2024-05-11T08:06:51.079 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.077+0000 7fa48a7fc640 1 -- 172.21.15.167:0/3103620113 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa47c032810 con 0x7fa48c0cd610 2024-05-11T08:06:51.080 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.078+0000 7fa49202a640 1 -- 172.21.15.167:0/3103620113 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa48c0cd610 msgr2=0x7fa48c163be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:51.080 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.078+0000 7fa49202a640 1 --2- 172.21.15.167:0/3103620113 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa48c0cd610 0x7fa48c163be0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fa48c0d05d0 tx=0x7fa47c02a270 comp rx=0 tx=0).stop 2024-05-11T08:06:51.080 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.078+0000 7fa49202a640 1 -- 172.21.15.167:0/3103620113 shutdown_connections 2024-05-11T08:06:51.080 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.078+0000 7fa49202a640 1 --2- 172.21.15.167:0/3103620113 >> v2:172.21.15.204:3300/0 conn(0x7fa48c0cd9f0 0x7fa48c160230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:51.080 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.078+0000 7fa49202a640 1 --2- 172.21.15.167:0/3103620113 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa48c0cd610 0x7fa48c163be0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:51.080 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.078+0000 7fa49202a640 1 -- 172.21.15.167:0/3103620113 >> 172.21.15.167:0/3103620113 conn(0x7fa48c072730 msgr2=0x7fa48c0d2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:51.080 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.078+0000 7fa49202a640 1 -- 172.21.15.167:0/3103620113 shutdown_connections 2024-05-11T08:06:51.080 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.078+0000 7fa49202a640 1 -- 172.21.15.167:0/3103620113 wait complete. 2024-05-11T08:06:51.080 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.079+0000 7fa49202a640 1 Processor -- start 2024-05-11T08:06:51.080 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.079+0000 7fa49202a640 1 -- start start 2024-05-11T08:06:51.082 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.080+0000 7fa49202a640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa48c0cd610 0x7fa48c0ccc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:51.082 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.080+0000 7fa49202a640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa48c0cd9f0 0x7fa48c159d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:51.082 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.080+0000 7fa49202a640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa48c160770 0x7fa48c15cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:51.082 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.080+0000 7fa49202a640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa48c0cde50 con 0x7fa48c0cd610 2024-05-11T08:06:51.082 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.080+0000 7fa49202a640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa48c0d2a80 con 0x7fa48c0cd9f0 2024-05-11T08:06:51.083 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.080+0000 7fa49202a640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fa48c0b5e90 con 0x7fa48c160770 2024-05-11T08:06:51.083 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.080+0000 7fa48b7fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa48c0cd610 0x7fa48c0ccc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:51.083 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.080+0000 7fa48b7fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa48c0cd610 0x7fa48c0ccc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:44002/0 (socket says 172.21.15.167:44002) 2024-05-11T08:06:51.083 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.080+0000 7fa48b7fe640 1 -- 172.21.15.167:0/879411433 learned_addr learned my addr 172.21.15.167:0/879411433 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:51.083 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.081+0000 7fa48affd640 1 --2- 172.21.15.167:0/879411433 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa48c0cd9f0 0x7fa48c159d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:51.083 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.081+0000 7fa48b7fe640 1 -- 172.21.15.167:0/879411433 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa48c160770 msgr2=0x7fa48c15cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:51.083 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.081+0000 7fa48b7fe640 1 --2- 172.21.15.167:0/879411433 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa48c160770 0x7fa48c15cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:51.083 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.081+0000 7fa48b7fe640 1 -- 172.21.15.167:0/879411433 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa48c0cd9f0 msgr2=0x7fa48c159d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:51.083 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.081+0000 7fa48b7fe640 1 --2- 172.21.15.167:0/879411433 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa48c0cd9f0 0x7fa48c159d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:51.083 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.081+0000 7fa48b7fe640 1 -- 172.21.15.167:0/879411433 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa48c15bb60 con 0x7fa48c0cd610 2024-05-11T08:06:51.083 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.081+0000 7fa48b7fe640 1 --2- 172.21.15.167:0/879411433 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa48c0cd610 0x7fa48c0ccc10 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fa47c000c00 tx=0x7fa47c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:51.083 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.081+0000 7fa473fff640 1 -- 172.21.15.167:0/879411433 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa47c041020 con 0x7fa48c0cd610 2024-05-11T08:06:51.083 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.081+0000 7fa473fff640 1 -- 172.21.15.167:0/879411433 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa47c030970 con 0x7fa48c0cd610 2024-05-11T08:06:51.083 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.081+0000 7fa49202a640 1 -- 172.21.15.167:0/879411433 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa48c15be40 con 0x7fa48c0cd610 2024-05-11T08:06:51.083 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.081+0000 7fa473fff640 1 -- 172.21.15.167:0/879411433 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa47c037040 con 0x7fa48c0cd610 2024-05-11T08:06:51.083 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.081+0000 7fa49202a640 1 -- 172.21.15.167:0/879411433 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa48c15c400 con 0x7fa48c0cd610 2024-05-11T08:06:51.084 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.082+0000 7fa49202a640 1 -- 172.21.15.167:0/879411433 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa48c0c9b50 con 0x7fa48c0cd610 2024-05-11T08:06:51.084 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.083+0000 7fa473fff640 1 -- 172.21.15.167:0/879411433 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fa47c044020 con 0x7fa48c0cd610 2024-05-11T08:06:51.084 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.083+0000 7fa473fff640 1 --2- 172.21.15.167:0/879411433 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fa4780762d0 0x7fa478078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:51.084 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.083+0000 7fa473fff640 1 -- 172.21.15.167:0/879411433 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6101+0+0 (secure 0 0 0) 0x7fa47c0b81d0 con 0x7fa48c0cd610 2024-05-11T08:06:51.084 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.083+0000 7fa48affd640 1 --2- 172.21.15.167:0/879411433 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fa4780762d0 0x7fa478078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:51.085 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.084+0000 7fa48affd640 1 --2- 172.21.15.167:0/879411433 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fa4780762d0 0x7fa478078790 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa48c0c9a80 tx=0x7fa480008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:51.087 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.087+0000 7fa473fff640 1 -- 172.21.15.167:0/879411433 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fa48c0c9b50 con 0x7fa48c0cd610 2024-05-11T08:06:51.271 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.271+0000 7fa49202a640 1 -- 172.21.15.167:0/879411433 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa48c069590 con 0x7fa48c0cd610 2024-05-11T08:06:51.273 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.273+0000 7fa473fff640 1 -- 172.21.15.167:0/879411433 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) ==== 74+0+16990 (secure 0 0 0) 0x7fa47c04b030 con 0x7fa48c0cd610 2024-05-11T08:06:51.273 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T08:06:51.273 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":41,"fsid":"2034778d-8ff1-4823-aa44-ded32b518241","created":"2024-05-11T08:05:31.612421+0000","modified":"2024-05-11T08:06:49.590973+0000","last_up_change":"2024-05-11T08:05:42.557382+0000","last_in_change":"2024-05-11T08:05:33.757311+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-11T08:05:43.931263+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-05-11T08:06:45.857156+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-05-11T08:06:46.898591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"587eaced-5ded-406d-884b-30c1caa8686b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":3863307356},{"type":"v1","addr":"172.21.15.167:6806","nonce":3863307356}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":3863307356},{"type":"v1","addr":"172.21.15.167:6810","nonce":3863307356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":3863307356},{"type":"v1","addr":"172.21.15.167:6815","nonce":3863307356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":3863307356},{"type":"v1","addr":"172.21.15.167:6813","nonce":3863307356}]},"public_addr":"172.21.15.167:6806/3863307356","cluster_addr":"172.21.15.167:6810/3863307356","heartbeat_back_addr":"172.21.15.167:6815/3863307356","heartbeat_front_addr":"172.21.15.167:6813/3863307356","state":["exists","up"]},{"osd":1,"uuid":"37c59104-2dd0-4cf9-85ea-56636f6b02a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":1585024048},{"type":"v1","addr":"172.21.15.167:6817","nonce":1585024048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1585024048},{"type":"v1","addr":"172.21.15.167:6819","nonce":1585024048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1585024048},{"type":"v1","addr":"172.21.15.167:6823","nonce":1585024048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":1585024048},{"type":"v1","addr":"172.21.15.167:6821","nonce":1585024048}]},"public_addr":"172.21.15.167:6817/1585024048","cluster_addr":"172.21.15.167:6819/1585024048","heartbeat_back_addr":"172.21.15.167:6823/1585024048","heartbeat_front_addr":"172.21.15.167:6821/1585024048","state":["exists","up"]},{"osd":2,"uuid":"2bb4f7f5-c698-451b-b4bb-78e716921291","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":3242166900},{"type":"v1","addr":"172.21.15.167:6801","nonce":3242166900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":3242166900},{"type":"v1","addr":"172.21.15.167:6803","nonce":3242166900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6809","nonce":3242166900},{"type":"v1","addr":"172.21.15.167:6811","nonce":3242166900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6805","nonce":3242166900},{"type":"v1","addr":"172.21.15.167:6807","nonce":3242166900}]},"public_addr":"172.21.15.167:6801/3242166900","cluster_addr":"172.21.15.167:6803/3242166900","heartbeat_back_addr":"172.21.15.167:6811/3242166900","heartbeat_front_addr":"172.21.15.167:6807/3242166900","state":["exists","up"]},{"osd":3,"uuid":"9dd26759-a338-49d2-8103-45d068b9c000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":453689293},{"type":"v1","addr":"172.21.15.167:6825","nonce":453689293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":453689293},{"type":"v1","addr":"172.21.15.167:6827","nonce":453689293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":453689293},{"type":"v1","addr":"172.21.15.167:6831","nonce":453689293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":453689293},{"type":"v1","addr":"172.21.15.167:6829","nonce":453689293}]},"public_addr":"172.21.15.167:6825/453689293","cluster_addr":"172.21.15.167:6827/453689293","heartbeat_back_addr":"172.21.15.167:6831/453689293","heartbeat_front_addr":"172.21.15.167:6829/453689293","state":["exists","up"]},{"osd":4,"uuid":"cb27250d-2a80-4c7b-8cef-3a7af05c8a16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1311831500},{"type":"v1","addr":"172.21.15.204:6801","nonce":1311831500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1311831500},{"type":"v1","addr":"172.21.15.204:6803","nonce":1311831500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1311831500},{"type":"v1","addr":"172.21.15.204:6807","nonce":1311831500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1311831500},{"type":"v1","addr":"172.21.15.204:6805","nonce":1311831500}]},"public_addr":"172.21.15.204:6801/1311831500","cluster_addr":"172.21.15.204:6803/1311831500","heartbeat_back_addr":"172.21.15.204:6807/1311831500","heartbeat_front_addr":"172.21.15.204:6805/1311831500","state":["exists","up"]},{"osd":5,"uuid":"92998541-ffec-430b-ab76-8c3cb7c2752d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2921847521},{"type":"v1","addr":"172.21.15.204:6825","nonce":2921847521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2921847521},{"type":"v1","addr":"172.21.15.204:6827","nonce":2921847521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2921847521},{"type":"v1","addr":"172.21.15.204:6831","nonce":2921847521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2921847521},{"type":"v1","addr":"172.21.15.204:6829","nonce":2921847521}]},"public_addr":"172.21.15.204:6825/2921847521","cluster_addr":"172.21.15.204:6827/2921847521","heartbeat_back_addr":"172.21.15.204:6831/2921847521","heartbeat_front_addr":"172.21.15.204:6829/2921847521","state":["exists","up"]},{"osd":6,"uuid":"9543479c-2ec5-42a8-8933-707bfed2e048","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":2889851077},{"type":"v1","addr":"172.21.15.204:6817","nonce":2889851077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":2889851077},{"type":"v1","addr":"172.21.15.204:6819","nonce":2889851077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":2889851077},{"type":"v1","addr":"172.21.15.204:6823","nonce":2889851077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":2889851077},{"type":"v1","addr":"172.21.15.204:6821","nonce":2889851077}]},"public_addr":"172.21.15.204:6817/2889851077","cluster_addr":"172.21.15.204:6819/2889851077","heartbeat_back_addr":"172.21.15.204:6823/2889851077","heartbeat_front_addr":"172.21.15.204:6821/2889851077","state":["exists","up"]},{"osd":7,"uuid":"c45a5647-8358-409e-af91-c05dda65f0b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":1949534184},{"type":"v1","addr":"172.21.15.204:6809","nonce":1949534184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":1949534184},{"type":"v1","addr":"172.21.15.204:6811","nonce":1949534184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1949534184},{"type":"v1","addr":"172.21.15.204:6815","nonce":1949534184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1949534184},{"type":"v1","addr":"172.21.15.204:6813","nonce":1949534184}]},"public_addr":"172.21.15.204:6809/1949534184","cluster_addr":"172.21.15.204:6811/1949534184","heartbeat_back_addr":"172.21.15.204:6815/1949534184","heartbeat_front_addr":"172.21.15.204:6813/1949534184","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.952253+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.940689+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.860682+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.929350+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.901627+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.957451+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.987249+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.951197+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-11T08:06:51.275 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.275+0000 7fa49202a640 1 -- 172.21.15.167:0/879411433 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fa4780762d0 msgr2=0x7fa478078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:51.275 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.275+0000 7fa49202a640 1 --2- 172.21.15.167:0/879411433 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fa4780762d0 0x7fa478078790 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa48c0c9a80 tx=0x7fa480008040 comp rx=0 tx=0).stop 2024-05-11T08:06:51.275 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.275+0000 7fa49202a640 1 -- 172.21.15.167:0/879411433 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa48c0cd610 msgr2=0x7fa48c0ccc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:51.275 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.275+0000 7fa49202a640 1 --2- 172.21.15.167:0/879411433 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa48c0cd610 0x7fa48c0ccc10 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fa47c000c00 tx=0x7fa47c0046c0 comp rx=0 tx=0).stop 2024-05-11T08:06:51.275 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.275+0000 7fa49202a640 1 -- 172.21.15.167:0/879411433 shutdown_connections 2024-05-11T08:06:51.276 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.275+0000 7fa49202a640 1 --2- 172.21.15.167:0/879411433 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fa4780762d0 0x7fa478078790 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:51.276 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.275+0000 7fa49202a640 1 --2- 172.21.15.167:0/879411433 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa48c160770 0x7fa48c15cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:51.276 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.275+0000 7fa49202a640 1 --2- 172.21.15.167:0/879411433 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa48c0cd9f0 0x7fa48c159d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:51.276 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.275+0000 7fa49202a640 1 --2- 172.21.15.167:0/879411433 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa48c0cd610 0x7fa48c0ccc10 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:51.276 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.275+0000 7fa49202a640 1 -- 172.21.15.167:0/879411433 >> 172.21.15.167:0/879411433 conn(0x7fa48c072730 msgr2=0x7fa48c15d600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:51.276 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.275+0000 7fa49202a640 1 -- 172.21.15.167:0/879411433 shutdown_connections 2024-05-11T08:06:51.276 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.275+0000 7fa49202a640 1 -- 172.21.15.167:0/879411433 wait complete. 2024-05-11T08:06:51.286 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-11T08:06:51.286 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-11T08:06:51.387 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.387+0000 7f52e9546640 1 Processor -- start 2024-05-11T08:06:51.388 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.387+0000 7f52e9546640 1 -- start start 2024-05-11T08:06:51.389 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.388+0000 7f52e9546640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f52e4148900 0x7f52e40710f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:51.389 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.388+0000 7f52e9546640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f52e4071630 0x7f52e40d0500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:51.389 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.388+0000 7f52e9546640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52e40b4f50 con 0x7f52e40d0a40 2024-05-11T08:06:51.389 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.388+0000 7f52e9546640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f52e4071a70 con 0x7f52e4071630 2024-05-11T08:06:51.389 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.388+0000 7f52e9546640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f52e40b5230 con 0x7f52e4148900 2024-05-11T08:06:51.389 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.388+0000 7f52e27fc640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f52e4071630 0x7f52e40d0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:51.389 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.388+0000 7f52e27fc640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f52e4071630 0x7f52e40d0500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:44004/0 (socket says 172.21.15.167:44004) 2024-05-11T08:06:51.389 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.388+0000 7f52e27fc640 1 -- 172.21.15.167:0/2368137440 learned_addr learned my addr 172.21.15.167:0/2368137440 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:51.389 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.389+0000 7f52e27fc640 1 -- 172.21.15.167:0/2368137440 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f52e4148900 msgr2=0x7f52e40710f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:51.389 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.389+0000 7f52e2ffd640 1 --2- 172.21.15.167:0/2368137440 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f52e4148900 0x7f52e40710f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:51.389 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.389+0000 7f52e27fc640 1 --2- 172.21.15.167:0/2368137440 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f52e4148900 0x7f52e40710f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:51.389 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.389+0000 7f52e27fc640 1 -- 172.21.15.167:0/2368137440 >> v1:172.21.15.204:6789/0 conn(0x7f52e40d0a40 legacy=0x7f52e415ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:51.389 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.389+0000 7f52e27fc640 1 -- 172.21.15.167:0/2368137440 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52e40b5510 con 0x7f52e4071630 2024-05-11T08:06:51.391 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.389+0000 7f52e27fc640 1 --2- 172.21.15.167:0/2368137440 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f52e4071630 0x7f52e40d0500 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f52d8009870 tx=0x7f52d802a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b96100bc396d7755 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:51.391 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.389+0000 7f52e1ffb640 1 -- 172.21.15.167:0/2368137440 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52d8037070 con 0x7f52e4071630 2024-05-11T08:06:51.391 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.389+0000 7f52e1ffb640 1 -- 172.21.15.167:0/2368137440 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52d8032440 con 0x7f52e4071630 2024-05-11T08:06:51.391 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.389+0000 7f52e1ffb640 1 -- 172.21.15.167:0/2368137440 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52d8032810 con 0x7f52e4071630 2024-05-11T08:06:51.391 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.390+0000 7f52e9546640 1 -- 172.21.15.167:0/2368137440 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f52e4071630 msgr2=0x7f52e40d0500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:51.391 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.390+0000 7f52e9546640 1 --2- 172.21.15.167:0/2368137440 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f52e4071630 0x7f52e40d0500 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f52d8009870 tx=0x7f52d802a1c0 comp rx=0 tx=0).stop 2024-05-11T08:06:51.391 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.390+0000 7f52e9546640 1 -- 172.21.15.167:0/2368137440 shutdown_connections 2024-05-11T08:06:51.391 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.390+0000 7f52e9546640 1 --2- 172.21.15.167:0/2368137440 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f52e4071630 0x7f52e40d0500 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:51.391 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.390+0000 7f52e9546640 1 --2- 172.21.15.167:0/2368137440 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f52e4148900 0x7f52e40710f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:51.391 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.390+0000 7f52e9546640 1 -- 172.21.15.167:0/2368137440 >> 172.21.15.167:0/2368137440 conn(0x7f52e40d1e40 msgr2=0x7f52e40d2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:51.391 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.390+0000 7f52e9546640 1 -- 172.21.15.167:0/2368137440 shutdown_connections 2024-05-11T08:06:51.391 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.390+0000 7f52e9546640 1 -- 172.21.15.167:0/2368137440 wait complete. 2024-05-11T08:06:51.393 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.391+0000 7f52e9546640 1 Processor -- start 2024-05-11T08:06:51.394 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.391+0000 7f52e9546640 1 -- start start 2024-05-11T08:06:51.394 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.392+0000 7f52e9546640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f52e4071630 0x7f52e4159950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:51.394 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.392+0000 7f52e9546640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f52e40d0a40 0x7f52e413c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:51.394 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.392+0000 7f52e9546640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f52e4148900 0x7f52e415fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:51.394 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.392+0000 7f52e9546640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f52e4148ce0 con 0x7f52e40d0a40 2024-05-11T08:06:51.394 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.392+0000 7f52e9546640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f52e40b5ad0 con 0x7f52e4071630 2024-05-11T08:06:51.394 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.392+0000 7f52e9546640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f52e415f900 con 0x7f52e4148900 2024-05-11T08:06:51.394 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.392+0000 7f52e27fc640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f52e40d0a40 0x7f52e413c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:51.394 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.392+0000 7f52e27fc640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f52e40d0a40 0x7f52e413c180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:44016/0 (socket says 172.21.15.167:44016) 2024-05-11T08:06:51.394 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.392+0000 7f52e27fc640 1 -- 172.21.15.167:0/2236953464 learned_addr learned my addr 172.21.15.167:0/2236953464 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:51.394 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.392+0000 7f52e2ffd640 1 --2- 172.21.15.167:0/2236953464 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f52e4071630 0x7f52e4159950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:51.395 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.392+0000 7f52e37fe640 1 --2- 172.21.15.167:0/2236953464 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f52e4148900 0x7f52e415fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:51.395 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.393+0000 7f52e27fc640 1 -- 172.21.15.167:0/2236953464 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f52e4148900 msgr2=0x7f52e415fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:51.395 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.393+0000 7f52e27fc640 1 --2- 172.21.15.167:0/2236953464 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f52e4148900 0x7f52e415fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:51.395 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.393+0000 7f52e27fc640 1 -- 172.21.15.167:0/2236953464 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f52e4071630 msgr2=0x7f52e4159950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:51.395 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.393+0000 7f52e27fc640 1 --2- 172.21.15.167:0/2236953464 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f52e4071630 0x7f52e4159950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:51.395 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.393+0000 7f52e27fc640 1 -- 172.21.15.167:0/2236953464 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52e413c9d0 con 0x7f52e40d0a40 2024-05-11T08:06:51.395 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.393+0000 7f52e37fe640 1 --2- 172.21.15.167:0/2236953464 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f52e4148900 0x7f52e415fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T08:06:51.395 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.393+0000 7f52e27fc640 1 --2- 172.21.15.167:0/2236953464 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f52e40d0a40 0x7f52e413c180 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f52d8009870 tx=0x7f52d80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:51.395 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.393+0000 7f52cb7fe640 1 -- 172.21.15.167:0/2236953464 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52d8041020 con 0x7f52e40d0a40 2024-05-11T08:06:51.395 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.393+0000 7f52cb7fe640 1 -- 172.21.15.167:0/2236953464 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52d8030970 con 0x7f52e40d0a40 2024-05-11T08:06:51.395 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.393+0000 7f52e9546640 1 -- 172.21.15.167:0/2236953464 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52e413ccb0 con 0x7f52e40d0a40 2024-05-11T08:06:51.395 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.393+0000 7f52cb7fe640 1 -- 172.21.15.167:0/2236953464 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52d8037040 con 0x7f52e40d0a40 2024-05-11T08:06:51.395 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.393+0000 7f52e9546640 1 -- 172.21.15.167:0/2236953464 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52e413d270 con 0x7f52e40d0a40 2024-05-11T08:06:51.396 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.394+0000 7f52e9546640 1 -- 172.21.15.167:0/2236953464 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52e4069bc0 con 0x7f52e40d0a40 2024-05-11T08:06:51.396 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.395+0000 7f52cb7fe640 1 -- 172.21.15.167:0/2236953464 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f52d8044020 con 0x7f52e40d0a40 2024-05-11T08:06:51.396 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.395+0000 7f52cb7fe640 1 --2- 172.21.15.167:0/2236953464 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f52d00762d0 0x7f52d0078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:51.396 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.395+0000 7f52cb7fe640 1 -- 172.21.15.167:0/2236953464 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6101+0+0 (secure 0 0 0) 0x7f52d80b81c0 con 0x7f52e40d0a40 2024-05-11T08:06:51.396 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.395+0000 7f52e2ffd640 1 --2- 172.21.15.167:0/2236953464 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f52d00762d0 0x7f52d0078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:51.396 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.396+0000 7f52e2ffd640 1 --2- 172.21.15.167:0/2236953464 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f52d00762d0 0x7f52d0078790 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f52d400a970 tx=0x7f52d4008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:51.399 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.400+0000 7f52cb7fe640 1 -- 172.21.15.167:0/2236953464 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f52d804b030 con 0x7f52e40d0a40 2024-05-11T08:06:51.578 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.578+0000 7f52e9546640 1 -- 172.21.15.167:0/2236953464 --> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f52e40b3760 con 0x7f52d00762d0 2024-05-11T08:06:51.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.584+0000 7f52cb7fe640 1 -- 172.21.15.167:0/2236953464 <== mgr.4113 v2:172.21.15.167:6832/253569496 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+245401 (secure 0 0 0) 0x7f52e40b3760 con 0x7f52d00762d0 2024-05-11T08:06:51.587 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T08:06:51.588 INFO:teuthology.orchestra.run.smithi167.stderr:dumped all 2024-05-11T08:06:51.589 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.587+0000 7f52e9546640 1 -- 172.21.15.167:0/2236953464 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f52d00762d0 msgr2=0x7f52d0078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:51.589 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.587+0000 7f52e9546640 1 --2- 172.21.15.167:0/2236953464 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f52d00762d0 0x7f52d0078790 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f52d400a970 tx=0x7f52d4008040 comp rx=0 tx=0).stop 2024-05-11T08:06:51.589 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.587+0000 7f52e9546640 1 -- 172.21.15.167:0/2236953464 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f52e40d0a40 msgr2=0x7f52e413c180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:51.589 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.587+0000 7f52e9546640 1 --2- 172.21.15.167:0/2236953464 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f52e40d0a40 0x7f52e413c180 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f52d8009870 tx=0x7f52d80046c0 comp rx=0 tx=0).stop 2024-05-11T08:06:51.590 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.587+0000 7f52e9546640 1 -- 172.21.15.167:0/2236953464 shutdown_connections 2024-05-11T08:06:51.590 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.587+0000 7f52e9546640 1 --2- 172.21.15.167:0/2236953464 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f52d00762d0 0x7f52d0078790 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:51.590 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.587+0000 7f52e9546640 1 --2- 172.21.15.167:0/2236953464 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f52e4148900 0x7f52e415fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:51.590 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.587+0000 7f52e9546640 1 --2- 172.21.15.167:0/2236953464 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f52e40d0a40 0x7f52e413c180 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:51.590 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.587+0000 7f52e9546640 1 --2- 172.21.15.167:0/2236953464 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f52e4071630 0x7f52e4159950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:51.590 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.587+0000 7f52e9546640 1 -- 172.21.15.167:0/2236953464 >> 172.21.15.167:0/2236953464 conn(0x7f52e40d1e40 msgr2=0x7f52e4145ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:51.590 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.588+0000 7f52e9546640 1 -- 172.21.15.167:0/2236953464 shutdown_connections 2024-05-11T08:06:51.590 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.588+0000 7f52e9546640 1 -- 172.21.15.167:0/2236953464 wait complete. 2024-05-11T08:06:51.599 INFO:teuthology.orchestra.run.smithi167.stdout:{"pg_ready":true,"pg_map":{"version":69,"stamp":"2024-05-11T08:06:49.927791+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":172,"acting":172,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":23,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":220692,"kb_used_data":4532,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754028,"statfs":{"total":773094113280,"available":772868124672,"internally_reserved":0,"allocated":4640768,"data_stored":2626285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000554"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.580160+0000","last_change":"2024-05-11T08:06:47.580160+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.580160+0000","last_undegraded":"2024-05-11T08:06:47.580160+0000","last_fullsized":"2024-05-11T08:06:47.580160+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.580981+0000","last_change":"2024-05-11T08:06:47.580981+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.580981+0000","last_undegraded":"2024-05-11T08:06:47.580981+0000","last_fullsized":"2024-05-11T08:06:47.580981+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.587939+0000","last_change":"2024-05-11T08:06:47.587939+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.587939+0000","last_undegraded":"2024-05-11T08:06:47.587939+0000","last_fullsized":"2024-05-11T08:06:47.587939+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T08:06:47.573468+0000","last_change":"2024-05-11T08:06:47.573468+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.573468+0000","last_undegraded":"2024-05-11T08:06:47.573468+0000","last_fullsized":"2024-05-11T08:06:47.573468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T08:06:47.573468+0000","last_change":"2024-05-11T08:06:47.573468+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.573468+0000","last_undegraded":"2024-05-11T08:06:47.573468+0000","last_fullsized":"2024-05-11T08:06:47.573468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.586507+0000","last_change":"2024-05-11T08:06:47.586507+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.586507+0000","last_undegraded":"2024-05-11T08:06:47.586507+0000","last_fullsized":"2024-05-11T08:06:47.586507+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.580887+0000","last_change":"2024-05-11T08:06:47.580887+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.580887+0000","last_undegraded":"2024-05-11T08:06:47.580887+0000","last_fullsized":"2024-05-11T08:06:47.580887+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.579778+0000","last_change":"2024-05-11T08:06:47.579778+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.579778+0000","last_undegraded":"2024-05-11T08:06:47.579778+0000","last_fullsized":"2024-05-11T08:06:47.579778+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.585551+0000","last_change":"2024-05-11T08:06:47.585551+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.585551+0000","last_undegraded":"2024-05-11T08:06:47.585551+0000","last_fullsized":"2024-05-11T08:06:47.585551+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.579247+0000","last_change":"2024-05-11T08:06:47.579247+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.579247+0000","last_undegraded":"2024-05-11T08:06:47.579247+0000","last_fullsized":"2024-05-11T08:06:47.579247+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.580547+0000","last_change":"2024-05-11T08:06:47.580547+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.580547+0000","last_undegraded":"2024-05-11T08:06:47.580547+0000","last_fullsized":"2024-05-11T08:06:47.580547+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.584544+0000","last_change":"2024-05-11T08:06:47.584544+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.584544+0000","last_undegraded":"2024-05-11T08:06:47.584544+0000","last_fullsized":"2024-05-11T08:06:47.584544+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.580055+0000","last_change":"2024-05-11T08:06:47.580055+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.580055+0000","last_undegraded":"2024-05-11T08:06:47.580055+0000","last_fullsized":"2024-05-11T08:06:47.580055+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.585515+0000","last_change":"2024-05-11T08:06:47.585515+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.585515+0000","last_undegraded":"2024-05-11T08:06:47.585515+0000","last_fullsized":"2024-05-11T08:06:47.585515+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.579888+0000","last_change":"2024-05-11T08:06:47.579888+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.579888+0000","last_undegraded":"2024-05-11T08:06:47.579888+0000","last_fullsized":"2024-05-11T08:06:47.579888+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.585758+0000","last_change":"2024-05-11T08:06:47.585758+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.585758+0000","last_undegraded":"2024-05-11T08:06:47.585758+0000","last_fullsized":"2024-05-11T08:06:47.585758+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.587418+0000","last_change":"2024-05-11T08:06:47.587418+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.587418+0000","last_undegraded":"2024-05-11T08:06:47.587418+0000","last_fullsized":"2024-05-11T08:06:47.587418+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.588413+0000","last_change":"2024-05-11T08:06:47.588413+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.588413+0000","last_undegraded":"2024-05-11T08:06:47.588413+0000","last_fullsized":"2024-05-11T08:06:47.588413+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T08:06:47.573468+0000","last_change":"2024-05-11T08:06:47.573468+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.573468+0000","last_undegraded":"2024-05-11T08:06:47.573468+0000","last_fullsized":"2024-05-11T08:06:47.573468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.587907+0000","last_change":"2024-05-11T08:06:47.587907+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.587907+0000","last_undegraded":"2024-05-11T08:06:47.587907+0000","last_fullsized":"2024-05-11T08:06:47.587907+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.579095+0000","last_change":"2024-05-11T08:06:47.579095+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.579095+0000","last_undegraded":"2024-05-11T08:06:47.579095+0000","last_fullsized":"2024-05-11T08:06:47.579095+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.583273+0000","last_change":"2024-05-11T08:06:47.583273+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.583273+0000","last_undegraded":"2024-05-11T08:06:47.583273+0000","last_fullsized":"2024-05-11T08:06:47.583273+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.583780+0000","last_change":"2024-05-11T08:06:47.583780+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.583780+0000","last_undegraded":"2024-05-11T08:06:47.583780+0000","last_fullsized":"2024-05-11T08:06:47.583780+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T08:06:47.573468+0000","last_change":"2024-05-11T08:06:47.573468+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.573468+0000","last_undegraded":"2024-05-11T08:06:47.573468+0000","last_fullsized":"2024-05-11T08:06:47.573468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.589116+0000","last_change":"2024-05-11T08:06:47.589116+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.589116+0000","last_undegraded":"2024-05-11T08:06:47.589116+0000","last_fullsized":"2024-05-11T08:06:47.589116+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.586184+0000","last_change":"2024-05-11T08:06:47.586184+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.586184+0000","last_undegraded":"2024-05-11T08:06:47.586184+0000","last_fullsized":"2024-05-11T08:06:47.586184+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.587324+0000","last_change":"2024-05-11T08:06:47.587324+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.587324+0000","last_undegraded":"2024-05-11T08:06:47.587324+0000","last_fullsized":"2024-05-11T08:06:47.587324+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.578757+0000","last_change":"2024-05-11T08:06:47.578757+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.578757+0000","last_undegraded":"2024-05-11T08:06:47.578757+0000","last_fullsized":"2024-05-11T08:06:47.578757+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.588841+0000","last_change":"2024-05-11T08:06:47.588841+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.588841+0000","last_undegraded":"2024-05-11T08:06:47.588841+0000","last_fullsized":"2024-05-11T08:06:47.588841+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.585718+0000","last_change":"2024-05-11T08:06:47.585718+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.585718+0000","last_undegraded":"2024-05-11T08:06:47.585718+0000","last_fullsized":"2024-05-11T08:06:47.585718+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.583214+0000","last_change":"2024-05-11T08:06:47.583214+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.583214+0000","last_undegraded":"2024-05-11T08:06:47.583214+0000","last_fullsized":"2024-05-11T08:06:47.583214+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.587217+0000","last_change":"2024-05-11T08:06:47.587217+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.587217+0000","last_undegraded":"2024-05-11T08:06:47.587217+0000","last_fullsized":"2024-05-11T08:06:47.587217+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.584847+0000","last_change":"2024-05-11T08:06:47.584847+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.584847+0000","last_undegraded":"2024-05-11T08:06:47.584847+0000","last_fullsized":"2024-05-11T08:06:47.584847+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T08:06:47.594702+0000","last_change":"2024-05-11T08:06:47.591074+0000","last_active":"2024-05-11T08:06:47.594702+0000","last_peered":"2024-05-11T08:06:47.594702+0000","last_clean":"2024-05-11T08:06:47.594702+0000","last_became_active":"2024-05-11T08:06:47.590747+0000","last_became_peered":"2024-05-11T08:06:47.590747+0000","last_unstale":"2024-05-11T08:06:47.594702+0000","last_undegraded":"2024-05-11T08:06:47.594702+0000","last_fullsized":"2024-05-11T08:06:47.594702+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:11:53.548277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.585679+0000","last_change":"2024-05-11T08:06:47.585679+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.585679+0000","last_undegraded":"2024-05-11T08:06:47.585679+0000","last_fullsized":"2024-05-11T08:06:47.585679+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T08:06:47.601797+0000","last_change":"2024-05-11T08:06:47.597547+0000","last_active":"2024-05-11T08:06:47.601797+0000","last_peered":"2024-05-11T08:06:47.601797+0000","last_clean":"2024-05-11T08:06:47.601797+0000","last_became_active":"2024-05-11T08:06:47.597260+0000","last_became_peered":"2024-05-11T08:06:47.597260+0000","last_unstale":"2024-05-11T08:06:47.601797+0000","last_undegraded":"2024-05-11T08:06:47.601797+0000","last_fullsized":"2024-05-11T08:06:47.601797+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:08:22.364810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.584309+0000","last_change":"2024-05-11T08:06:47.584309+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.584309+0000","last_undegraded":"2024-05-11T08:06:47.584309+0000","last_fullsized":"2024-05-11T08:06:47.584309+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T08:06:47.601604+0000","last_change":"2024-05-11T08:06:47.597745+0000","last_active":"2024-05-11T08:06:47.601604+0000","last_peered":"2024-05-11T08:06:47.601604+0000","last_clean":"2024-05-11T08:06:47.601604+0000","last_became_active":"2024-05-11T08:06:47.597472+0000","last_became_peered":"2024-05-11T08:06:47.597472+0000","last_unstale":"2024-05-11T08:06:47.601604+0000","last_undegraded":"2024-05-11T08:06:47.601604+0000","last_fullsized":"2024-05-11T08:06:47.601604+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:56:21.008947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.580834+0000","last_change":"2024-05-11T08:06:47.580834+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.580834+0000","last_undegraded":"2024-05-11T08:06:47.580834+0000","last_fullsized":"2024-05-11T08:06:47.580834+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T08:06:47.596539+0000","last_change":"2024-05-11T08:06:47.594580+0000","last_active":"2024-05-11T08:06:47.596539+0000","last_peered":"2024-05-11T08:06:47.596539+0000","last_clean":"2024-05-11T08:06:47.596539+0000","last_became_active":"2024-05-11T08:06:47.594315+0000","last_became_peered":"2024-05-11T08:06:47.594315+0000","last_unstale":"2024-05-11T08:06:47.596539+0000","last_undegraded":"2024-05-11T08:06:47.596539+0000","last_fullsized":"2024-05-11T08:06:47.596539+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:56:21.008947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.586542+0000","last_change":"2024-05-11T08:06:47.586542+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.586542+0000","last_undegraded":"2024-05-11T08:06:47.586542+0000","last_fullsized":"2024-05-11T08:06:47.586542+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T08:06:47.602415+0000","last_change":"2024-05-11T08:06:47.600311+0000","last_active":"2024-05-11T08:06:47.602415+0000","last_peered":"2024-05-11T08:06:47.602415+0000","last_clean":"2024-05-11T08:06:47.602415+0000","last_became_active":"2024-05-11T08:06:47.600072+0000","last_became_peered":"2024-05-11T08:06:47.600072+0000","last_unstale":"2024-05-11T08:06:47.602415+0000","last_undegraded":"2024-05-11T08:06:47.602415+0000","last_fullsized":"2024-05-11T08:06:47.602415+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:13:57.487739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.578355+0000","last_change":"2024-05-11T08:06:47.578355+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.578355+0000","last_undegraded":"2024-05-11T08:06:47.578355+0000","last_fullsized":"2024-05-11T08:06:47.578355+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T08:06:47.602942+0000","last_change":"2024-05-11T08:06:47.598857+0000","last_active":"2024-05-11T08:06:47.602942+0000","last_peered":"2024-05-11T08:06:47.602942+0000","last_clean":"2024-05-11T08:06:47.602942+0000","last_became_active":"2024-05-11T08:06:47.598102+0000","last_became_peered":"2024-05-11T08:06:47.598102+0000","last_unstale":"2024-05-11T08:06:47.602942+0000","last_undegraded":"2024-05-11T08:06:47.602942+0000","last_fullsized":"2024-05-11T08:06:47.602942+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:13:57.487739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.583760+0000","last_change":"2024-05-11T08:06:47.583760+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.583760+0000","last_undegraded":"2024-05-11T08:06:47.583760+0000","last_fullsized":"2024-05-11T08:06:47.583760+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T08:06:47.599146+0000","last_change":"2024-05-11T08:06:47.595791+0000","last_active":"2024-05-11T08:06:47.599146+0000","last_peered":"2024-05-11T08:06:47.599146+0000","last_clean":"2024-05-11T08:06:47.599146+0000","last_became_active":"2024-05-11T08:06:47.595551+0000","last_became_peered":"2024-05-11T08:06:47.595551+0000","last_unstale":"2024-05-11T08:06:47.599146+0000","last_undegraded":"2024-05-11T08:06:47.599146+0000","last_fullsized":"2024-05-11T08:06:47.599146+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:00:29.291380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.581488+0000","last_change":"2024-05-11T08:06:47.581488+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.581488+0000","last_undegraded":"2024-05-11T08:06:47.581488+0000","last_fullsized":"2024-05-11T08:06:47.581488+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T08:06:47.596271+0000","last_change":"2024-05-11T08:06:47.590670+0000","last_active":"2024-05-11T08:06:47.596271+0000","last_peered":"2024-05-11T08:06:47.596271+0000","last_clean":"2024-05-11T08:06:47.596271+0000","last_became_active":"2024-05-11T08:06:47.590380+0000","last_became_peered":"2024-05-11T08:06:47.590380+0000","last_unstale":"2024-05-11T08:06:47.596271+0000","last_undegraded":"2024-05-11T08:06:47.596271+0000","last_fullsized":"2024-05-11T08:06:47.596271+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:00:29.291380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T08:06:47.573468+0000","last_change":"2024-05-11T08:06:47.573468+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.573468+0000","last_undegraded":"2024-05-11T08:06:47.573468+0000","last_fullsized":"2024-05-11T08:06:47.573468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T08:06:47.593461+0000","last_change":"2024-05-11T08:06:47.590630+0000","last_active":"2024-05-11T08:06:47.593461+0000","last_peered":"2024-05-11T08:06:47.593461+0000","last_clean":"2024-05-11T08:06:47.593461+0000","last_became_active":"2024-05-11T08:06:47.590361+0000","last_became_peered":"2024-05-11T08:06:47.590361+0000","last_unstale":"2024-05-11T08:06:47.593461+0000","last_undegraded":"2024-05-11T08:06:47.593461+0000","last_fullsized":"2024-05-11T08:06:47.593461+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:20:42.258886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.584566+0000","last_change":"2024-05-11T08:06:47.584566+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.584566+0000","last_undegraded":"2024-05-11T08:06:47.584566+0000","last_fullsized":"2024-05-11T08:06:47.584566+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T08:06:47.595167+0000","last_change":"2024-05-11T08:06:47.587418+0000","last_active":"2024-05-11T08:06:47.595167+0000","last_peered":"2024-05-11T08:06:47.595167+0000","last_clean":"2024-05-11T08:06:47.595167+0000","last_became_active":"2024-05-11T08:06:47.586715+0000","last_became_peered":"2024-05-11T08:06:47.586715+0000","last_unstale":"2024-05-11T08:06:47.595167+0000","last_undegraded":"2024-05-11T08:06:47.595167+0000","last_fullsized":"2024-05-11T08:06:47.595167+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:56:21.008947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.584770+0000","last_change":"2024-05-11T08:06:47.584770+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.584770+0000","last_undegraded":"2024-05-11T08:06:47.584770+0000","last_fullsized":"2024-05-11T08:06:47.584770+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T08:06:47.597011+0000","last_change":"2024-05-11T08:06:47.594319+0000","last_active":"2024-05-11T08:06:47.597011+0000","last_peered":"2024-05-11T08:06:47.597011+0000","last_clean":"2024-05-11T08:06:47.597011+0000","last_became_active":"2024-05-11T08:06:47.593840+0000","last_became_peered":"2024-05-11T08:06:47.593840+0000","last_unstale":"2024-05-11T08:06:47.597011+0000","last_undegraded":"2024-05-11T08:06:47.597011+0000","last_fullsized":"2024-05-11T08:06:47.597011+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:12:27.297983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.584560+0000","last_change":"2024-05-11T08:06:47.584560+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.584560+0000","last_undegraded":"2024-05-11T08:06:47.584560+0000","last_fullsized":"2024-05-11T08:06:47.584560+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T08:06:47.596789+0000","last_change":"2024-05-11T08:06:47.592968+0000","last_active":"2024-05-11T08:06:47.596789+0000","last_peered":"2024-05-11T08:06:47.596789+0000","last_clean":"2024-05-11T08:06:47.596789+0000","last_became_active":"2024-05-11T08:06:47.592672+0000","last_became_peered":"2024-05-11T08:06:47.592672+0000","last_unstale":"2024-05-11T08:06:47.596789+0000","last_undegraded":"2024-05-11T08:06:47.596789+0000","last_fullsized":"2024-05-11T08:06:47.596789+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:46:05.539312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.585804+0000","last_change":"2024-05-11T08:06:47.585804+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.585804+0000","last_undegraded":"2024-05-11T08:06:47.585804+0000","last_fullsized":"2024-05-11T08:06:47.585804+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T08:06:47.604642+0000","last_change":"2024-05-11T08:06:47.600533+0000","last_active":"2024-05-11T08:06:47.604642+0000","last_peered":"2024-05-11T08:06:47.604642+0000","last_clean":"2024-05-11T08:06:47.604642+0000","last_became_active":"2024-05-11T08:06:47.600226+0000","last_became_peered":"2024-05-11T08:06:47.600226+0000","last_unstale":"2024-05-11T08:06:47.604642+0000","last_undegraded":"2024-05-11T08:06:47.604642+0000","last_fullsized":"2024-05-11T08:06:47.604642+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:49:09.098705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.579895+0000","last_change":"2024-05-11T08:06:47.579895+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.579895+0000","last_undegraded":"2024-05-11T08:06:47.579895+0000","last_fullsized":"2024-05-11T08:06:47.579895+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T08:06:47.601569+0000","last_change":"2024-05-11T08:06:47.597276+0000","last_active":"2024-05-11T08:06:47.601569+0000","last_peered":"2024-05-11T08:06:47.601569+0000","last_clean":"2024-05-11T08:06:47.601569+0000","last_became_active":"2024-05-11T08:06:47.596986+0000","last_became_peered":"2024-05-11T08:06:47.596986+0000","last_unstale":"2024-05-11T08:06:47.601569+0000","last_undegraded":"2024-05-11T08:06:47.601569+0000","last_fullsized":"2024-05-11T08:06:47.601569+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:08:22.364810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.584152+0000","last_change":"2024-05-11T08:06:47.584152+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.584152+0000","last_undegraded":"2024-05-11T08:06:47.584152+0000","last_fullsized":"2024-05-11T08:06:47.584152+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T08:06:47.594681+0000","last_change":"2024-05-11T08:06:47.591052+0000","last_active":"2024-05-11T08:06:47.594681+0000","last_peered":"2024-05-11T08:06:47.594681+0000","last_clean":"2024-05-11T08:06:47.594681+0000","last_became_active":"2024-05-11T08:06:47.590726+0000","last_became_peered":"2024-05-11T08:06:47.590726+0000","last_unstale":"2024-05-11T08:06:47.594681+0000","last_undegraded":"2024-05-11T08:06:47.594681+0000","last_fullsized":"2024-05-11T08:06:47.594681+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:30:44.079909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.580872+0000","last_change":"2024-05-11T08:06:47.580872+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.580872+0000","last_undegraded":"2024-05-11T08:06:47.580872+0000","last_fullsized":"2024-05-11T08:06:47.580872+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T08:06:47.598143+0000","last_change":"2024-05-11T08:06:47.597077+0000","last_active":"2024-05-11T08:06:47.598143+0000","last_peered":"2024-05-11T08:06:47.598143+0000","last_clean":"2024-05-11T08:06:47.598143+0000","last_became_active":"2024-05-11T08:06:47.596753+0000","last_became_peered":"2024-05-11T08:06:47.596753+0000","last_unstale":"2024-05-11T08:06:47.598143+0000","last_undegraded":"2024-05-11T08:06:47.598143+0000","last_fullsized":"2024-05-11T08:06:47.598143+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:19:14.446185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.580505+0000","last_change":"2024-05-11T08:06:47.580505+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.580505+0000","last_undegraded":"2024-05-11T08:06:47.580505+0000","last_fullsized":"2024-05-11T08:06:47.580505+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T08:06:47.596912+0000","last_change":"2024-05-11T08:06:47.592140+0000","last_active":"2024-05-11T08:06:47.596912+0000","last_peered":"2024-05-11T08:06:47.596912+0000","last_clean":"2024-05-11T08:06:47.596912+0000","last_became_active":"2024-05-11T08:06:47.591867+0000","last_became_peered":"2024-05-11T08:06:47.591867+0000","last_unstale":"2024-05-11T08:06:47.596912+0000","last_undegraded":"2024-05-11T08:06:47.596912+0000","last_fullsized":"2024-05-11T08:06:47.596912+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:19:14.446185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.580040+0000","last_change":"2024-05-11T08:06:47.580040+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.580040+0000","last_undegraded":"2024-05-11T08:06:47.580040+0000","last_fullsized":"2024-05-11T08:06:47.580040+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T08:06:47.597734+0000","last_change":"2024-05-11T08:06:47.595937+0000","last_active":"2024-05-11T08:06:47.597734+0000","last_peered":"2024-05-11T08:06:47.597734+0000","last_clean":"2024-05-11T08:06:47.597734+0000","last_became_active":"2024-05-11T08:06:47.595642+0000","last_became_peered":"2024-05-11T08:06:47.595642+0000","last_unstale":"2024-05-11T08:06:47.597734+0000","last_undegraded":"2024-05-11T08:06:47.597734+0000","last_fullsized":"2024-05-11T08:06:47.597734+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:38:12.898297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.585260+0000","last_change":"2024-05-11T08:06:47.585260+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.585260+0000","last_undegraded":"2024-05-11T08:06:47.585260+0000","last_fullsized":"2024-05-11T08:06:47.585260+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T08:06:46.564761+0000","last_change":"2024-05-11T08:06:46.564761+0000","last_active":"2024-05-11T08:06:46.564761+0000","last_peered":"2024-05-11T08:06:46.564761+0000","last_clean":"2024-05-11T08:06:46.564761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:46.564761+0000","last_undegraded":"2024-05-11T08:06:46.564761+0000","last_fullsized":"2024-05-11T08:06:46.564761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T08:06:47.573468+0000","last_change":"2024-05-11T08:06:47.573468+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.573468+0000","last_undegraded":"2024-05-11T08:06:47.573468+0000","last_fullsized":"2024-05-11T08:06:47.573468+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T08:06:47.596598+0000","last_change":"2024-05-11T08:06:47.592090+0000","last_active":"2024-05-11T08:06:47.596598+0000","last_peered":"2024-05-11T08:06:47.596598+0000","last_clean":"2024-05-11T08:06:47.596598+0000","last_became_active":"2024-05-11T08:06:47.591813+0000","last_became_peered":"2024-05-11T08:06:47.591813+0000","last_unstale":"2024-05-11T08:06:47.596598+0000","last_undegraded":"2024-05-11T08:06:47.596598+0000","last_fullsized":"2024-05-11T08:06:47.596598+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:27:11.749333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.585274+0000","last_change":"2024-05-11T08:06:47.585274+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.585274+0000","last_undegraded":"2024-05-11T08:06:47.585274+0000","last_fullsized":"2024-05-11T08:06:47.585274+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T08:06:46.564761+0000","last_change":"2024-05-11T08:06:46.564761+0000","last_active":"2024-05-11T08:06:46.564761+0000","last_peered":"2024-05-11T08:06:46.564761+0000","last_clean":"2024-05-11T08:06:46.564761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:46.564761+0000","last_undegraded":"2024-05-11T08:06:46.564761+0000","last_fullsized":"2024-05-11T08:06:46.564761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.585232+0000","last_change":"2024-05-11T08:06:47.585232+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.585232+0000","last_undegraded":"2024-05-11T08:06:47.585232+0000","last_fullsized":"2024-05-11T08:06:47.585232+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T08:06:46.564761+0000","last_change":"2024-05-11T08:06:46.564761+0000","last_active":"2024-05-11T08:06:46.564761+0000","last_peered":"2024-05-11T08:06:46.564761+0000","last_clean":"2024-05-11T08:06:46.564761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:46.564761+0000","last_undegraded":"2024-05-11T08:06:46.564761+0000","last_fullsized":"2024-05-11T08:06:46.564761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.578838+0000","last_change":"2024-05-11T08:06:47.578838+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.578838+0000","last_undegraded":"2024-05-11T08:06:47.578838+0000","last_fullsized":"2024-05-11T08:06:47.578838+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T08:06:47.598256+0000","last_change":"2024-05-11T08:06:47.595398+0000","last_active":"2024-05-11T08:06:47.598256+0000","last_peered":"2024-05-11T08:06:47.598256+0000","last_clean":"2024-05-11T08:06:47.598256+0000","last_became_active":"2024-05-11T08:06:47.595129+0000","last_became_peered":"2024-05-11T08:06:47.595129+0000","last_unstale":"2024-05-11T08:06:47.598256+0000","last_undegraded":"2024-05-11T08:06:47.598256+0000","last_fullsized":"2024-05-11T08:06:47.598256+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:38:12.898297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.580042+0000","last_change":"2024-05-11T08:06:47.580042+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.580042+0000","last_undegraded":"2024-05-11T08:06:47.580042+0000","last_fullsized":"2024-05-11T08:06:47.580042+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T08:06:46.564761+0000","last_change":"2024-05-11T08:06:46.564761+0000","last_active":"2024-05-11T08:06:46.564761+0000","last_peered":"2024-05-11T08:06:46.564761+0000","last_clean":"2024-05-11T08:06:46.564761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:46.564761+0000","last_undegraded":"2024-05-11T08:06:46.564761+0000","last_fullsized":"2024-05-11T08:06:46.564761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.585245+0000","last_change":"2024-05-11T08:06:47.585245+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.585245+0000","last_undegraded":"2024-05-11T08:06:47.585245+0000","last_fullsized":"2024-05-11T08:06:47.585245+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T08:06:47.596881+0000","last_change":"2024-05-11T08:06:47.588756+0000","last_active":"2024-05-11T08:06:47.596881+0000","last_peered":"2024-05-11T08:06:47.596881+0000","last_clean":"2024-05-11T08:06:47.596881+0000","last_became_active":"2024-05-11T08:06:47.588460+0000","last_became_peered":"2024-05-11T08:06:47.588460+0000","last_unstale":"2024-05-11T08:06:47.596881+0000","last_undegraded":"2024-05-11T08:06:47.596881+0000","last_fullsized":"2024-05-11T08:06:47.596881+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:38:12.898297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.584127+0000","last_change":"2024-05-11T08:06:47.584127+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.584127+0000","last_undegraded":"2024-05-11T08:06:47.584127+0000","last_fullsized":"2024-05-11T08:06:47.584127+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-11T08:06:46.564761+0000","last_change":"2024-05-11T08:06:46.564761+0000","last_active":"2024-05-11T08:06:46.564761+0000","last_peered":"2024-05-11T08:06:46.564761+0000","last_clean":"2024-05-11T08:06:46.564761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:46.564761+0000","last_undegraded":"2024-05-11T08:06:46.564761+0000","last_fullsized":"2024-05-11T08:06:46.564761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.579855+0000","last_change":"2024-05-11T08:06:47.579855+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.579855+0000","last_undegraded":"2024-05-11T08:06:47.579855+0000","last_fullsized":"2024-05-11T08:06:47.579855+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T08:06:47.604460+0000","last_change":"2024-05-11T08:06:47.599942+0000","last_active":"2024-05-11T08:06:47.604460+0000","last_peered":"2024-05-11T08:06:47.604460+0000","last_clean":"2024-05-11T08:06:47.604460+0000","last_became_active":"2024-05-11T08:06:47.599513+0000","last_became_peered":"2024-05-11T08:06:47.599513+0000","last_unstale":"2024-05-11T08:06:47.604460+0000","last_undegraded":"2024-05-11T08:06:47.604460+0000","last_fullsized":"2024-05-11T08:06:47.604460+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:56:21.008947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.584668+0000","last_change":"2024-05-11T08:06:47.584668+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.584668+0000","last_undegraded":"2024-05-11T08:06:47.584668+0000","last_fullsized":"2024-05-11T08:06:47.584668+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T08:06:47.604497+0000","last_change":"2024-05-11T08:06:47.600002+0000","last_active":"2024-05-11T08:06:47.604497+0000","last_peered":"2024-05-11T08:06:47.604497+0000","last_clean":"2024-05-11T08:06:47.604497+0000","last_became_active":"2024-05-11T08:06:47.599594+0000","last_became_peered":"2024-05-11T08:06:47.599594+0000","last_unstale":"2024-05-11T08:06:47.604497+0000","last_undegraded":"2024-05-11T08:06:47.604497+0000","last_fullsized":"2024-05-11T08:06:47.604497+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:38:12.898297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.584762+0000","last_change":"2024-05-11T08:06:47.584762+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.584762+0000","last_undegraded":"2024-05-11T08:06:47.584762+0000","last_fullsized":"2024-05-11T08:06:47.584762+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T08:06:47.602973+0000","last_change":"2024-05-11T08:06:47.600101+0000","last_active":"2024-05-11T08:06:47.602973+0000","last_peered":"2024-05-11T08:06:47.602973+0000","last_clean":"2024-05-11T08:06:47.602973+0000","last_became_active":"2024-05-11T08:06:47.599765+0000","last_became_peered":"2024-05-11T08:06:47.599765+0000","last_unstale":"2024-05-11T08:06:47.602973+0000","last_undegraded":"2024-05-11T08:06:47.602973+0000","last_fullsized":"2024-05-11T08:06:47.602973+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:27:11.749333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.585048+0000","last_change":"2024-05-11T08:06:47.585048+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.585048+0000","last_undegraded":"2024-05-11T08:06:47.585048+0000","last_fullsized":"2024-05-11T08:06:47.585048+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T08:06:47.596399+0000","last_change":"2024-05-11T08:06:47.592149+0000","last_active":"2024-05-11T08:06:47.596399+0000","last_peered":"2024-05-11T08:06:47.596399+0000","last_clean":"2024-05-11T08:06:47.596399+0000","last_became_active":"2024-05-11T08:06:47.591790+0000","last_became_peered":"2024-05-11T08:06:47.591790+0000","last_unstale":"2024-05-11T08:06:47.596399+0000","last_undegraded":"2024-05-11T08:06:47.596399+0000","last_fullsized":"2024-05-11T08:06:47.596399+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:08:22.364810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.583351+0000","last_change":"2024-05-11T08:06:47.583351+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.583351+0000","last_undegraded":"2024-05-11T08:06:47.583351+0000","last_fullsized":"2024-05-11T08:06:47.583351+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T08:06:47.591117+0000","last_change":"2024-05-11T08:06:47.589050+0000","last_active":"2024-05-11T08:06:47.591117+0000","last_peered":"2024-05-11T08:06:47.591117+0000","last_clean":"2024-05-11T08:06:47.591117+0000","last_became_active":"2024-05-11T08:06:47.588780+0000","last_became_peered":"2024-05-11T08:06:47.588780+0000","last_unstale":"2024-05-11T08:06:47.591117+0000","last_undegraded":"2024-05-11T08:06:47.591117+0000","last_fullsized":"2024-05-11T08:06:47.591117+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:58:03.492050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.580419+0000","last_change":"2024-05-11T08:06:47.580419+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.580419+0000","last_undegraded":"2024-05-11T08:06:47.580419+0000","last_fullsized":"2024-05-11T08:06:47.580419+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T08:06:47.602453+0000","last_change":"2024-05-11T08:06:47.597382+0000","last_active":"2024-05-11T08:06:47.602453+0000","last_peered":"2024-05-11T08:06:47.602453+0000","last_clean":"2024-05-11T08:06:47.602453+0000","last_became_active":"2024-05-11T08:06:47.597062+0000","last_became_peered":"2024-05-11T08:06:47.597062+0000","last_unstale":"2024-05-11T08:06:47.602453+0000","last_undegraded":"2024-05-11T08:06:47.602453+0000","last_fullsized":"2024-05-11T08:06:47.602453+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:12:27.297983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":142,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T08:06:47.587680+0000","last_change":"2024-05-11T08:05:45.772368+0000","last_active":"2024-05-11T08:06:47.587680+0000","last_peered":"2024-05-11T08:06:47.587680+0000","last_clean":"2024-05-11T08:06:47.587680+0000","last_became_active":"2024-05-11T08:05:45.771373+0000","last_became_peered":"2024-05-11T08:05:45.771373+0000","last_unstale":"2024-05-11T08:06:47.587680+0000","last_undegraded":"2024-05-11T08:06:47.587680+0000","last_fullsized":"2024-05-11T08:06:47.587680+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:44.616617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:44.616617+0000","last_clean_scrub_stamp":"2024-05-11T08:05:44.616617+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:02:08.078748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":116,"acting":116,"num_store_stats":7},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":54,"acting":54,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509457,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28032,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343808,"statfs":{"total":96636764160,"available":96608059392,"internally_reserved":0,"allocated":1036288,"data_stored":776838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509457,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27452,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344388,"statfs":{"total":96636764160,"available":96608653312,"internally_reserved":0,"allocated":442368,"data_stored":186470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509457,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27440,"kb_used_data":420,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":430080,"data_stored":179803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509457,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27440,"kb_used_data":420,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":430080,"data_stored":179803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509458,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27440,"kb_used_data":420,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":430080,"data_stored":179803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509458,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27440,"kb_used_data":420,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":430080,"data_stored":179803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509457,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28020,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343820,"statfs":{"total":96636764160,"available":96608071680,"internally_reserved":0,"allocated":1024000,"data_stored":770171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574865,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27428,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":417792,"data_stored":173594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-11T08:06:51.602 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-11T08:06:51.739 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.739+0000 7f6331151640 1 Processor -- start 2024-05-11T08:06:51.739 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.739+0000 7f6331151640 1 -- start start 2024-05-11T08:06:51.740 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.740+0000 7f6331151640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f632c15f360 0x7f632c0cfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:51.740 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.740+0000 7f6331151640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f632c0b5e90 0x7f632c15f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:51.741 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.740+0000 7f6331151640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f632c0d2410 con 0x7f632c15fc80 2024-05-11T08:06:51.741 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.740+0000 7f6331151640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f632c0d26f0 con 0x7f632c0b5e90 2024-05-11T08:06:51.741 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.740+0000 7f6331151640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f632c0d29d0 con 0x7f632c15f360 2024-05-11T08:06:51.741 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.740+0000 7f632259c640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f632c0b5e90 0x7f632c15f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:51.741 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.740+0000 7f632259c640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f632c0b5e90 0x7f632c15f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:44030/0 (socket says 172.21.15.167:44030) 2024-05-11T08:06:51.741 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.740+0000 7f632259c640 1 -- 172.21.15.167:0/1515506973 learned_addr learned my addr 172.21.15.167:0/1515506973 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:51.741 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.740+0000 7f632259c640 1 -- 172.21.15.167:0/1515506973 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f632c15f360 msgr2=0x7f632c0cfdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:51.741 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.740+0000 7f632259c640 1 --2- 172.21.15.167:0/1515506973 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f632c15f360 0x7f632c0cfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:51.741 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.740+0000 7f632259c640 1 -- 172.21.15.167:0/1515506973 >> v1:172.21.15.204:6789/0 conn(0x7f632c15fc80 legacy=0x7f632c0cd610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:51.741 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.741+0000 7f632259c640 1 -- 172.21.15.167:0/1515506973 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f632c0b7e20 con 0x7f632c0b5e90 2024-05-11T08:06:51.742 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.741+0000 7f632259c640 1 --2- 172.21.15.167:0/1515506973 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f632c0b5e90 0x7f632c15f740 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f6314009870 tx=0x7f631402a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5747066212cd5e73 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:51.742 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.741+0000 7f632a59c640 1 -- 172.21.15.167:0/1515506973 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6314037070 con 0x7f632c0b5e90 2024-05-11T08:06:51.743 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.741+0000 7f632a59c640 1 -- 172.21.15.167:0/1515506973 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6314032440 con 0x7f632c0b5e90 2024-05-11T08:06:51.743 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.741+0000 7f632a59c640 1 -- 172.21.15.167:0/1515506973 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6314032810 con 0x7f632c0b5e90 2024-05-11T08:06:51.743 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.742+0000 7f6331151640 1 -- 172.21.15.167:0/1515506973 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f632c0b5e90 msgr2=0x7f632c15f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:51.743 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.742+0000 7f6331151640 1 --2- 172.21.15.167:0/1515506973 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f632c0b5e90 0x7f632c15f740 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f6314009870 tx=0x7f631402a130 comp rx=0 tx=0).stop 2024-05-11T08:06:51.743 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.742+0000 7f6331151640 1 -- 172.21.15.167:0/1515506973 shutdown_connections 2024-05-11T08:06:51.743 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.742+0000 7f6331151640 1 --2- 172.21.15.167:0/1515506973 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f632c0b5e90 0x7f632c15f740 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:51.743 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.742+0000 7f6331151640 1 --2- 172.21.15.167:0/1515506973 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f632c15f360 0x7f632c0cfdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:51.743 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.742+0000 7f6331151640 1 -- 172.21.15.167:0/1515506973 >> 172.21.15.167:0/1515506973 conn(0x7f632c072730 msgr2=0x7f632c0d2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:51.743 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.742+0000 7f6331151640 1 -- 172.21.15.167:0/1515506973 shutdown_connections 2024-05-11T08:06:51.743 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.742+0000 7f6331151640 1 -- 172.21.15.167:0/1515506973 wait complete. 2024-05-11T08:06:51.745 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.743+0000 7f6331151640 1 Processor -- start 2024-05-11T08:06:51.745 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.744+0000 7f6331151640 1 -- start start 2024-05-11T08:06:51.746 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.744+0000 7f6331151640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f632c0b5e90 0x7f632c0702f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:51.746 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.744+0000 7f6331151640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f632c15f360 0x7f632c156600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:51.746 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.744+0000 7f6331151640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f632c15fc80 0x7f632c15cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:51.746 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.744+0000 7f6331151640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f632c0cdd20 con 0x7f632c0b5e90 2024-05-11T08:06:51.746 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.744+0000 7f6331151640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f632c0b7ad0 con 0x7f632c15fc80 2024-05-11T08:06:51.746 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.744+0000 7f6331151640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f632c0cdea0 con 0x7f632c15f360 2024-05-11T08:06:51.746 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.744+0000 7f632ad9d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f632c0b5e90 0x7f632c0702f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:51.746 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.744+0000 7f632ad9d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f632c0b5e90 0x7f632c0702f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:44036/0 (socket says 172.21.15.167:44036) 2024-05-11T08:06:51.746 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.744+0000 7f632ad9d640 1 -- 172.21.15.167:0/1191248410 learned_addr learned my addr 172.21.15.167:0/1191248410 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:51.746 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.745+0000 7f632259c640 1 --2- 172.21.15.167:0/1191248410 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f632c15f360 0x7f632c156600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:51.746 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.745+0000 7f632b59e640 1 --2- 172.21.15.167:0/1191248410 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f632c15fc80 0x7f632c15cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:51.746 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.745+0000 7f632ad9d640 1 -- 172.21.15.167:0/1191248410 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f632c15f360 msgr2=0x7f632c156600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:51.746 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.745+0000 7f632ad9d640 1 --2- 172.21.15.167:0/1191248410 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f632c15f360 0x7f632c156600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:51.747 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.745+0000 7f632ad9d640 1 -- 172.21.15.167:0/1191248410 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f632c15fc80 msgr2=0x7f632c15cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:51.747 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.745+0000 7f632ad9d640 1 --2- 172.21.15.167:0/1191248410 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f632c15fc80 0x7f632c15cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:51.747 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.745+0000 7f632ad9d640 1 -- 172.21.15.167:0/1191248410 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f632c070990 con 0x7f632c0b5e90 2024-05-11T08:06:51.747 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.745+0000 7f632ad9d640 1 --2- 172.21.15.167:0/1191248410 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f632c0b5e90 0x7f632c0702f0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f631c00b4a0 tx=0x7f631c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:51.747 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.745+0000 7f6323fff640 1 -- 172.21.15.167:0/1191248410 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f631c014070 con 0x7f632c0b5e90 2024-05-11T08:06:51.747 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.745+0000 7f6323fff640 1 -- 172.21.15.167:0/1191248410 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f631c010610 con 0x7f632c0b5e90 2024-05-11T08:06:51.747 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.745+0000 7f6331151640 1 -- 172.21.15.167:0/1191248410 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f632c070c70 con 0x7f632c0b5e90 2024-05-11T08:06:51.747 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.745+0000 7f6323fff640 1 -- 172.21.15.167:0/1191248410 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f631c010a00 con 0x7f632c0b5e90 2024-05-11T08:06:51.747 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.745+0000 7f6331151640 1 -- 172.21.15.167:0/1191248410 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f632c071230 con 0x7f632c0b5e90 2024-05-11T08:06:51.748 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.746+0000 7f6331151640 1 -- 172.21.15.167:0/1191248410 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f632c069590 con 0x7f632c0b5e90 2024-05-11T08:06:51.748 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.747+0000 7f6323fff640 1 -- 172.21.15.167:0/1191248410 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f631c017070 con 0x7f632c0b5e90 2024-05-11T08:06:51.748 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.747+0000 7f6323fff640 1 --2- 172.21.15.167:0/1191248410 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f630c076300 0x7f630c0787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:51.748 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.747+0000 7f6323fff640 1 -- 172.21.15.167:0/1191248410 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6101+0+0 (secure 0 0 0) 0x7f631c097100 con 0x7f632c0b5e90 2024-05-11T08:06:51.748 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.747+0000 7f632259c640 1 --2- 172.21.15.167:0/1191248410 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f630c076300 0x7f630c0787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:51.748 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.748+0000 7f632259c640 1 --2- 172.21.15.167:0/1191248410 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f630c076300 0x7f630c0787c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f6314034000 tx=0x7f631402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:51.751 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.752+0000 7f6323fff640 1 -- 172.21.15.167:0/1191248410 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f631c060960 con 0x7f632c0b5e90 2024-05-11T08:06:51.930 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.930+0000 7f6331151640 1 -- 172.21.15.167:0/1191248410 --> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f632c15e980 con 0x7f630c076300 2024-05-11T08:06:51.936 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.936+0000 7f6323fff640 1 -- 172.21.15.167:0/1191248410 <== mgr.4113 v2:172.21.15.167:6832/253569496 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247131 (secure 0 0 0) 0x7f632c15e980 con 0x7f630c076300 2024-05-11T08:06:51.939 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T08:06:51.940 INFO:teuthology.orchestra.run.smithi167.stderr:dumped all 2024-05-11T08:06:51.942 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.939+0000 7f6331151640 1 -- 172.21.15.167:0/1191248410 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f630c076300 msgr2=0x7f630c0787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:51.942 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.939+0000 7f6331151640 1 --2- 172.21.15.167:0/1191248410 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f630c076300 0x7f630c0787c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f6314034000 tx=0x7f631402e000 comp rx=0 tx=0).stop 2024-05-11T08:06:51.942 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.939+0000 7f6331151640 1 -- 172.21.15.167:0/1191248410 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f632c0b5e90 msgr2=0x7f632c0702f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:51.942 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.939+0000 7f6331151640 1 --2- 172.21.15.167:0/1191248410 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f632c0b5e90 0x7f632c0702f0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f631c00b4a0 tx=0x7f631c00b960 comp rx=0 tx=0).stop 2024-05-11T08:06:51.942 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.939+0000 7f6331151640 1 -- 172.21.15.167:0/1191248410 shutdown_connections 2024-05-11T08:06:51.942 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.939+0000 7f6331151640 1 --2- 172.21.15.167:0/1191248410 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f630c076300 0x7f630c0787c0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:51.942 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.939+0000 7f6331151640 1 --2- 172.21.15.167:0/1191248410 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f632c15fc80 0x7f632c15cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:51.942 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.939+0000 7f6331151640 1 --2- 172.21.15.167:0/1191248410 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f632c15f360 0x7f632c156600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:51.942 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.939+0000 7f6331151640 1 --2- 172.21.15.167:0/1191248410 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f632c0b5e90 0x7f632c0702f0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:51.942 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.939+0000 7f6331151640 1 -- 172.21.15.167:0/1191248410 >> 172.21.15.167:0/1191248410 conn(0x7f632c072730 msgr2=0x7f632c0d2fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:51.942 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.939+0000 7f6331151640 1 -- 172.21.15.167:0/1191248410 shutdown_connections 2024-05-11T08:06:51.943 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:51.939+0000 7f6331151640 1 -- 172.21.15.167:0/1191248410 wait complete. 2024-05-11T08:06:51.950 INFO:teuthology.orchestra.run.smithi167.stdout:{"pg_ready":true,"pg_map":{"version":70,"stamp":"2024-05-11T08:06:51.928061+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590438,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":104,"num_write_kb":1629,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":46,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220744,"kb_used_data":4584,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753976,"statfs":{"total":773094113280,"available":772868071424,"internally_reserved":0,"allocated":4694016,"data_stored":2653090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":70,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.351869"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.580160+0000","last_change":"2024-05-11T08:06:47.580160+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.580160+0000","last_undegraded":"2024-05-11T08:06:47.580160+0000","last_fullsized":"2024-05-11T08:06:47.580160+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.580981+0000","last_change":"2024-05-11T08:06:47.580981+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.580981+0000","last_undegraded":"2024-05-11T08:06:47.580981+0000","last_fullsized":"2024-05-11T08:06:47.580981+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.587939+0000","last_change":"2024-05-11T08:06:47.587939+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.587939+0000","last_undegraded":"2024-05-11T08:06:47.587939+0000","last_fullsized":"2024-05-11T08:06:47.587939+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T08:06:49.613242+0000","last_change":"2024-05-11T08:06:48.616894+0000","last_active":"2024-05-11T08:06:49.613242+0000","last_peered":"2024-05-11T08:06:49.613242+0000","last_clean":"2024-05-11T08:06:49.613242+0000","last_became_active":"2024-05-11T08:06:48.616597+0000","last_became_peered":"2024-05-11T08:06:48.616597+0000","last_unstale":"2024-05-11T08:06:49.613242+0000","last_undegraded":"2024-05-11T08:06:49.613242+0000","last_fullsized":"2024-05-11T08:06:49.613242+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:47:37.702924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T08:06:49.613143+0000","last_change":"2024-05-11T08:06:48.616535+0000","last_active":"2024-05-11T08:06:49.613143+0000","last_peered":"2024-05-11T08:06:49.613143+0000","last_clean":"2024-05-11T08:06:49.613143+0000","last_became_active":"2024-05-11T08:06:48.616283+0000","last_became_peered":"2024-05-11T08:06:48.616283+0000","last_unstale":"2024-05-11T08:06:49.613143+0000","last_undegraded":"2024-05-11T08:06:49.613143+0000","last_fullsized":"2024-05-11T08:06:49.613143+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:12:46.331254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.586507+0000","last_change":"2024-05-11T08:06:47.586507+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.586507+0000","last_undegraded":"2024-05-11T08:06:47.586507+0000","last_fullsized":"2024-05-11T08:06:47.586507+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.580887+0000","last_change":"2024-05-11T08:06:47.580887+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.580887+0000","last_undegraded":"2024-05-11T08:06:47.580887+0000","last_fullsized":"2024-05-11T08:06:47.580887+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.579778+0000","last_change":"2024-05-11T08:06:47.579778+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.579778+0000","last_undegraded":"2024-05-11T08:06:47.579778+0000","last_fullsized":"2024-05-11T08:06:47.579778+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.585551+0000","last_change":"2024-05-11T08:06:47.585551+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.585551+0000","last_undegraded":"2024-05-11T08:06:47.585551+0000","last_fullsized":"2024-05-11T08:06:47.585551+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.579247+0000","last_change":"2024-05-11T08:06:47.579247+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.579247+0000","last_undegraded":"2024-05-11T08:06:47.579247+0000","last_fullsized":"2024-05-11T08:06:47.579247+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.580547+0000","last_change":"2024-05-11T08:06:47.580547+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.580547+0000","last_undegraded":"2024-05-11T08:06:47.580547+0000","last_fullsized":"2024-05-11T08:06:47.580547+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.584544+0000","last_change":"2024-05-11T08:06:47.584544+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.584544+0000","last_undegraded":"2024-05-11T08:06:47.584544+0000","last_fullsized":"2024-05-11T08:06:47.584544+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.580055+0000","last_change":"2024-05-11T08:06:47.580055+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.580055+0000","last_undegraded":"2024-05-11T08:06:47.580055+0000","last_fullsized":"2024-05-11T08:06:47.580055+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.585515+0000","last_change":"2024-05-11T08:06:47.585515+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.585515+0000","last_undegraded":"2024-05-11T08:06:47.585515+0000","last_fullsized":"2024-05-11T08:06:47.585515+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.579888+0000","last_change":"2024-05-11T08:06:47.579888+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.579888+0000","last_undegraded":"2024-05-11T08:06:47.579888+0000","last_fullsized":"2024-05-11T08:06:47.579888+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.585758+0000","last_change":"2024-05-11T08:06:47.585758+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.585758+0000","last_undegraded":"2024-05-11T08:06:47.585758+0000","last_fullsized":"2024-05-11T08:06:47.585758+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.587418+0000","last_change":"2024-05-11T08:06:47.587418+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.587418+0000","last_undegraded":"2024-05-11T08:06:47.587418+0000","last_fullsized":"2024-05-11T08:06:47.587418+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.588413+0000","last_change":"2024-05-11T08:06:47.588413+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.588413+0000","last_undegraded":"2024-05-11T08:06:47.588413+0000","last_fullsized":"2024-05-11T08:06:47.588413+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T08:06:49.613351+0000","last_change":"2024-05-11T08:06:48.606813+0000","last_active":"2024-05-11T08:06:49.613351+0000","last_peered":"2024-05-11T08:06:49.613351+0000","last_clean":"2024-05-11T08:06:49.613351+0000","last_became_active":"2024-05-11T08:06:48.606548+0000","last_became_peered":"2024-05-11T08:06:48.606548+0000","last_unstale":"2024-05-11T08:06:49.613351+0000","last_undegraded":"2024-05-11T08:06:49.613351+0000","last_fullsized":"2024-05-11T08:06:49.613351+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:40:53.505484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.587907+0000","last_change":"2024-05-11T08:06:47.587907+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.587907+0000","last_undegraded":"2024-05-11T08:06:47.587907+0000","last_fullsized":"2024-05-11T08:06:47.587907+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.579095+0000","last_change":"2024-05-11T08:06:47.579095+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.579095+0000","last_undegraded":"2024-05-11T08:06:47.579095+0000","last_fullsized":"2024-05-11T08:06:47.579095+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.583273+0000","last_change":"2024-05-11T08:06:47.583273+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.583273+0000","last_undegraded":"2024-05-11T08:06:47.583273+0000","last_fullsized":"2024-05-11T08:06:47.583273+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.583780+0000","last_change":"2024-05-11T08:06:47.583780+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.583780+0000","last_undegraded":"2024-05-11T08:06:47.583780+0000","last_fullsized":"2024-05-11T08:06:47.583780+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T08:06:49.613144+0000","last_change":"2024-05-11T08:06:48.615211+0000","last_active":"2024-05-11T08:06:49.613144+0000","last_peered":"2024-05-11T08:06:49.613144+0000","last_clean":"2024-05-11T08:06:49.613144+0000","last_became_active":"2024-05-11T08:06:48.614902+0000","last_became_peered":"2024-05-11T08:06:48.614902+0000","last_unstale":"2024-05-11T08:06:49.613144+0000","last_undegraded":"2024-05-11T08:06:49.613144+0000","last_fullsized":"2024-05-11T08:06:49.613144+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:20:23.234025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.589116+0000","last_change":"2024-05-11T08:06:47.589116+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.589116+0000","last_undegraded":"2024-05-11T08:06:47.589116+0000","last_fullsized":"2024-05-11T08:06:47.589116+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.586184+0000","last_change":"2024-05-11T08:06:47.586184+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.586184+0000","last_undegraded":"2024-05-11T08:06:47.586184+0000","last_fullsized":"2024-05-11T08:06:47.586184+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.587324+0000","last_change":"2024-05-11T08:06:47.587324+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.587324+0000","last_undegraded":"2024-05-11T08:06:47.587324+0000","last_fullsized":"2024-05-11T08:06:47.587324+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.578757+0000","last_change":"2024-05-11T08:06:47.578757+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.578757+0000","last_undegraded":"2024-05-11T08:06:47.578757+0000","last_fullsized":"2024-05-11T08:06:47.578757+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.588841+0000","last_change":"2024-05-11T08:06:47.588841+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.588841+0000","last_undegraded":"2024-05-11T08:06:47.588841+0000","last_fullsized":"2024-05-11T08:06:47.588841+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.585718+0000","last_change":"2024-05-11T08:06:47.585718+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.585718+0000","last_undegraded":"2024-05-11T08:06:47.585718+0000","last_fullsized":"2024-05-11T08:06:47.585718+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.583214+0000","last_change":"2024-05-11T08:06:47.583214+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.583214+0000","last_undegraded":"2024-05-11T08:06:47.583214+0000","last_fullsized":"2024-05-11T08:06:47.583214+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.587217+0000","last_change":"2024-05-11T08:06:47.587217+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.587217+0000","last_undegraded":"2024-05-11T08:06:47.587217+0000","last_fullsized":"2024-05-11T08:06:47.587217+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.584847+0000","last_change":"2024-05-11T08:06:47.584847+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.584847+0000","last_undegraded":"2024-05-11T08:06:47.584847+0000","last_fullsized":"2024-05-11T08:06:47.584847+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T08:06:47.594702+0000","last_change":"2024-05-11T08:06:47.591074+0000","last_active":"2024-05-11T08:06:47.594702+0000","last_peered":"2024-05-11T08:06:47.594702+0000","last_clean":"2024-05-11T08:06:47.594702+0000","last_became_active":"2024-05-11T08:06:47.590747+0000","last_became_peered":"2024-05-11T08:06:47.590747+0000","last_unstale":"2024-05-11T08:06:47.594702+0000","last_undegraded":"2024-05-11T08:06:47.594702+0000","last_fullsized":"2024-05-11T08:06:47.594702+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:11:53.548277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.585679+0000","last_change":"2024-05-11T08:06:47.585679+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.585679+0000","last_undegraded":"2024-05-11T08:06:47.585679+0000","last_fullsized":"2024-05-11T08:06:47.585679+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T08:06:47.601797+0000","last_change":"2024-05-11T08:06:47.597547+0000","last_active":"2024-05-11T08:06:47.601797+0000","last_peered":"2024-05-11T08:06:47.601797+0000","last_clean":"2024-05-11T08:06:47.601797+0000","last_became_active":"2024-05-11T08:06:47.597260+0000","last_became_peered":"2024-05-11T08:06:47.597260+0000","last_unstale":"2024-05-11T08:06:47.601797+0000","last_undegraded":"2024-05-11T08:06:47.601797+0000","last_fullsized":"2024-05-11T08:06:47.601797+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:08:22.364810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.584309+0000","last_change":"2024-05-11T08:06:47.584309+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.584309+0000","last_undegraded":"2024-05-11T08:06:47.584309+0000","last_fullsized":"2024-05-11T08:06:47.584309+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T08:06:47.601604+0000","last_change":"2024-05-11T08:06:47.597745+0000","last_active":"2024-05-11T08:06:47.601604+0000","last_peered":"2024-05-11T08:06:47.601604+0000","last_clean":"2024-05-11T08:06:47.601604+0000","last_became_active":"2024-05-11T08:06:47.597472+0000","last_became_peered":"2024-05-11T08:06:47.597472+0000","last_unstale":"2024-05-11T08:06:47.601604+0000","last_undegraded":"2024-05-11T08:06:47.601604+0000","last_fullsized":"2024-05-11T08:06:47.601604+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:56:21.008947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.580834+0000","last_change":"2024-05-11T08:06:47.580834+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.580834+0000","last_undegraded":"2024-05-11T08:06:47.580834+0000","last_fullsized":"2024-05-11T08:06:47.580834+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T08:06:47.596539+0000","last_change":"2024-05-11T08:06:47.594580+0000","last_active":"2024-05-11T08:06:47.596539+0000","last_peered":"2024-05-11T08:06:47.596539+0000","last_clean":"2024-05-11T08:06:47.596539+0000","last_became_active":"2024-05-11T08:06:47.594315+0000","last_became_peered":"2024-05-11T08:06:47.594315+0000","last_unstale":"2024-05-11T08:06:47.596539+0000","last_undegraded":"2024-05-11T08:06:47.596539+0000","last_fullsized":"2024-05-11T08:06:47.596539+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:56:21.008947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.586542+0000","last_change":"2024-05-11T08:06:47.586542+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.586542+0000","last_undegraded":"2024-05-11T08:06:47.586542+0000","last_fullsized":"2024-05-11T08:06:47.586542+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T08:06:47.602415+0000","last_change":"2024-05-11T08:06:47.600311+0000","last_active":"2024-05-11T08:06:47.602415+0000","last_peered":"2024-05-11T08:06:47.602415+0000","last_clean":"2024-05-11T08:06:47.602415+0000","last_became_active":"2024-05-11T08:06:47.600072+0000","last_became_peered":"2024-05-11T08:06:47.600072+0000","last_unstale":"2024-05-11T08:06:47.602415+0000","last_undegraded":"2024-05-11T08:06:47.602415+0000","last_fullsized":"2024-05-11T08:06:47.602415+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:13:57.487739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.578355+0000","last_change":"2024-05-11T08:06:47.578355+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.578355+0000","last_undegraded":"2024-05-11T08:06:47.578355+0000","last_fullsized":"2024-05-11T08:06:47.578355+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T08:06:47.602942+0000","last_change":"2024-05-11T08:06:47.598857+0000","last_active":"2024-05-11T08:06:47.602942+0000","last_peered":"2024-05-11T08:06:47.602942+0000","last_clean":"2024-05-11T08:06:47.602942+0000","last_became_active":"2024-05-11T08:06:47.598102+0000","last_became_peered":"2024-05-11T08:06:47.598102+0000","last_unstale":"2024-05-11T08:06:47.602942+0000","last_undegraded":"2024-05-11T08:06:47.602942+0000","last_fullsized":"2024-05-11T08:06:47.602942+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:13:57.487739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.583760+0000","last_change":"2024-05-11T08:06:47.583760+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.583760+0000","last_undegraded":"2024-05-11T08:06:47.583760+0000","last_fullsized":"2024-05-11T08:06:47.583760+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T08:06:47.599146+0000","last_change":"2024-05-11T08:06:47.595791+0000","last_active":"2024-05-11T08:06:47.599146+0000","last_peered":"2024-05-11T08:06:47.599146+0000","last_clean":"2024-05-11T08:06:47.599146+0000","last_became_active":"2024-05-11T08:06:47.595551+0000","last_became_peered":"2024-05-11T08:06:47.595551+0000","last_unstale":"2024-05-11T08:06:47.599146+0000","last_undegraded":"2024-05-11T08:06:47.599146+0000","last_fullsized":"2024-05-11T08:06:47.599146+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:00:29.291380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.581488+0000","last_change":"2024-05-11T08:06:47.581488+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.581488+0000","last_undegraded":"2024-05-11T08:06:47.581488+0000","last_fullsized":"2024-05-11T08:06:47.581488+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T08:06:47.596271+0000","last_change":"2024-05-11T08:06:47.590670+0000","last_active":"2024-05-11T08:06:47.596271+0000","last_peered":"2024-05-11T08:06:47.596271+0000","last_clean":"2024-05-11T08:06:47.596271+0000","last_became_active":"2024-05-11T08:06:47.590380+0000","last_became_peered":"2024-05-11T08:06:47.590380+0000","last_unstale":"2024-05-11T08:06:47.596271+0000","last_undegraded":"2024-05-11T08:06:47.596271+0000","last_fullsized":"2024-05-11T08:06:47.596271+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:00:29.291380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T08:06:49.613537+0000","last_change":"2024-05-11T08:06:48.616855+0000","last_active":"2024-05-11T08:06:49.613537+0000","last_peered":"2024-05-11T08:06:49.613537+0000","last_clean":"2024-05-11T08:06:49.613537+0000","last_became_active":"2024-05-11T08:06:48.616287+0000","last_became_peered":"2024-05-11T08:06:48.616287+0000","last_unstale":"2024-05-11T08:06:49.613537+0000","last_undegraded":"2024-05-11T08:06:49.613537+0000","last_fullsized":"2024-05-11T08:06:49.613537+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:31:41.113021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T08:06:47.593461+0000","last_change":"2024-05-11T08:06:47.590630+0000","last_active":"2024-05-11T08:06:47.593461+0000","last_peered":"2024-05-11T08:06:47.593461+0000","last_clean":"2024-05-11T08:06:47.593461+0000","last_became_active":"2024-05-11T08:06:47.590361+0000","last_became_peered":"2024-05-11T08:06:47.590361+0000","last_unstale":"2024-05-11T08:06:47.593461+0000","last_undegraded":"2024-05-11T08:06:47.593461+0000","last_fullsized":"2024-05-11T08:06:47.593461+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:20:42.258886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.584566+0000","last_change":"2024-05-11T08:06:47.584566+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.584566+0000","last_undegraded":"2024-05-11T08:06:47.584566+0000","last_fullsized":"2024-05-11T08:06:47.584566+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T08:06:47.595167+0000","last_change":"2024-05-11T08:06:47.587418+0000","last_active":"2024-05-11T08:06:47.595167+0000","last_peered":"2024-05-11T08:06:47.595167+0000","last_clean":"2024-05-11T08:06:47.595167+0000","last_became_active":"2024-05-11T08:06:47.586715+0000","last_became_peered":"2024-05-11T08:06:47.586715+0000","last_unstale":"2024-05-11T08:06:47.595167+0000","last_undegraded":"2024-05-11T08:06:47.595167+0000","last_fullsized":"2024-05-11T08:06:47.595167+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:56:21.008947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.584770+0000","last_change":"2024-05-11T08:06:47.584770+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.584770+0000","last_undegraded":"2024-05-11T08:06:47.584770+0000","last_fullsized":"2024-05-11T08:06:47.584770+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T08:06:47.597011+0000","last_change":"2024-05-11T08:06:47.594319+0000","last_active":"2024-05-11T08:06:47.597011+0000","last_peered":"2024-05-11T08:06:47.597011+0000","last_clean":"2024-05-11T08:06:47.597011+0000","last_became_active":"2024-05-11T08:06:47.593840+0000","last_became_peered":"2024-05-11T08:06:47.593840+0000","last_unstale":"2024-05-11T08:06:47.597011+0000","last_undegraded":"2024-05-11T08:06:47.597011+0000","last_fullsized":"2024-05-11T08:06:47.597011+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:12:27.297983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.584560+0000","last_change":"2024-05-11T08:06:47.584560+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.584560+0000","last_undegraded":"2024-05-11T08:06:47.584560+0000","last_fullsized":"2024-05-11T08:06:47.584560+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T08:06:47.596789+0000","last_change":"2024-05-11T08:06:47.592968+0000","last_active":"2024-05-11T08:06:47.596789+0000","last_peered":"2024-05-11T08:06:47.596789+0000","last_clean":"2024-05-11T08:06:47.596789+0000","last_became_active":"2024-05-11T08:06:47.592672+0000","last_became_peered":"2024-05-11T08:06:47.592672+0000","last_unstale":"2024-05-11T08:06:47.596789+0000","last_undegraded":"2024-05-11T08:06:47.596789+0000","last_fullsized":"2024-05-11T08:06:47.596789+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:46:05.539312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.585804+0000","last_change":"2024-05-11T08:06:47.585804+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.585804+0000","last_undegraded":"2024-05-11T08:06:47.585804+0000","last_fullsized":"2024-05-11T08:06:47.585804+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T08:06:47.604642+0000","last_change":"2024-05-11T08:06:47.600533+0000","last_active":"2024-05-11T08:06:47.604642+0000","last_peered":"2024-05-11T08:06:47.604642+0000","last_clean":"2024-05-11T08:06:47.604642+0000","last_became_active":"2024-05-11T08:06:47.600226+0000","last_became_peered":"2024-05-11T08:06:47.600226+0000","last_unstale":"2024-05-11T08:06:47.604642+0000","last_undegraded":"2024-05-11T08:06:47.604642+0000","last_fullsized":"2024-05-11T08:06:47.604642+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:49:09.098705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.579895+0000","last_change":"2024-05-11T08:06:47.579895+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.579895+0000","last_undegraded":"2024-05-11T08:06:47.579895+0000","last_fullsized":"2024-05-11T08:06:47.579895+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T08:06:47.601569+0000","last_change":"2024-05-11T08:06:47.597276+0000","last_active":"2024-05-11T08:06:47.601569+0000","last_peered":"2024-05-11T08:06:47.601569+0000","last_clean":"2024-05-11T08:06:47.601569+0000","last_became_active":"2024-05-11T08:06:47.596986+0000","last_became_peered":"2024-05-11T08:06:47.596986+0000","last_unstale":"2024-05-11T08:06:47.601569+0000","last_undegraded":"2024-05-11T08:06:47.601569+0000","last_fullsized":"2024-05-11T08:06:47.601569+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:08:22.364810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.584152+0000","last_change":"2024-05-11T08:06:47.584152+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.584152+0000","last_undegraded":"2024-05-11T08:06:47.584152+0000","last_fullsized":"2024-05-11T08:06:47.584152+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T08:06:47.594681+0000","last_change":"2024-05-11T08:06:47.591052+0000","last_active":"2024-05-11T08:06:47.594681+0000","last_peered":"2024-05-11T08:06:47.594681+0000","last_clean":"2024-05-11T08:06:47.594681+0000","last_became_active":"2024-05-11T08:06:47.590726+0000","last_became_peered":"2024-05-11T08:06:47.590726+0000","last_unstale":"2024-05-11T08:06:47.594681+0000","last_undegraded":"2024-05-11T08:06:47.594681+0000","last_fullsized":"2024-05-11T08:06:47.594681+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:30:44.079909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.580872+0000","last_change":"2024-05-11T08:06:47.580872+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.580872+0000","last_undegraded":"2024-05-11T08:06:47.580872+0000","last_fullsized":"2024-05-11T08:06:47.580872+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T08:06:47.598143+0000","last_change":"2024-05-11T08:06:47.597077+0000","last_active":"2024-05-11T08:06:47.598143+0000","last_peered":"2024-05-11T08:06:47.598143+0000","last_clean":"2024-05-11T08:06:47.598143+0000","last_became_active":"2024-05-11T08:06:47.596753+0000","last_became_peered":"2024-05-11T08:06:47.596753+0000","last_unstale":"2024-05-11T08:06:47.598143+0000","last_undegraded":"2024-05-11T08:06:47.598143+0000","last_fullsized":"2024-05-11T08:06:47.598143+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:19:14.446185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.580505+0000","last_change":"2024-05-11T08:06:47.580505+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.580505+0000","last_undegraded":"2024-05-11T08:06:47.580505+0000","last_fullsized":"2024-05-11T08:06:47.580505+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T08:06:47.596912+0000","last_change":"2024-05-11T08:06:47.592140+0000","last_active":"2024-05-11T08:06:47.596912+0000","last_peered":"2024-05-11T08:06:47.596912+0000","last_clean":"2024-05-11T08:06:47.596912+0000","last_became_active":"2024-05-11T08:06:47.591867+0000","last_became_peered":"2024-05-11T08:06:47.591867+0000","last_unstale":"2024-05-11T08:06:47.596912+0000","last_undegraded":"2024-05-11T08:06:47.596912+0000","last_fullsized":"2024-05-11T08:06:47.596912+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:19:14.446185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.580040+0000","last_change":"2024-05-11T08:06:47.580040+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.580040+0000","last_undegraded":"2024-05-11T08:06:47.580040+0000","last_fullsized":"2024-05-11T08:06:47.580040+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T08:06:47.597734+0000","last_change":"2024-05-11T08:06:47.595937+0000","last_active":"2024-05-11T08:06:47.597734+0000","last_peered":"2024-05-11T08:06:47.597734+0000","last_clean":"2024-05-11T08:06:47.597734+0000","last_became_active":"2024-05-11T08:06:47.595642+0000","last_became_peered":"2024-05-11T08:06:47.595642+0000","last_unstale":"2024-05-11T08:06:47.597734+0000","last_undegraded":"2024-05-11T08:06:47.597734+0000","last_fullsized":"2024-05-11T08:06:47.597734+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:38:12.898297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.585260+0000","last_change":"2024-05-11T08:06:47.585260+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.585260+0000","last_undegraded":"2024-05-11T08:06:47.585260+0000","last_fullsized":"2024-05-11T08:06:47.585260+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"40'1","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T08:06:49.613353+0000","last_change":"2024-05-11T08:06:47.599057+0000","last_active":"2024-05-11T08:06:49.613353+0000","last_peered":"2024-05-11T08:06:49.613353+0000","last_clean":"2024-05-11T08:06:49.613353+0000","last_became_active":"2024-05-11T08:06:47.598647+0000","last_became_peered":"2024-05-11T08:06:47.598647+0000","last_unstale":"2024-05-11T08:06:49.613353+0000","last_undegraded":"2024-05-11T08:06:49.613353+0000","last_fullsized":"2024-05-11T08:06:49.613353+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:40:41.542604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T08:06:49.613419+0000","last_change":"2024-05-11T08:06:48.606563+0000","last_active":"2024-05-11T08:06:49.613419+0000","last_peered":"2024-05-11T08:06:49.613419+0000","last_clean":"2024-05-11T08:06:49.613419+0000","last_became_active":"2024-05-11T08:06:48.606287+0000","last_became_peered":"2024-05-11T08:06:48.606287+0000","last_unstale":"2024-05-11T08:06:49.613419+0000","last_undegraded":"2024-05-11T08:06:49.613419+0000","last_fullsized":"2024-05-11T08:06:49.613419+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:23:43.027012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T08:06:47.596598+0000","last_change":"2024-05-11T08:06:47.592090+0000","last_active":"2024-05-11T08:06:47.596598+0000","last_peered":"2024-05-11T08:06:47.596598+0000","last_clean":"2024-05-11T08:06:47.596598+0000","last_became_active":"2024-05-11T08:06:47.591813+0000","last_became_peered":"2024-05-11T08:06:47.591813+0000","last_unstale":"2024-05-11T08:06:47.596598+0000","last_undegraded":"2024-05-11T08:06:47.596598+0000","last_fullsized":"2024-05-11T08:06:47.596598+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:27:11.749333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.585274+0000","last_change":"2024-05-11T08:06:47.585274+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.585274+0000","last_undegraded":"2024-05-11T08:06:47.585274+0000","last_fullsized":"2024-05-11T08:06:47.585274+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"40'2","reported_seq":16,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T08:06:49.613224+0000","last_change":"2024-05-11T08:06:47.600614+0000","last_active":"2024-05-11T08:06:49.613224+0000","last_peered":"2024-05-11T08:06:49.613224+0000","last_clean":"2024-05-11T08:06:49.613224+0000","last_became_active":"2024-05-11T08:06:47.600104+0000","last_became_peered":"2024-05-11T08:06:47.600104+0000","last_unstale":"2024-05-11T08:06:49.613224+0000","last_undegraded":"2024-05-11T08:06:49.613224+0000","last_fullsized":"2024-05-11T08:06:49.613224+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:25:19.853657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.585232+0000","last_change":"2024-05-11T08:06:47.585232+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.585232+0000","last_undegraded":"2024-05-11T08:06:47.585232+0000","last_fullsized":"2024-05-11T08:06:47.585232+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":14,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T08:06:49.613139+0000","last_change":"2024-05-11T08:06:47.597958+0000","last_active":"2024-05-11T08:06:49.613139+0000","last_peered":"2024-05-11T08:06:49.613139+0000","last_clean":"2024-05-11T08:06:49.613139+0000","last_became_active":"2024-05-11T08:06:47.597638+0000","last_became_peered":"2024-05-11T08:06:47.597638+0000","last_unstale":"2024-05-11T08:06:49.613139+0000","last_undegraded":"2024-05-11T08:06:49.613139+0000","last_fullsized":"2024-05-11T08:06:49.613139+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:02:41.089263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.578838+0000","last_change":"2024-05-11T08:06:47.578838+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.578838+0000","last_undegraded":"2024-05-11T08:06:47.578838+0000","last_fullsized":"2024-05-11T08:06:47.578838+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T08:06:47.598256+0000","last_change":"2024-05-11T08:06:47.595398+0000","last_active":"2024-05-11T08:06:47.598256+0000","last_peered":"2024-05-11T08:06:47.598256+0000","last_clean":"2024-05-11T08:06:47.598256+0000","last_became_active":"2024-05-11T08:06:47.595129+0000","last_became_peered":"2024-05-11T08:06:47.595129+0000","last_unstale":"2024-05-11T08:06:47.598256+0000","last_undegraded":"2024-05-11T08:06:47.598256+0000","last_fullsized":"2024-05-11T08:06:47.598256+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:38:12.898297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.580042+0000","last_change":"2024-05-11T08:06:47.580042+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.580042+0000","last_undegraded":"2024-05-11T08:06:47.580042+0000","last_fullsized":"2024-05-11T08:06:47.580042+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":14,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T08:06:49.613084+0000","last_change":"2024-05-11T08:06:47.601435+0000","last_active":"2024-05-11T08:06:49.613084+0000","last_peered":"2024-05-11T08:06:49.613084+0000","last_clean":"2024-05-11T08:06:49.613084+0000","last_became_active":"2024-05-11T08:06:47.601171+0000","last_became_peered":"2024-05-11T08:06:47.601171+0000","last_unstale":"2024-05-11T08:06:49.613084+0000","last_undegraded":"2024-05-11T08:06:49.613084+0000","last_fullsized":"2024-05-11T08:06:49.613084+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:39:48.026570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.585245+0000","last_change":"2024-05-11T08:06:47.585245+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.585245+0000","last_undegraded":"2024-05-11T08:06:47.585245+0000","last_fullsized":"2024-05-11T08:06:47.585245+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T08:06:47.596881+0000","last_change":"2024-05-11T08:06:47.588756+0000","last_active":"2024-05-11T08:06:47.596881+0000","last_peered":"2024-05-11T08:06:47.596881+0000","last_clean":"2024-05-11T08:06:47.596881+0000","last_became_active":"2024-05-11T08:06:47.588460+0000","last_became_peered":"2024-05-11T08:06:47.588460+0000","last_unstale":"2024-05-11T08:06:47.596881+0000","last_undegraded":"2024-05-11T08:06:47.596881+0000","last_fullsized":"2024-05-11T08:06:47.596881+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:38:12.898297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.584127+0000","last_change":"2024-05-11T08:06:47.584127+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.584127+0000","last_undegraded":"2024-05-11T08:06:47.584127+0000","last_fullsized":"2024-05-11T08:06:47.584127+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"41'1","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T08:06:49.614712+0000","last_change":"2024-05-11T08:06:47.598474+0000","last_active":"2024-05-11T08:06:49.614712+0000","last_peered":"2024-05-11T08:06:49.614712+0000","last_clean":"2024-05-11T08:06:49.614712+0000","last_became_active":"2024-05-11T08:06:47.598238+0000","last_became_peered":"2024-05-11T08:06:47.598238+0000","last_unstale":"2024-05-11T08:06:49.614712+0000","last_undegraded":"2024-05-11T08:06:49.614712+0000","last_fullsized":"2024-05-11T08:06:49.614712+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:06:17.262543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.579855+0000","last_change":"2024-05-11T08:06:47.579855+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.579855+0000","last_undegraded":"2024-05-11T08:06:47.579855+0000","last_fullsized":"2024-05-11T08:06:47.579855+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T08:06:47.604460+0000","last_change":"2024-05-11T08:06:47.599942+0000","last_active":"2024-05-11T08:06:47.604460+0000","last_peered":"2024-05-11T08:06:47.604460+0000","last_clean":"2024-05-11T08:06:47.604460+0000","last_became_active":"2024-05-11T08:06:47.599513+0000","last_became_peered":"2024-05-11T08:06:47.599513+0000","last_unstale":"2024-05-11T08:06:47.604460+0000","last_undegraded":"2024-05-11T08:06:47.604460+0000","last_fullsized":"2024-05-11T08:06:47.604460+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:56:21.008947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.584668+0000","last_change":"2024-05-11T08:06:47.584668+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.584668+0000","last_undegraded":"2024-05-11T08:06:47.584668+0000","last_fullsized":"2024-05-11T08:06:47.584668+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T08:06:47.604497+0000","last_change":"2024-05-11T08:06:47.600002+0000","last_active":"2024-05-11T08:06:47.604497+0000","last_peered":"2024-05-11T08:06:47.604497+0000","last_clean":"2024-05-11T08:06:47.604497+0000","last_became_active":"2024-05-11T08:06:47.599594+0000","last_became_peered":"2024-05-11T08:06:47.599594+0000","last_unstale":"2024-05-11T08:06:47.604497+0000","last_undegraded":"2024-05-11T08:06:47.604497+0000","last_fullsized":"2024-05-11T08:06:47.604497+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:38:12.898297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.584762+0000","last_change":"2024-05-11T08:06:47.584762+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.584762+0000","last_undegraded":"2024-05-11T08:06:47.584762+0000","last_fullsized":"2024-05-11T08:06:47.584762+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T08:06:47.602973+0000","last_change":"2024-05-11T08:06:47.600101+0000","last_active":"2024-05-11T08:06:47.602973+0000","last_peered":"2024-05-11T08:06:47.602973+0000","last_clean":"2024-05-11T08:06:47.602973+0000","last_became_active":"2024-05-11T08:06:47.599765+0000","last_became_peered":"2024-05-11T08:06:47.599765+0000","last_unstale":"2024-05-11T08:06:47.602973+0000","last_undegraded":"2024-05-11T08:06:47.602973+0000","last_fullsized":"2024-05-11T08:06:47.602973+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:27:11.749333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.585048+0000","last_change":"2024-05-11T08:06:47.585048+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.585048+0000","last_undegraded":"2024-05-11T08:06:47.585048+0000","last_fullsized":"2024-05-11T08:06:47.585048+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T08:06:47.596399+0000","last_change":"2024-05-11T08:06:47.592149+0000","last_active":"2024-05-11T08:06:47.596399+0000","last_peered":"2024-05-11T08:06:47.596399+0000","last_clean":"2024-05-11T08:06:47.596399+0000","last_became_active":"2024-05-11T08:06:47.591790+0000","last_became_peered":"2024-05-11T08:06:47.591790+0000","last_unstale":"2024-05-11T08:06:47.596399+0000","last_undegraded":"2024-05-11T08:06:47.596399+0000","last_fullsized":"2024-05-11T08:06:47.596399+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:08:22.364810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.583351+0000","last_change":"2024-05-11T08:06:47.583351+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.583351+0000","last_undegraded":"2024-05-11T08:06:47.583351+0000","last_fullsized":"2024-05-11T08:06:47.583351+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T08:06:47.591117+0000","last_change":"2024-05-11T08:06:47.589050+0000","last_active":"2024-05-11T08:06:47.591117+0000","last_peered":"2024-05-11T08:06:47.591117+0000","last_clean":"2024-05-11T08:06:47.591117+0000","last_became_active":"2024-05-11T08:06:47.588780+0000","last_became_peered":"2024-05-11T08:06:47.588780+0000","last_unstale":"2024-05-11T08:06:47.591117+0000","last_undegraded":"2024-05-11T08:06:47.591117+0000","last_fullsized":"2024-05-11T08:06:47.591117+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:58:03.492050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.580419+0000","last_change":"2024-05-11T08:06:47.580419+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.580419+0000","last_undegraded":"2024-05-11T08:06:47.580419+0000","last_fullsized":"2024-05-11T08:06:47.580419+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T08:06:47.602453+0000","last_change":"2024-05-11T08:06:47.597382+0000","last_active":"2024-05-11T08:06:47.602453+0000","last_peered":"2024-05-11T08:06:47.602453+0000","last_clean":"2024-05-11T08:06:47.602453+0000","last_became_active":"2024-05-11T08:06:47.597062+0000","last_became_peered":"2024-05-11T08:06:47.597062+0000","last_unstale":"2024-05-11T08:06:47.602453+0000","last_undegraded":"2024-05-11T08:06:47.602453+0000","last_fullsized":"2024-05-11T08:06:47.602453+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:12:27.297983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":142,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T08:06:47.587680+0000","last_change":"2024-05-11T08:05:45.772368+0000","last_active":"2024-05-11T08:06:47.587680+0000","last_peered":"2024-05-11T08:06:47.587680+0000","last_clean":"2024-05-11T08:06:47.587680+0000","last_became_active":"2024-05-11T08:05:45.771373+0000","last_became_peered":"2024-05-11T08:05:45.771373+0000","last_unstale":"2024-05-11T08:06:47.587680+0000","last_undegraded":"2024-05-11T08:06:47.587680+0000","last_fullsized":"2024-05-11T08:06:47.587680+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:44.616617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:44.616617+0000","last_clean_scrub_stamp":"2024-05-11T08:05:44.616617+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:02:08.078748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":70,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509457,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28032,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343808,"statfs":{"total":96636764160,"available":96608059392,"internally_reserved":0,"allocated":1036288,"data_stored":776838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509457,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27452,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344388,"statfs":{"total":96636764160,"available":96608653312,"internally_reserved":0,"allocated":442368,"data_stored":186470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509457,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27440,"kb_used_data":420,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":430080,"data_stored":179803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509457,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27440,"kb_used_data":420,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":430080,"data_stored":179803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509458,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27440,"kb_used_data":420,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":430080,"data_stored":179803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509458,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27440,"kb_used_data":420,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":430080,"data_stored":179803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509457,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28020,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343820,"statfs":{"total":96636764160,"available":96608071680,"internally_reserved":0,"allocated":1024000,"data_stored":770171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574866,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27480,"kb_used_data":460,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344360,"statfs":{"total":96636764160,"available":96608624640,"internally_reserved":0,"allocated":471040,"data_stored":200399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-11T08:06:51.954 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-11T08:06:52.092 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.091+0000 7f2b6d124640 1 Processor -- start 2024-05-11T08:06:52.092 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.091+0000 7f2b6d124640 1 -- start start 2024-05-11T08:06:52.095 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.092+0000 7f2b6d124640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2b6815f220 0x7f2b680d1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:52.095 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.092+0000 7f2b6d124640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2b680d1a50 con 0x7f2b680b74c0 2024-05-11T08:06:52.096 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.092+0000 7f2b6d124640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2b680d27c0 con 0x7f2b680cdc90 2024-05-11T08:06:52.096 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.092+0000 7f2b6d124640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f2b6815f600 con 0x7f2b6815f220 2024-05-11T08:06:52.096 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.092+0000 7f2b6659c640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f2b680b74c0 0x7f2b680cd580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:55992/0 (socket says 172.21.15.167:55992) 2024-05-11T08:06:52.096 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.092+0000 7f2b6659c640 1 -- 172.21.15.167:0/4060106153 learned_addr learned my addr 172.21.15.167:0/4060106153 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T08:06:52.096 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.093+0000 7f2b66d9d640 1 --2- 172.21.15.167:0/4060106153 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2b6815f220 0x7f2b680d1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:52.096 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.093+0000 7f2b65d9b640 1 -- 172.21.15.167:0/4060106153 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3296167072 0 0) 0x7f2b5c001110 con 0x7f2b680b74c0 2024-05-11T08:06:52.096 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.093+0000 7f2b65d9b640 1 -- 172.21.15.167:0/4060106153 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2b50003520 con 0x7f2b680b74c0 2024-05-11T08:06:52.096 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.093+0000 7f2b65d9b640 1 -- 172.21.15.167:0/4060106153 <== mon.0 v1:172.21.15.167:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2856248866 0 0) 0x7f2b50003520 con 0x7f2b680b74c0 2024-05-11T08:06:52.096 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.093+0000 7f2b65d9b640 1 -- 172.21.15.167:0/4060106153 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2b6815f220 msgr2=0x7f2b680d1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:52.096 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.093+0000 7f2b65d9b640 1 --2- 172.21.15.167:0/4060106153 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2b6815f220 0x7f2b680d1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:52.096 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.093+0000 7f2b65d9b640 1 -- 172.21.15.167:0/4060106153 >> v1:172.21.15.204:6789/0 conn(0x7f2b680cdc90 legacy=0x7f2b680cf8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:52.096 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.093+0000 7f2b65d9b640 1 -- 172.21.15.167:0/4060106153 --> v1:172.21.15.167:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b680cee00 con 0x7f2b680b74c0 2024-05-11T08:06:52.097 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.093+0000 7f2b65d9b640 1 -- 172.21.15.167:0/4060106153 <== mon.0 v1:172.21.15.167:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3602277846 0 0) 0x7f2b5c0018c0 con 0x7f2b680b74c0 2024-05-11T08:06:52.097 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.093+0000 7f2b65d9b640 1 -- 172.21.15.167:0/4060106153 >> v1:172.21.15.167:6789/0 conn(0x7f2b680b74c0 legacy=0x7f2b680cd580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:52.097 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.093+0000 7f2b65d9b640 1 --2- 172.21.15.167:0/4060106153 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2b50003a30 0x7f2b50003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:52.097 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.093+0000 7f2b65d9b640 1 --2- 172.21.15.167:0/4060106153 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2b6815f220 0x7f2b50004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:52.097 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.093+0000 7f2b65d9b640 1 --2- 172.21.15.167:0/4060106153 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2b50004d40 0x7f2b500051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:52.097 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.093+0000 7f2b65d9b640 1 -- 172.21.15.167:0/4060106153 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2b5000bac0 con 0x7f2b50003a30 2024-05-11T08:06:52.097 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.093+0000 7f2b65d9b640 1 -- 172.21.15.167:0/4060106153 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f2b5000bc70 con 0x7f2b50004d40 2024-05-11T08:06:52.097 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.093+0000 7f2b65d9b640 1 -- 172.21.15.167:0/4060106153 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f2b5000be70 con 0x7f2b6815f220 2024-05-11T08:06:52.097 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.093+0000 7f2b66d9d640 1 --2- 172.21.15.167:0/4060106153 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2b50003a30 0x7f2b50003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:52.097 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.093+0000 7f2b6659c640 1 --2- 172.21.15.167:0/4060106153 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2b6815f220 0x7f2b50004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:52.097 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.093+0000 7f2b6759e640 1 --2- 172.21.15.167:0/4060106153 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2b50004d40 0x7f2b500051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:52.097 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.093+0000 7f2b66d9d640 1 -- 172.21.15.167:0/4060106153 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2b6815f220 msgr2=0x7f2b50004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:52.097 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.093+0000 7f2b66d9d640 1 --2- 172.21.15.167:0/4060106153 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2b6815f220 0x7f2b50004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:52.097 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.093+0000 7f2b66d9d640 1 -- 172.21.15.167:0/4060106153 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2b50004d40 msgr2=0x7f2b500051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:52.097 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.093+0000 7f2b66d9d640 1 --2- 172.21.15.167:0/4060106153 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2b50004d40 0x7f2b500051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:52.098 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.093+0000 7f2b66d9d640 1 -- 172.21.15.167:0/4060106153 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b50003520 con 0x7f2b50003a30 2024-05-11T08:06:52.098 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.093+0000 7f2b6659c640 1 --2- 172.21.15.167:0/4060106153 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2b6815f220 0x7f2b50004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T08:06:52.098 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.093+0000 7f2b6759e640 1 --2- 172.21.15.167:0/4060106153 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2b50004d40 0x7f2b500051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T08:06:52.098 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.094+0000 7f2b66d9d640 1 --2- 172.21.15.167:0/4060106153 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2b50003a30 0x7f2b50003e50 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f2b58009ac0 tx=0x7f2b50005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cde7b97898e7aa07 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:52.098 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.094+0000 7f2b65d9b640 1 -- 172.21.15.167:0/4060106153 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b58036070 con 0x7f2b50003a30 2024-05-11T08:06:52.098 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.094+0000 7f2b65d9b640 1 -- 172.21.15.167:0/4060106153 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b58031400 con 0x7f2b50003a30 2024-05-11T08:06:52.098 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.094+0000 7f2b65d9b640 1 -- 172.21.15.167:0/4060106153 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b58009690 con 0x7f2b50003a30 2024-05-11T08:06:52.098 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.095+0000 7f2b6d124640 1 -- 172.21.15.167:0/4060106153 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2b50003a30 msgr2=0x7f2b50003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:52.098 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.095+0000 7f2b6d124640 1 --2- 172.21.15.167:0/4060106153 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2b50003a30 0x7f2b50003e50 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f2b58009ac0 tx=0x7f2b50005830 comp rx=0 tx=0).stop 2024-05-11T08:06:52.098 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.095+0000 7f2b6759e640 1 -- 172.21.15.167:0/4060106153 reap_dead start 2024-05-11T08:06:52.098 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.095+0000 7f2b6d124640 1 -- 172.21.15.167:0/4060106153 shutdown_connections 2024-05-11T08:06:52.099 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.095+0000 7f2b6d124640 1 -- 172.21.15.167:0/4060106153 >> 172.21.15.167:0/4060106153 conn(0x7f2b680b4ae0 msgr2=0x7f2b680b4ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:52.099 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.095+0000 7f2b6d124640 1 -- 172.21.15.167:0/4060106153 shutdown_connections 2024-05-11T08:06:52.099 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.095+0000 7f2b6d124640 1 -- 172.21.15.167:0/4060106153 wait complete. 2024-05-11T08:06:52.101 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.096+0000 7f2b6d124640 1 Processor -- start 2024-05-11T08:06:52.101 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.096+0000 7f2b6d124640 1 -- start start 2024-05-11T08:06:52.101 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.097+0000 7f2b6d124640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2b50003a30 0x7f2b680cbdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:52.101 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.097+0000 7f2b6d124640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2b68070e20 0x7f2b68156680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:52.101 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.097+0000 7f2b6d124640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2b681f5a00 0x7f2b68157f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:52.102 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.097+0000 7f2b6d124640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2b680ce2e0 con 0x7f2b68070e20 2024-05-11T08:06:52.102 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.097+0000 7f2b6d124640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f2b680cf450 con 0x7f2b50003a30 2024-05-11T08:06:52.102 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.097+0000 7f2b6d124640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f2b680d2400 con 0x7f2b681f5a00 2024-05-11T08:06:52.102 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.097+0000 7f2b6659c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2b68070e20 0x7f2b68156680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:52.102 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.097+0000 7f2b6659c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2b68070e20 0x7f2b68156680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:44064/0 (socket says 172.21.15.167:44064) 2024-05-11T08:06:52.102 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.097+0000 7f2b6659c640 1 -- 172.21.15.167:0/1849371229 learned_addr learned my addr 172.21.15.167:0/1849371229 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:52.102 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.097+0000 7f2b6759e640 1 --2- 172.21.15.167:0/1849371229 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2b681f5a00 0x7f2b68157f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:52.102 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.097+0000 7f2b66d9d640 1 --2- 172.21.15.167:0/1849371229 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2b50003a30 0x7f2b680cbdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:52.102 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.097+0000 7f2b6659c640 1 -- 172.21.15.167:0/1849371229 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2b681f5a00 msgr2=0x7f2b68157f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:52.102 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.097+0000 7f2b6659c640 1 --2- 172.21.15.167:0/1849371229 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2b681f5a00 0x7f2b68157f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:52.102 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.097+0000 7f2b6659c640 1 -- 172.21.15.167:0/1849371229 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2b50003a30 msgr2=0x7f2b680cbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:52.102 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.097+0000 7f2b6659c640 1 --2- 172.21.15.167:0/1849371229 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2b50003a30 0x7f2b680cbdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:52.102 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.097+0000 7f2b6659c640 1 -- 172.21.15.167:0/1849371229 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b681f5240 con 0x7f2b68070e20 2024-05-11T08:06:52.102 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.098+0000 7f2b6659c640 1 --2- 172.21.15.167:0/1849371229 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2b68070e20 0x7f2b68156680 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c001d60 tx=0x7f2b5c007790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:52.102 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.098+0000 7f2b4f7fe640 1 -- 172.21.15.167:0/1849371229 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b5c00e920 con 0x7f2b68070e20 2024-05-11T08:06:52.102 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.098+0000 7f2b4f7fe640 1 -- 172.21.15.167:0/1849371229 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b5c018070 con 0x7f2b68070e20 2024-05-11T08:06:52.103 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.098+0000 7f2b4f7fe640 1 -- 172.21.15.167:0/1849371229 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b5c00ece0 con 0x7f2b68070e20 2024-05-11T08:06:52.103 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.098+0000 7f2b6d124640 1 -- 172.21.15.167:0/1849371229 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b680716a0 con 0x7f2b68070e20 2024-05-11T08:06:52.103 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.098+0000 7f2b6d124640 1 -- 172.21.15.167:0/1849371229 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b681f54e0 con 0x7f2b68070e20 2024-05-11T08:06:52.103 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.099+0000 7f2b6d124640 1 -- 172.21.15.167:0/1849371229 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b68069bc0 con 0x7f2b68070e20 2024-05-11T08:06:52.103 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.099+0000 7f2b4f7fe640 1 -- 172.21.15.167:0/1849371229 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f2b5c01d460 con 0x7f2b68070e20 2024-05-11T08:06:52.103 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.100+0000 7f2b4f7fe640 1 --2- 172.21.15.167:0/1849371229 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f2b540762d0 0x7f2b54078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:52.103 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.100+0000 7f2b66d9d640 1 --2- 172.21.15.167:0/1849371229 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f2b540762d0 0x7f2b54078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:52.103 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.100+0000 7f2b4f7fe640 1 -- 172.21.15.167:0/1849371229 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6101+0+0 (secure 0 0 0) 0x7f2b5c097760 con 0x7f2b68070e20 2024-05-11T08:06:52.103 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.101+0000 7f2b66d9d640 1 --2- 172.21.15.167:0/1849371229 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f2b540762d0 0x7f2b54078790 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f2b58033000 tx=0x7f2b58008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:52.104 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.104+0000 7f2b4f7fe640 1 -- 172.21.15.167:0/1849371229 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f2b680d1a50 con 0x7f2b68070e20 2024-05-11T08:06:52.286 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.286+0000 7f2b6d124640 1 -- 172.21.15.167:0/1849371229 --> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2b680c6a60 con 0x7f2b540762d0 2024-05-11T08:06:52.292 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.292+0000 7f2b4f7fe640 1 -- 172.21.15.167:0/1849371229 <== mgr.4113 v2:172.21.15.167:6832/253569496 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247131 (secure 0 0 0) 0x7f2b680c6a60 con 0x7f2b540762d0 2024-05-11T08:06:52.295 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T08:06:52.296 INFO:teuthology.orchestra.run.smithi167.stderr:dumped all 2024-05-11T08:06:52.298 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.296+0000 7f2b6d124640 1 -- 172.21.15.167:0/1849371229 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f2b540762d0 msgr2=0x7f2b54078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:52.298 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.296+0000 7f2b6d124640 1 --2- 172.21.15.167:0/1849371229 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f2b540762d0 0x7f2b54078790 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f2b58033000 tx=0x7f2b58008030 comp rx=0 tx=0).stop 2024-05-11T08:06:52.298 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.296+0000 7f2b6d124640 1 -- 172.21.15.167:0/1849371229 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2b68070e20 msgr2=0x7f2b68156680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:52.298 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.296+0000 7f2b6d124640 1 --2- 172.21.15.167:0/1849371229 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2b68070e20 0x7f2b68156680 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c001d60 tx=0x7f2b5c007790 comp rx=0 tx=0).stop 2024-05-11T08:06:52.298 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.296+0000 7f2b6d124640 1 -- 172.21.15.167:0/1849371229 shutdown_connections 2024-05-11T08:06:52.298 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.296+0000 7f2b6d124640 1 --2- 172.21.15.167:0/1849371229 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f2b540762d0 0x7f2b54078790 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:52.298 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.296+0000 7f2b6d124640 1 --2- 172.21.15.167:0/1849371229 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2b681f5a00 0x7f2b68157f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:52.298 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.296+0000 7f2b6d124640 1 --2- 172.21.15.167:0/1849371229 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2b68070e20 0x7f2b68156680 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:52.298 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.296+0000 7f2b6d124640 1 --2- 172.21.15.167:0/1849371229 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2b50003a30 0x7f2b680cbdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:52.298 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.296+0000 7f2b6d124640 1 -- 172.21.15.167:0/1849371229 >> 172.21.15.167:0/1849371229 conn(0x7f2b680b4ae0 msgr2=0x7f2b680c6c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:52.298 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.296+0000 7f2b6d124640 1 -- 172.21.15.167:0/1849371229 shutdown_connections 2024-05-11T08:06:52.298 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.296+0000 7f2b6d124640 1 -- 172.21.15.167:0/1849371229 wait complete. 2024-05-11T08:06:52.307 INFO:teuthology.orchestra.run.smithi167.stdout:{"pg_ready":true,"pg_map":{"version":70,"stamp":"2024-05-11T08:06:51.928061+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590438,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":104,"num_write_kb":1629,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":46,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220744,"kb_used_data":4584,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753976,"statfs":{"total":773094113280,"available":772868071424,"internally_reserved":0,"allocated":4694016,"data_stored":2653090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":70,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.351869"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.580160+0000","last_change":"2024-05-11T08:06:47.580160+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.580160+0000","last_undegraded":"2024-05-11T08:06:47.580160+0000","last_fullsized":"2024-05-11T08:06:47.580160+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.580981+0000","last_change":"2024-05-11T08:06:47.580981+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.580981+0000","last_undegraded":"2024-05-11T08:06:47.580981+0000","last_fullsized":"2024-05-11T08:06:47.580981+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.587939+0000","last_change":"2024-05-11T08:06:47.587939+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.587939+0000","last_undegraded":"2024-05-11T08:06:47.587939+0000","last_fullsized":"2024-05-11T08:06:47.587939+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T08:06:49.613242+0000","last_change":"2024-05-11T08:06:48.616894+0000","last_active":"2024-05-11T08:06:49.613242+0000","last_peered":"2024-05-11T08:06:49.613242+0000","last_clean":"2024-05-11T08:06:49.613242+0000","last_became_active":"2024-05-11T08:06:48.616597+0000","last_became_peered":"2024-05-11T08:06:48.616597+0000","last_unstale":"2024-05-11T08:06:49.613242+0000","last_undegraded":"2024-05-11T08:06:49.613242+0000","last_fullsized":"2024-05-11T08:06:49.613242+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:47:37.702924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T08:06:49.613143+0000","last_change":"2024-05-11T08:06:48.616535+0000","last_active":"2024-05-11T08:06:49.613143+0000","last_peered":"2024-05-11T08:06:49.613143+0000","last_clean":"2024-05-11T08:06:49.613143+0000","last_became_active":"2024-05-11T08:06:48.616283+0000","last_became_peered":"2024-05-11T08:06:48.616283+0000","last_unstale":"2024-05-11T08:06:49.613143+0000","last_undegraded":"2024-05-11T08:06:49.613143+0000","last_fullsized":"2024-05-11T08:06:49.613143+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:12:46.331254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.586507+0000","last_change":"2024-05-11T08:06:47.586507+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.586507+0000","last_undegraded":"2024-05-11T08:06:47.586507+0000","last_fullsized":"2024-05-11T08:06:47.586507+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.580887+0000","last_change":"2024-05-11T08:06:47.580887+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.580887+0000","last_undegraded":"2024-05-11T08:06:47.580887+0000","last_fullsized":"2024-05-11T08:06:47.580887+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.579778+0000","last_change":"2024-05-11T08:06:47.579778+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.579778+0000","last_undegraded":"2024-05-11T08:06:47.579778+0000","last_fullsized":"2024-05-11T08:06:47.579778+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.585551+0000","last_change":"2024-05-11T08:06:47.585551+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.585551+0000","last_undegraded":"2024-05-11T08:06:47.585551+0000","last_fullsized":"2024-05-11T08:06:47.585551+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.579247+0000","last_change":"2024-05-11T08:06:47.579247+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.579247+0000","last_undegraded":"2024-05-11T08:06:47.579247+0000","last_fullsized":"2024-05-11T08:06:47.579247+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.580547+0000","last_change":"2024-05-11T08:06:47.580547+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.580547+0000","last_undegraded":"2024-05-11T08:06:47.580547+0000","last_fullsized":"2024-05-11T08:06:47.580547+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.584544+0000","last_change":"2024-05-11T08:06:47.584544+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.584544+0000","last_undegraded":"2024-05-11T08:06:47.584544+0000","last_fullsized":"2024-05-11T08:06:47.584544+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.580055+0000","last_change":"2024-05-11T08:06:47.580055+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.580055+0000","last_undegraded":"2024-05-11T08:06:47.580055+0000","last_fullsized":"2024-05-11T08:06:47.580055+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.585515+0000","last_change":"2024-05-11T08:06:47.585515+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.585515+0000","last_undegraded":"2024-05-11T08:06:47.585515+0000","last_fullsized":"2024-05-11T08:06:47.585515+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.579888+0000","last_change":"2024-05-11T08:06:47.579888+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.579888+0000","last_undegraded":"2024-05-11T08:06:47.579888+0000","last_fullsized":"2024-05-11T08:06:47.579888+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.585758+0000","last_change":"2024-05-11T08:06:47.585758+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.585758+0000","last_undegraded":"2024-05-11T08:06:47.585758+0000","last_fullsized":"2024-05-11T08:06:47.585758+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.587418+0000","last_change":"2024-05-11T08:06:47.587418+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.587418+0000","last_undegraded":"2024-05-11T08:06:47.587418+0000","last_fullsized":"2024-05-11T08:06:47.587418+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.588413+0000","last_change":"2024-05-11T08:06:47.588413+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.588413+0000","last_undegraded":"2024-05-11T08:06:47.588413+0000","last_fullsized":"2024-05-11T08:06:47.588413+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T08:06:49.613351+0000","last_change":"2024-05-11T08:06:48.606813+0000","last_active":"2024-05-11T08:06:49.613351+0000","last_peered":"2024-05-11T08:06:49.613351+0000","last_clean":"2024-05-11T08:06:49.613351+0000","last_became_active":"2024-05-11T08:06:48.606548+0000","last_became_peered":"2024-05-11T08:06:48.606548+0000","last_unstale":"2024-05-11T08:06:49.613351+0000","last_undegraded":"2024-05-11T08:06:49.613351+0000","last_fullsized":"2024-05-11T08:06:49.613351+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:40:53.505484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.587907+0000","last_change":"2024-05-11T08:06:47.587907+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.587907+0000","last_undegraded":"2024-05-11T08:06:47.587907+0000","last_fullsized":"2024-05-11T08:06:47.587907+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.579095+0000","last_change":"2024-05-11T08:06:47.579095+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.579095+0000","last_undegraded":"2024-05-11T08:06:47.579095+0000","last_fullsized":"2024-05-11T08:06:47.579095+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.583273+0000","last_change":"2024-05-11T08:06:47.583273+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.583273+0000","last_undegraded":"2024-05-11T08:06:47.583273+0000","last_fullsized":"2024-05-11T08:06:47.583273+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.583780+0000","last_change":"2024-05-11T08:06:47.583780+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.583780+0000","last_undegraded":"2024-05-11T08:06:47.583780+0000","last_fullsized":"2024-05-11T08:06:47.583780+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T08:06:49.613144+0000","last_change":"2024-05-11T08:06:48.615211+0000","last_active":"2024-05-11T08:06:49.613144+0000","last_peered":"2024-05-11T08:06:49.613144+0000","last_clean":"2024-05-11T08:06:49.613144+0000","last_became_active":"2024-05-11T08:06:48.614902+0000","last_became_peered":"2024-05-11T08:06:48.614902+0000","last_unstale":"2024-05-11T08:06:49.613144+0000","last_undegraded":"2024-05-11T08:06:49.613144+0000","last_fullsized":"2024-05-11T08:06:49.613144+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:20:23.234025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.589116+0000","last_change":"2024-05-11T08:06:47.589116+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.589116+0000","last_undegraded":"2024-05-11T08:06:47.589116+0000","last_fullsized":"2024-05-11T08:06:47.589116+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.586184+0000","last_change":"2024-05-11T08:06:47.586184+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.586184+0000","last_undegraded":"2024-05-11T08:06:47.586184+0000","last_fullsized":"2024-05-11T08:06:47.586184+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.587324+0000","last_change":"2024-05-11T08:06:47.587324+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.587324+0000","last_undegraded":"2024-05-11T08:06:47.587324+0000","last_fullsized":"2024-05-11T08:06:47.587324+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.578757+0000","last_change":"2024-05-11T08:06:47.578757+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.578757+0000","last_undegraded":"2024-05-11T08:06:47.578757+0000","last_fullsized":"2024-05-11T08:06:47.578757+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.588841+0000","last_change":"2024-05-11T08:06:47.588841+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.588841+0000","last_undegraded":"2024-05-11T08:06:47.588841+0000","last_fullsized":"2024-05-11T08:06:47.588841+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.585718+0000","last_change":"2024-05-11T08:06:47.585718+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.585718+0000","last_undegraded":"2024-05-11T08:06:47.585718+0000","last_fullsized":"2024-05-11T08:06:47.585718+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.583214+0000","last_change":"2024-05-11T08:06:47.583214+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.583214+0000","last_undegraded":"2024-05-11T08:06:47.583214+0000","last_fullsized":"2024-05-11T08:06:47.583214+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.587217+0000","last_change":"2024-05-11T08:06:47.587217+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.587217+0000","last_undegraded":"2024-05-11T08:06:47.587217+0000","last_fullsized":"2024-05-11T08:06:47.587217+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.584847+0000","last_change":"2024-05-11T08:06:47.584847+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.584847+0000","last_undegraded":"2024-05-11T08:06:47.584847+0000","last_fullsized":"2024-05-11T08:06:47.584847+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T08:06:47.594702+0000","last_change":"2024-05-11T08:06:47.591074+0000","last_active":"2024-05-11T08:06:47.594702+0000","last_peered":"2024-05-11T08:06:47.594702+0000","last_clean":"2024-05-11T08:06:47.594702+0000","last_became_active":"2024-05-11T08:06:47.590747+0000","last_became_peered":"2024-05-11T08:06:47.590747+0000","last_unstale":"2024-05-11T08:06:47.594702+0000","last_undegraded":"2024-05-11T08:06:47.594702+0000","last_fullsized":"2024-05-11T08:06:47.594702+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:11:53.548277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.585679+0000","last_change":"2024-05-11T08:06:47.585679+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.585679+0000","last_undegraded":"2024-05-11T08:06:47.585679+0000","last_fullsized":"2024-05-11T08:06:47.585679+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T08:06:47.601797+0000","last_change":"2024-05-11T08:06:47.597547+0000","last_active":"2024-05-11T08:06:47.601797+0000","last_peered":"2024-05-11T08:06:47.601797+0000","last_clean":"2024-05-11T08:06:47.601797+0000","last_became_active":"2024-05-11T08:06:47.597260+0000","last_became_peered":"2024-05-11T08:06:47.597260+0000","last_unstale":"2024-05-11T08:06:47.601797+0000","last_undegraded":"2024-05-11T08:06:47.601797+0000","last_fullsized":"2024-05-11T08:06:47.601797+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:08:22.364810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.584309+0000","last_change":"2024-05-11T08:06:47.584309+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.584309+0000","last_undegraded":"2024-05-11T08:06:47.584309+0000","last_fullsized":"2024-05-11T08:06:47.584309+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T08:06:47.601604+0000","last_change":"2024-05-11T08:06:47.597745+0000","last_active":"2024-05-11T08:06:47.601604+0000","last_peered":"2024-05-11T08:06:47.601604+0000","last_clean":"2024-05-11T08:06:47.601604+0000","last_became_active":"2024-05-11T08:06:47.597472+0000","last_became_peered":"2024-05-11T08:06:47.597472+0000","last_unstale":"2024-05-11T08:06:47.601604+0000","last_undegraded":"2024-05-11T08:06:47.601604+0000","last_fullsized":"2024-05-11T08:06:47.601604+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:56:21.008947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.580834+0000","last_change":"2024-05-11T08:06:47.580834+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.580834+0000","last_undegraded":"2024-05-11T08:06:47.580834+0000","last_fullsized":"2024-05-11T08:06:47.580834+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T08:06:47.596539+0000","last_change":"2024-05-11T08:06:47.594580+0000","last_active":"2024-05-11T08:06:47.596539+0000","last_peered":"2024-05-11T08:06:47.596539+0000","last_clean":"2024-05-11T08:06:47.596539+0000","last_became_active":"2024-05-11T08:06:47.594315+0000","last_became_peered":"2024-05-11T08:06:47.594315+0000","last_unstale":"2024-05-11T08:06:47.596539+0000","last_undegraded":"2024-05-11T08:06:47.596539+0000","last_fullsized":"2024-05-11T08:06:47.596539+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:56:21.008947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.586542+0000","last_change":"2024-05-11T08:06:47.586542+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.586542+0000","last_undegraded":"2024-05-11T08:06:47.586542+0000","last_fullsized":"2024-05-11T08:06:47.586542+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T08:06:47.602415+0000","last_change":"2024-05-11T08:06:47.600311+0000","last_active":"2024-05-11T08:06:47.602415+0000","last_peered":"2024-05-11T08:06:47.602415+0000","last_clean":"2024-05-11T08:06:47.602415+0000","last_became_active":"2024-05-11T08:06:47.600072+0000","last_became_peered":"2024-05-11T08:06:47.600072+0000","last_unstale":"2024-05-11T08:06:47.602415+0000","last_undegraded":"2024-05-11T08:06:47.602415+0000","last_fullsized":"2024-05-11T08:06:47.602415+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:13:57.487739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.578355+0000","last_change":"2024-05-11T08:06:47.578355+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.578355+0000","last_undegraded":"2024-05-11T08:06:47.578355+0000","last_fullsized":"2024-05-11T08:06:47.578355+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T08:06:47.602942+0000","last_change":"2024-05-11T08:06:47.598857+0000","last_active":"2024-05-11T08:06:47.602942+0000","last_peered":"2024-05-11T08:06:47.602942+0000","last_clean":"2024-05-11T08:06:47.602942+0000","last_became_active":"2024-05-11T08:06:47.598102+0000","last_became_peered":"2024-05-11T08:06:47.598102+0000","last_unstale":"2024-05-11T08:06:47.602942+0000","last_undegraded":"2024-05-11T08:06:47.602942+0000","last_fullsized":"2024-05-11T08:06:47.602942+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:13:57.487739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.583760+0000","last_change":"2024-05-11T08:06:47.583760+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.583760+0000","last_undegraded":"2024-05-11T08:06:47.583760+0000","last_fullsized":"2024-05-11T08:06:47.583760+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T08:06:47.599146+0000","last_change":"2024-05-11T08:06:47.595791+0000","last_active":"2024-05-11T08:06:47.599146+0000","last_peered":"2024-05-11T08:06:47.599146+0000","last_clean":"2024-05-11T08:06:47.599146+0000","last_became_active":"2024-05-11T08:06:47.595551+0000","last_became_peered":"2024-05-11T08:06:47.595551+0000","last_unstale":"2024-05-11T08:06:47.599146+0000","last_undegraded":"2024-05-11T08:06:47.599146+0000","last_fullsized":"2024-05-11T08:06:47.599146+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:00:29.291380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.581488+0000","last_change":"2024-05-11T08:06:47.581488+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.581488+0000","last_undegraded":"2024-05-11T08:06:47.581488+0000","last_fullsized":"2024-05-11T08:06:47.581488+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T08:06:47.596271+0000","last_change":"2024-05-11T08:06:47.590670+0000","last_active":"2024-05-11T08:06:47.596271+0000","last_peered":"2024-05-11T08:06:47.596271+0000","last_clean":"2024-05-11T08:06:47.596271+0000","last_became_active":"2024-05-11T08:06:47.590380+0000","last_became_peered":"2024-05-11T08:06:47.590380+0000","last_unstale":"2024-05-11T08:06:47.596271+0000","last_undegraded":"2024-05-11T08:06:47.596271+0000","last_fullsized":"2024-05-11T08:06:47.596271+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:00:29.291380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T08:06:49.613537+0000","last_change":"2024-05-11T08:06:48.616855+0000","last_active":"2024-05-11T08:06:49.613537+0000","last_peered":"2024-05-11T08:06:49.613537+0000","last_clean":"2024-05-11T08:06:49.613537+0000","last_became_active":"2024-05-11T08:06:48.616287+0000","last_became_peered":"2024-05-11T08:06:48.616287+0000","last_unstale":"2024-05-11T08:06:49.613537+0000","last_undegraded":"2024-05-11T08:06:49.613537+0000","last_fullsized":"2024-05-11T08:06:49.613537+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:31:41.113021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T08:06:47.593461+0000","last_change":"2024-05-11T08:06:47.590630+0000","last_active":"2024-05-11T08:06:47.593461+0000","last_peered":"2024-05-11T08:06:47.593461+0000","last_clean":"2024-05-11T08:06:47.593461+0000","last_became_active":"2024-05-11T08:06:47.590361+0000","last_became_peered":"2024-05-11T08:06:47.590361+0000","last_unstale":"2024-05-11T08:06:47.593461+0000","last_undegraded":"2024-05-11T08:06:47.593461+0000","last_fullsized":"2024-05-11T08:06:47.593461+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:20:42.258886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.584566+0000","last_change":"2024-05-11T08:06:47.584566+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.584566+0000","last_undegraded":"2024-05-11T08:06:47.584566+0000","last_fullsized":"2024-05-11T08:06:47.584566+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T08:06:47.595167+0000","last_change":"2024-05-11T08:06:47.587418+0000","last_active":"2024-05-11T08:06:47.595167+0000","last_peered":"2024-05-11T08:06:47.595167+0000","last_clean":"2024-05-11T08:06:47.595167+0000","last_became_active":"2024-05-11T08:06:47.586715+0000","last_became_peered":"2024-05-11T08:06:47.586715+0000","last_unstale":"2024-05-11T08:06:47.595167+0000","last_undegraded":"2024-05-11T08:06:47.595167+0000","last_fullsized":"2024-05-11T08:06:47.595167+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:56:21.008947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.584770+0000","last_change":"2024-05-11T08:06:47.584770+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.584770+0000","last_undegraded":"2024-05-11T08:06:47.584770+0000","last_fullsized":"2024-05-11T08:06:47.584770+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T08:06:47.597011+0000","last_change":"2024-05-11T08:06:47.594319+0000","last_active":"2024-05-11T08:06:47.597011+0000","last_peered":"2024-05-11T08:06:47.597011+0000","last_clean":"2024-05-11T08:06:47.597011+0000","last_became_active":"2024-05-11T08:06:47.593840+0000","last_became_peered":"2024-05-11T08:06:47.593840+0000","last_unstale":"2024-05-11T08:06:47.597011+0000","last_undegraded":"2024-05-11T08:06:47.597011+0000","last_fullsized":"2024-05-11T08:06:47.597011+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:12:27.297983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.584560+0000","last_change":"2024-05-11T08:06:47.584560+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.584560+0000","last_undegraded":"2024-05-11T08:06:47.584560+0000","last_fullsized":"2024-05-11T08:06:47.584560+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T08:06:47.596789+0000","last_change":"2024-05-11T08:06:47.592968+0000","last_active":"2024-05-11T08:06:47.596789+0000","last_peered":"2024-05-11T08:06:47.596789+0000","last_clean":"2024-05-11T08:06:47.596789+0000","last_became_active":"2024-05-11T08:06:47.592672+0000","last_became_peered":"2024-05-11T08:06:47.592672+0000","last_unstale":"2024-05-11T08:06:47.596789+0000","last_undegraded":"2024-05-11T08:06:47.596789+0000","last_fullsized":"2024-05-11T08:06:47.596789+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:46:05.539312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.585804+0000","last_change":"2024-05-11T08:06:47.585804+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.585804+0000","last_undegraded":"2024-05-11T08:06:47.585804+0000","last_fullsized":"2024-05-11T08:06:47.585804+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T08:06:47.604642+0000","last_change":"2024-05-11T08:06:47.600533+0000","last_active":"2024-05-11T08:06:47.604642+0000","last_peered":"2024-05-11T08:06:47.604642+0000","last_clean":"2024-05-11T08:06:47.604642+0000","last_became_active":"2024-05-11T08:06:47.600226+0000","last_became_peered":"2024-05-11T08:06:47.600226+0000","last_unstale":"2024-05-11T08:06:47.604642+0000","last_undegraded":"2024-05-11T08:06:47.604642+0000","last_fullsized":"2024-05-11T08:06:47.604642+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:49:09.098705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.579895+0000","last_change":"2024-05-11T08:06:47.579895+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.579895+0000","last_undegraded":"2024-05-11T08:06:47.579895+0000","last_fullsized":"2024-05-11T08:06:47.579895+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T08:06:47.601569+0000","last_change":"2024-05-11T08:06:47.597276+0000","last_active":"2024-05-11T08:06:47.601569+0000","last_peered":"2024-05-11T08:06:47.601569+0000","last_clean":"2024-05-11T08:06:47.601569+0000","last_became_active":"2024-05-11T08:06:47.596986+0000","last_became_peered":"2024-05-11T08:06:47.596986+0000","last_unstale":"2024-05-11T08:06:47.601569+0000","last_undegraded":"2024-05-11T08:06:47.601569+0000","last_fullsized":"2024-05-11T08:06:47.601569+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:08:22.364810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.584152+0000","last_change":"2024-05-11T08:06:47.584152+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.584152+0000","last_undegraded":"2024-05-11T08:06:47.584152+0000","last_fullsized":"2024-05-11T08:06:47.584152+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T08:06:47.594681+0000","last_change":"2024-05-11T08:06:47.591052+0000","last_active":"2024-05-11T08:06:47.594681+0000","last_peered":"2024-05-11T08:06:47.594681+0000","last_clean":"2024-05-11T08:06:47.594681+0000","last_became_active":"2024-05-11T08:06:47.590726+0000","last_became_peered":"2024-05-11T08:06:47.590726+0000","last_unstale":"2024-05-11T08:06:47.594681+0000","last_undegraded":"2024-05-11T08:06:47.594681+0000","last_fullsized":"2024-05-11T08:06:47.594681+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:30:44.079909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.580872+0000","last_change":"2024-05-11T08:06:47.580872+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.580872+0000","last_undegraded":"2024-05-11T08:06:47.580872+0000","last_fullsized":"2024-05-11T08:06:47.580872+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T08:06:47.598143+0000","last_change":"2024-05-11T08:06:47.597077+0000","last_active":"2024-05-11T08:06:47.598143+0000","last_peered":"2024-05-11T08:06:47.598143+0000","last_clean":"2024-05-11T08:06:47.598143+0000","last_became_active":"2024-05-11T08:06:47.596753+0000","last_became_peered":"2024-05-11T08:06:47.596753+0000","last_unstale":"2024-05-11T08:06:47.598143+0000","last_undegraded":"2024-05-11T08:06:47.598143+0000","last_fullsized":"2024-05-11T08:06:47.598143+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:19:14.446185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.580505+0000","last_change":"2024-05-11T08:06:47.580505+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.580505+0000","last_undegraded":"2024-05-11T08:06:47.580505+0000","last_fullsized":"2024-05-11T08:06:47.580505+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T08:06:47.596912+0000","last_change":"2024-05-11T08:06:47.592140+0000","last_active":"2024-05-11T08:06:47.596912+0000","last_peered":"2024-05-11T08:06:47.596912+0000","last_clean":"2024-05-11T08:06:47.596912+0000","last_became_active":"2024-05-11T08:06:47.591867+0000","last_became_peered":"2024-05-11T08:06:47.591867+0000","last_unstale":"2024-05-11T08:06:47.596912+0000","last_undegraded":"2024-05-11T08:06:47.596912+0000","last_fullsized":"2024-05-11T08:06:47.596912+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:19:14.446185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.580040+0000","last_change":"2024-05-11T08:06:47.580040+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.580040+0000","last_undegraded":"2024-05-11T08:06:47.580040+0000","last_fullsized":"2024-05-11T08:06:47.580040+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T08:06:47.597734+0000","last_change":"2024-05-11T08:06:47.595937+0000","last_active":"2024-05-11T08:06:47.597734+0000","last_peered":"2024-05-11T08:06:47.597734+0000","last_clean":"2024-05-11T08:06:47.597734+0000","last_became_active":"2024-05-11T08:06:47.595642+0000","last_became_peered":"2024-05-11T08:06:47.595642+0000","last_unstale":"2024-05-11T08:06:47.597734+0000","last_undegraded":"2024-05-11T08:06:47.597734+0000","last_fullsized":"2024-05-11T08:06:47.597734+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:38:12.898297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.585260+0000","last_change":"2024-05-11T08:06:47.585260+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.585260+0000","last_undegraded":"2024-05-11T08:06:47.585260+0000","last_fullsized":"2024-05-11T08:06:47.585260+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"40'1","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T08:06:49.613353+0000","last_change":"2024-05-11T08:06:47.599057+0000","last_active":"2024-05-11T08:06:49.613353+0000","last_peered":"2024-05-11T08:06:49.613353+0000","last_clean":"2024-05-11T08:06:49.613353+0000","last_became_active":"2024-05-11T08:06:47.598647+0000","last_became_peered":"2024-05-11T08:06:47.598647+0000","last_unstale":"2024-05-11T08:06:49.613353+0000","last_undegraded":"2024-05-11T08:06:49.613353+0000","last_fullsized":"2024-05-11T08:06:49.613353+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:40:41.542604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T08:06:49.613419+0000","last_change":"2024-05-11T08:06:48.606563+0000","last_active":"2024-05-11T08:06:49.613419+0000","last_peered":"2024-05-11T08:06:49.613419+0000","last_clean":"2024-05-11T08:06:49.613419+0000","last_became_active":"2024-05-11T08:06:48.606287+0000","last_became_peered":"2024-05-11T08:06:48.606287+0000","last_unstale":"2024-05-11T08:06:49.613419+0000","last_undegraded":"2024-05-11T08:06:49.613419+0000","last_fullsized":"2024-05-11T08:06:49.613419+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:23:43.027012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T08:06:47.596598+0000","last_change":"2024-05-11T08:06:47.592090+0000","last_active":"2024-05-11T08:06:47.596598+0000","last_peered":"2024-05-11T08:06:47.596598+0000","last_clean":"2024-05-11T08:06:47.596598+0000","last_became_active":"2024-05-11T08:06:47.591813+0000","last_became_peered":"2024-05-11T08:06:47.591813+0000","last_unstale":"2024-05-11T08:06:47.596598+0000","last_undegraded":"2024-05-11T08:06:47.596598+0000","last_fullsized":"2024-05-11T08:06:47.596598+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:27:11.749333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.585274+0000","last_change":"2024-05-11T08:06:47.585274+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.585274+0000","last_undegraded":"2024-05-11T08:06:47.585274+0000","last_fullsized":"2024-05-11T08:06:47.585274+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"40'2","reported_seq":16,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T08:06:49.613224+0000","last_change":"2024-05-11T08:06:47.600614+0000","last_active":"2024-05-11T08:06:49.613224+0000","last_peered":"2024-05-11T08:06:49.613224+0000","last_clean":"2024-05-11T08:06:49.613224+0000","last_became_active":"2024-05-11T08:06:47.600104+0000","last_became_peered":"2024-05-11T08:06:47.600104+0000","last_unstale":"2024-05-11T08:06:49.613224+0000","last_undegraded":"2024-05-11T08:06:49.613224+0000","last_fullsized":"2024-05-11T08:06:49.613224+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:25:19.853657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.585232+0000","last_change":"2024-05-11T08:06:47.585232+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.585232+0000","last_undegraded":"2024-05-11T08:06:47.585232+0000","last_fullsized":"2024-05-11T08:06:47.585232+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":14,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T08:06:49.613139+0000","last_change":"2024-05-11T08:06:47.597958+0000","last_active":"2024-05-11T08:06:49.613139+0000","last_peered":"2024-05-11T08:06:49.613139+0000","last_clean":"2024-05-11T08:06:49.613139+0000","last_became_active":"2024-05-11T08:06:47.597638+0000","last_became_peered":"2024-05-11T08:06:47.597638+0000","last_unstale":"2024-05-11T08:06:49.613139+0000","last_undegraded":"2024-05-11T08:06:49.613139+0000","last_fullsized":"2024-05-11T08:06:49.613139+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:02:41.089263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.578838+0000","last_change":"2024-05-11T08:06:47.578838+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.578838+0000","last_undegraded":"2024-05-11T08:06:47.578838+0000","last_fullsized":"2024-05-11T08:06:47.578838+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T08:06:47.598256+0000","last_change":"2024-05-11T08:06:47.595398+0000","last_active":"2024-05-11T08:06:47.598256+0000","last_peered":"2024-05-11T08:06:47.598256+0000","last_clean":"2024-05-11T08:06:47.598256+0000","last_became_active":"2024-05-11T08:06:47.595129+0000","last_became_peered":"2024-05-11T08:06:47.595129+0000","last_unstale":"2024-05-11T08:06:47.598256+0000","last_undegraded":"2024-05-11T08:06:47.598256+0000","last_fullsized":"2024-05-11T08:06:47.598256+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:38:12.898297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.580042+0000","last_change":"2024-05-11T08:06:47.580042+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.580042+0000","last_undegraded":"2024-05-11T08:06:47.580042+0000","last_fullsized":"2024-05-11T08:06:47.580042+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":14,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T08:06:49.613084+0000","last_change":"2024-05-11T08:06:47.601435+0000","last_active":"2024-05-11T08:06:49.613084+0000","last_peered":"2024-05-11T08:06:49.613084+0000","last_clean":"2024-05-11T08:06:49.613084+0000","last_became_active":"2024-05-11T08:06:47.601171+0000","last_became_peered":"2024-05-11T08:06:47.601171+0000","last_unstale":"2024-05-11T08:06:49.613084+0000","last_undegraded":"2024-05-11T08:06:49.613084+0000","last_fullsized":"2024-05-11T08:06:49.613084+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:39:48.026570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.585245+0000","last_change":"2024-05-11T08:06:47.585245+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.585245+0000","last_undegraded":"2024-05-11T08:06:47.585245+0000","last_fullsized":"2024-05-11T08:06:47.585245+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T08:06:47.596881+0000","last_change":"2024-05-11T08:06:47.588756+0000","last_active":"2024-05-11T08:06:47.596881+0000","last_peered":"2024-05-11T08:06:47.596881+0000","last_clean":"2024-05-11T08:06:47.596881+0000","last_became_active":"2024-05-11T08:06:47.588460+0000","last_became_peered":"2024-05-11T08:06:47.588460+0000","last_unstale":"2024-05-11T08:06:47.596881+0000","last_undegraded":"2024-05-11T08:06:47.596881+0000","last_fullsized":"2024-05-11T08:06:47.596881+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:38:12.898297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.584127+0000","last_change":"2024-05-11T08:06:47.584127+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.584127+0000","last_undegraded":"2024-05-11T08:06:47.584127+0000","last_fullsized":"2024-05-11T08:06:47.584127+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"41'1","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T08:06:49.614712+0000","last_change":"2024-05-11T08:06:47.598474+0000","last_active":"2024-05-11T08:06:49.614712+0000","last_peered":"2024-05-11T08:06:49.614712+0000","last_clean":"2024-05-11T08:06:49.614712+0000","last_became_active":"2024-05-11T08:06:47.598238+0000","last_became_peered":"2024-05-11T08:06:47.598238+0000","last_unstale":"2024-05-11T08:06:49.614712+0000","last_undegraded":"2024-05-11T08:06:49.614712+0000","last_fullsized":"2024-05-11T08:06:49.614712+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:06:17.262543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.579855+0000","last_change":"2024-05-11T08:06:47.579855+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.579855+0000","last_undegraded":"2024-05-11T08:06:47.579855+0000","last_fullsized":"2024-05-11T08:06:47.579855+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T08:06:47.604460+0000","last_change":"2024-05-11T08:06:47.599942+0000","last_active":"2024-05-11T08:06:47.604460+0000","last_peered":"2024-05-11T08:06:47.604460+0000","last_clean":"2024-05-11T08:06:47.604460+0000","last_became_active":"2024-05-11T08:06:47.599513+0000","last_became_peered":"2024-05-11T08:06:47.599513+0000","last_unstale":"2024-05-11T08:06:47.604460+0000","last_undegraded":"2024-05-11T08:06:47.604460+0000","last_fullsized":"2024-05-11T08:06:47.604460+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:56:21.008947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.584668+0000","last_change":"2024-05-11T08:06:47.584668+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.584668+0000","last_undegraded":"2024-05-11T08:06:47.584668+0000","last_fullsized":"2024-05-11T08:06:47.584668+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T08:06:47.604497+0000","last_change":"2024-05-11T08:06:47.600002+0000","last_active":"2024-05-11T08:06:47.604497+0000","last_peered":"2024-05-11T08:06:47.604497+0000","last_clean":"2024-05-11T08:06:47.604497+0000","last_became_active":"2024-05-11T08:06:47.599594+0000","last_became_peered":"2024-05-11T08:06:47.599594+0000","last_unstale":"2024-05-11T08:06:47.604497+0000","last_undegraded":"2024-05-11T08:06:47.604497+0000","last_fullsized":"2024-05-11T08:06:47.604497+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:38:12.898297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.584762+0000","last_change":"2024-05-11T08:06:47.584762+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.584762+0000","last_undegraded":"2024-05-11T08:06:47.584762+0000","last_fullsized":"2024-05-11T08:06:47.584762+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T08:06:47.602973+0000","last_change":"2024-05-11T08:06:47.600101+0000","last_active":"2024-05-11T08:06:47.602973+0000","last_peered":"2024-05-11T08:06:47.602973+0000","last_clean":"2024-05-11T08:06:47.602973+0000","last_became_active":"2024-05-11T08:06:47.599765+0000","last_became_peered":"2024-05-11T08:06:47.599765+0000","last_unstale":"2024-05-11T08:06:47.602973+0000","last_undegraded":"2024-05-11T08:06:47.602973+0000","last_fullsized":"2024-05-11T08:06:47.602973+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:27:11.749333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.585048+0000","last_change":"2024-05-11T08:06:47.585048+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.585048+0000","last_undegraded":"2024-05-11T08:06:47.585048+0000","last_fullsized":"2024-05-11T08:06:47.585048+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T08:06:47.596399+0000","last_change":"2024-05-11T08:06:47.592149+0000","last_active":"2024-05-11T08:06:47.596399+0000","last_peered":"2024-05-11T08:06:47.596399+0000","last_clean":"2024-05-11T08:06:47.596399+0000","last_became_active":"2024-05-11T08:06:47.591790+0000","last_became_peered":"2024-05-11T08:06:47.591790+0000","last_unstale":"2024-05-11T08:06:47.596399+0000","last_undegraded":"2024-05-11T08:06:47.596399+0000","last_fullsized":"2024-05-11T08:06:47.596399+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:08:22.364810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.583351+0000","last_change":"2024-05-11T08:06:47.583351+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.583351+0000","last_undegraded":"2024-05-11T08:06:47.583351+0000","last_fullsized":"2024-05-11T08:06:47.583351+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T08:06:47.591117+0000","last_change":"2024-05-11T08:06:47.589050+0000","last_active":"2024-05-11T08:06:47.591117+0000","last_peered":"2024-05-11T08:06:47.591117+0000","last_clean":"2024-05-11T08:06:47.591117+0000","last_became_active":"2024-05-11T08:06:47.588780+0000","last_became_peered":"2024-05-11T08:06:47.588780+0000","last_unstale":"2024-05-11T08:06:47.591117+0000","last_undegraded":"2024-05-11T08:06:47.591117+0000","last_fullsized":"2024-05-11T08:06:47.591117+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:58:03.492050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.580419+0000","last_change":"2024-05-11T08:06:47.580419+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.580419+0000","last_undegraded":"2024-05-11T08:06:47.580419+0000","last_fullsized":"2024-05-11T08:06:47.580419+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T08:06:47.602453+0000","last_change":"2024-05-11T08:06:47.597382+0000","last_active":"2024-05-11T08:06:47.602453+0000","last_peered":"2024-05-11T08:06:47.602453+0000","last_clean":"2024-05-11T08:06:47.602453+0000","last_became_active":"2024-05-11T08:06:47.597062+0000","last_became_peered":"2024-05-11T08:06:47.597062+0000","last_unstale":"2024-05-11T08:06:47.602453+0000","last_undegraded":"2024-05-11T08:06:47.602453+0000","last_fullsized":"2024-05-11T08:06:47.602453+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:12:27.297983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":142,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T08:06:47.587680+0000","last_change":"2024-05-11T08:05:45.772368+0000","last_active":"2024-05-11T08:06:47.587680+0000","last_peered":"2024-05-11T08:06:47.587680+0000","last_clean":"2024-05-11T08:06:47.587680+0000","last_became_active":"2024-05-11T08:05:45.771373+0000","last_became_peered":"2024-05-11T08:05:45.771373+0000","last_unstale":"2024-05-11T08:06:47.587680+0000","last_undegraded":"2024-05-11T08:06:47.587680+0000","last_fullsized":"2024-05-11T08:06:47.587680+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:44.616617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:44.616617+0000","last_clean_scrub_stamp":"2024-05-11T08:05:44.616617+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:02:08.078748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":70,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509457,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28032,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343808,"statfs":{"total":96636764160,"available":96608059392,"internally_reserved":0,"allocated":1036288,"data_stored":776838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509457,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27452,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344388,"statfs":{"total":96636764160,"available":96608653312,"internally_reserved":0,"allocated":442368,"data_stored":186470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509457,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27440,"kb_used_data":420,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":430080,"data_stored":179803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509457,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27440,"kb_used_data":420,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":430080,"data_stored":179803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509458,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27440,"kb_used_data":420,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":430080,"data_stored":179803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509458,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27440,"kb_used_data":420,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":430080,"data_stored":179803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509457,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28020,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343820,"statfs":{"total":96636764160,"available":96608071680,"internally_reserved":0,"allocated":1024000,"data_stored":770171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574866,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27480,"kb_used_data":460,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344360,"statfs":{"total":96636764160,"available":96608624640,"internally_reserved":0,"allocated":471040,"data_stored":200399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-11T08:06:52.311 INFO:tasks.ceph.ceph_manager.ceph:PG 6.3f is not active+clean 2024-05-11T08:06:52.311 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 39, 'state': 'creating+peering', 'last_fresh': '2024-05-11T08:06:47.580160+0000', 'last_change': '2024-05-11T08:06:47.580160+0000', 'last_active': '2024-05-11T08:06:47.573468+0000', 'last_peered': '2024-05-11T08:06:47.573468+0000', 'last_clean': '2024-05-11T08:06:47.573468+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T08:06:47.580160+0000', 'last_undegraded': '2024-05-11T08:06:47.580160+0000', 'last_fullsized': '2024-05-11T08:06:47.580160+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 39, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'last_clean_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-11T08:06:52.311 INFO:tasks.ceph.ceph_manager.ceph:PG 6.3e is not active+clean 2024-05-11T08:06:52.311 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 39, 'state': 'creating+peering', 'last_fresh': '2024-05-11T08:06:47.580981+0000', 'last_change': '2024-05-11T08:06:47.580981+0000', 'last_active': '2024-05-11T08:06:47.573468+0000', 'last_peered': '2024-05-11T08:06:47.573468+0000', 'last_clean': '2024-05-11T08:06:47.573468+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T08:06:47.580981+0000', 'last_undegraded': '2024-05-11T08:06:47.580981+0000', 'last_fullsized': '2024-05-11T08:06:47.580981+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 39, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'last_clean_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-11T08:06:52.311 INFO:tasks.ceph.ceph_manager.ceph:PG 6.3d is not active+clean 2024-05-11T08:06:52.311 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 39, 'state': 'creating+peering', 'last_fresh': '2024-05-11T08:06:47.587939+0000', 'last_change': '2024-05-11T08:06:47.587939+0000', 'last_active': '2024-05-11T08:06:47.573468+0000', 'last_peered': '2024-05-11T08:06:47.573468+0000', 'last_clean': '2024-05-11T08:06:47.573468+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T08:06:47.587939+0000', 'last_undegraded': '2024-05-11T08:06:47.587939+0000', 'last_fullsized': '2024-05-11T08:06:47.587939+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 39, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'last_clean_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-11T08:06:52.311 INFO:tasks.ceph.ceph_manager.ceph:PG 6.3a is not active+clean 2024-05-11T08:06:52.311 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 39, 'state': 'creating+peering', 'last_fresh': '2024-05-11T08:06:47.586507+0000', 'last_change': '2024-05-11T08:06:47.586507+0000', 'last_active': '2024-05-11T08:06:47.573468+0000', 'last_peered': '2024-05-11T08:06:47.573468+0000', 'last_clean': '2024-05-11T08:06:47.573468+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T08:06:47.586507+0000', 'last_undegraded': '2024-05-11T08:06:47.586507+0000', 'last_fullsized': '2024-05-11T08:06:47.586507+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 39, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'last_clean_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-11T08:06:52.311 INFO:tasks.ceph.ceph_manager.ceph:PG 6.39 is not active+clean 2024-05-11T08:06:52.311 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 39, 'state': 'creating+peering', 'last_fresh': '2024-05-11T08:06:47.580887+0000', 'last_change': '2024-05-11T08:06:47.580887+0000', 'last_active': '2024-05-11T08:06:47.573468+0000', 'last_peered': '2024-05-11T08:06:47.573468+0000', 'last_clean': '2024-05-11T08:06:47.573468+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T08:06:47.580887+0000', 'last_undegraded': '2024-05-11T08:06:47.580887+0000', 'last_fullsized': '2024-05-11T08:06:47.580887+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 39, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'last_clean_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-11T08:06:52.311 INFO:tasks.ceph.ceph_manager.ceph:PG 6.38 is not active+clean 2024-05-11T08:06:52.312 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 39, 'state': 'creating+peering', 'last_fresh': '2024-05-11T08:06:47.579778+0000', 'last_change': '2024-05-11T08:06:47.579778+0000', 'last_active': '2024-05-11T08:06:47.573468+0000', 'last_peered': '2024-05-11T08:06:47.573468+0000', 'last_clean': '2024-05-11T08:06:47.573468+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T08:06:47.579778+0000', 'last_undegraded': '2024-05-11T08:06:47.579778+0000', 'last_fullsized': '2024-05-11T08:06:47.579778+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 39, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'last_clean_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-11T08:06:52.312 INFO:tasks.ceph.ceph_manager.ceph:PG 6.37 is not active+clean 2024-05-11T08:06:52.312 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 39, 'state': 'creating+peering', 'last_fresh': '2024-05-11T08:06:47.585551+0000', 'last_change': '2024-05-11T08:06:47.585551+0000', 'last_active': '2024-05-11T08:06:47.573468+0000', 'last_peered': '2024-05-11T08:06:47.573468+0000', 'last_clean': '2024-05-11T08:06:47.573468+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T08:06:47.585551+0000', 'last_undegraded': '2024-05-11T08:06:47.585551+0000', 'last_fullsized': '2024-05-11T08:06:47.585551+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 39, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'last_clean_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-11T08:06:52.312 INFO:tasks.ceph.ceph_manager.ceph:PG 6.36 is not active+clean 2024-05-11T08:06:52.312 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 39, 'state': 'creating+peering', 'last_fresh': '2024-05-11T08:06:47.579247+0000', 'last_change': '2024-05-11T08:06:47.579247+0000', 'last_active': '2024-05-11T08:06:47.573468+0000', 'last_peered': '2024-05-11T08:06:47.573468+0000', 'last_clean': '2024-05-11T08:06:47.573468+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T08:06:47.579247+0000', 'last_undegraded': '2024-05-11T08:06:47.579247+0000', 'last_fullsized': '2024-05-11T08:06:47.579247+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 39, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'last_clean_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-11T08:06:52.312 INFO:tasks.ceph.ceph_manager.ceph:PG 6.35 is not active+clean 2024-05-11T08:06:52.312 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 39, 'state': 'creating+peering', 'last_fresh': '2024-05-11T08:06:47.580547+0000', 'last_change': '2024-05-11T08:06:47.580547+0000', 'last_active': '2024-05-11T08:06:47.573468+0000', 'last_peered': '2024-05-11T08:06:47.573468+0000', 'last_clean': '2024-05-11T08:06:47.573468+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T08:06:47.580547+0000', 'last_undegraded': '2024-05-11T08:06:47.580547+0000', 'last_fullsized': '2024-05-11T08:06:47.580547+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 39, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'last_clean_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-11T08:06:52.312 INFO:tasks.ceph.ceph_manager.ceph:PG 6.34 is not active+clean 2024-05-11T08:06:52.312 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 39, 'state': 'creating+peering', 'last_fresh': '2024-05-11T08:06:47.584544+0000', 'last_change': '2024-05-11T08:06:47.584544+0000', 'last_active': '2024-05-11T08:06:47.573468+0000', 'last_peered': '2024-05-11T08:06:47.573468+0000', 'last_clean': '2024-05-11T08:06:47.573468+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T08:06:47.584544+0000', 'last_undegraded': '2024-05-11T08:06:47.584544+0000', 'last_fullsized': '2024-05-11T08:06:47.584544+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 39, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'last_clean_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-11T08:06:52.313 INFO:tasks.ceph.ceph_manager.ceph:PG 6.33 is not active+clean 2024-05-11T08:06:52.313 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 39, 'state': 'creating+peering', 'last_fresh': '2024-05-11T08:06:47.580055+0000', 'last_change': '2024-05-11T08:06:47.580055+0000', 'last_active': '2024-05-11T08:06:47.573468+0000', 'last_peered': '2024-05-11T08:06:47.573468+0000', 'last_clean': '2024-05-11T08:06:47.573468+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T08:06:47.580055+0000', 'last_undegraded': '2024-05-11T08:06:47.580055+0000', 'last_fullsized': '2024-05-11T08:06:47.580055+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 39, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'last_clean_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-11T08:06:52.313 INFO:tasks.ceph.ceph_manager.ceph:PG 6.32 is not active+clean 2024-05-11T08:06:52.313 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 39, 'state': 'creating+peering', 'last_fresh': '2024-05-11T08:06:47.585515+0000', 'last_change': '2024-05-11T08:06:47.585515+0000', 'last_active': '2024-05-11T08:06:47.573468+0000', 'last_peered': '2024-05-11T08:06:47.573468+0000', 'last_clean': '2024-05-11T08:06:47.573468+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T08:06:47.585515+0000', 'last_undegraded': '2024-05-11T08:06:47.585515+0000', 'last_fullsized': '2024-05-11T08:06:47.585515+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 39, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'last_clean_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-11T08:06:52.313 INFO:tasks.ceph.ceph_manager.ceph:PG 6.31 is not active+clean 2024-05-11T08:06:52.313 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 39, 'state': 'creating+peering', 'last_fresh': '2024-05-11T08:06:47.579888+0000', 'last_change': '2024-05-11T08:06:47.579888+0000', 'last_active': '2024-05-11T08:06:47.573468+0000', 'last_peered': '2024-05-11T08:06:47.573468+0000', 'last_clean': '2024-05-11T08:06:47.573468+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T08:06:47.579888+0000', 'last_undegraded': '2024-05-11T08:06:47.579888+0000', 'last_fullsized': '2024-05-11T08:06:47.579888+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 39, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'last_clean_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-11T08:06:52.313 INFO:tasks.ceph.ceph_manager.ceph:PG 6.30 is not active+clean 2024-05-11T08:06:52.313 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 39, 'state': 'creating+peering', 'last_fresh': '2024-05-11T08:06:47.585758+0000', 'last_change': '2024-05-11T08:06:47.585758+0000', 'last_active': '2024-05-11T08:06:47.573468+0000', 'last_peered': '2024-05-11T08:06:47.573468+0000', 'last_clean': '2024-05-11T08:06:47.573468+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T08:06:47.585758+0000', 'last_undegraded': '2024-05-11T08:06:47.585758+0000', 'last_fullsized': '2024-05-11T08:06:47.585758+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 39, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'last_clean_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-11T08:06:52.313 INFO:tasks.ceph.ceph_manager.ceph:PG 6.2f is not active+clean 2024-05-11T08:06:52.313 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 39, 'state': 'creating+peering', 'last_fresh': '2024-05-11T08:06:47.587418+0000', 'last_change': '2024-05-11T08:06:47.587418+0000', 'last_active': '2024-05-11T08:06:47.573468+0000', 'last_peered': '2024-05-11T08:06:47.573468+0000', 'last_clean': '2024-05-11T08:06:47.573468+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T08:06:47.587418+0000', 'last_undegraded': '2024-05-11T08:06:47.587418+0000', 'last_fullsized': '2024-05-11T08:06:47.587418+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 39, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'last_clean_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-11T08:06:52.313 INFO:tasks.ceph.ceph_manager.ceph:PG 6.2e is not active+clean 2024-05-11T08:06:52.314 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 39, 'state': 'creating+peering', 'last_fresh': '2024-05-11T08:06:47.588413+0000', 'last_change': '2024-05-11T08:06:47.588413+0000', 'last_active': '2024-05-11T08:06:47.573468+0000', 'last_peered': '2024-05-11T08:06:47.573468+0000', 'last_clean': '2024-05-11T08:06:47.573468+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T08:06:47.588413+0000', 'last_undegraded': '2024-05-11T08:06:47.588413+0000', 'last_fullsized': '2024-05-11T08:06:47.588413+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 39, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'last_clean_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-11T08:06:52.314 INFO:tasks.ceph.ceph_manager.ceph:PG 6.2c is not active+clean 2024-05-11T08:06:52.314 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 39, 'state': 'creating+peering', 'last_fresh': '2024-05-11T08:06:47.587907+0000', 'last_change': '2024-05-11T08:06:47.587907+0000', 'last_active': '2024-05-11T08:06:47.573468+0000', 'last_peered': '2024-05-11T08:06:47.573468+0000', 'last_clean': '2024-05-11T08:06:47.573468+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T08:06:47.587907+0000', 'last_undegraded': '2024-05-11T08:06:47.587907+0000', 'last_fullsized': '2024-05-11T08:06:47.587907+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 39, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'last_clean_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-11T08:06:52.314 INFO:tasks.ceph.ceph_manager.ceph:PG 6.2b is not active+clean 2024-05-11T08:06:52.314 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 39, 'state': 'creating+peering', 'last_fresh': '2024-05-11T08:06:47.579095+0000', 'last_change': '2024-05-11T08:06:47.579095+0000', 'last_active': '2024-05-11T08:06:47.573468+0000', 'last_peered': '2024-05-11T08:06:47.573468+0000', 'last_clean': '2024-05-11T08:06:47.573468+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T08:06:47.579095+0000', 'last_undegraded': '2024-05-11T08:06:47.579095+0000', 'last_fullsized': '2024-05-11T08:06:47.579095+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 39, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'last_clean_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-11T08:06:52.314 INFO:tasks.ceph.ceph_manager.ceph:PG 6.2a is not active+clean 2024-05-11T08:06:52.314 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 39, 'state': 'creating+peering', 'last_fresh': '2024-05-11T08:06:47.583273+0000', 'last_change': '2024-05-11T08:06:47.583273+0000', 'last_active': '2024-05-11T08:06:47.573468+0000', 'last_peered': '2024-05-11T08:06:47.573468+0000', 'last_clean': '2024-05-11T08:06:47.573468+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T08:06:47.583273+0000', 'last_undegraded': '2024-05-11T08:06:47.583273+0000', 'last_fullsized': '2024-05-11T08:06:47.583273+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 39, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'last_clean_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-11T08:06:52.314 INFO:tasks.ceph.ceph_manager.ceph:PG 6.29 is not active+clean 2024-05-11T08:06:52.314 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 39, 'state': 'creating+peering', 'last_fresh': '2024-05-11T08:06:47.583780+0000', 'last_change': '2024-05-11T08:06:47.583780+0000', 'last_active': '2024-05-11T08:06:47.573468+0000', 'last_peered': '2024-05-11T08:06:47.573468+0000', 'last_clean': '2024-05-11T08:06:47.573468+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T08:06:47.583780+0000', 'last_undegraded': '2024-05-11T08:06:47.583780+0000', 'last_fullsized': '2024-05-11T08:06:47.583780+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 39, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'last_clean_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-11T08:06:52.314 INFO:tasks.ceph.ceph_manager.ceph:PG 6.27 is not active+clean 2024-05-11T08:06:52.315 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 39, 'state': 'creating+peering', 'last_fresh': '2024-05-11T08:06:47.589116+0000', 'last_change': '2024-05-11T08:06:47.589116+0000', 'last_active': '2024-05-11T08:06:47.573468+0000', 'last_peered': '2024-05-11T08:06:47.573468+0000', 'last_clean': '2024-05-11T08:06:47.573468+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T08:06:47.589116+0000', 'last_undegraded': '2024-05-11T08:06:47.589116+0000', 'last_fullsized': '2024-05-11T08:06:47.589116+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 39, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'last_clean_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-11T08:06:52.315 INFO:tasks.ceph.ceph_manager.ceph:PG 6.26 is not active+clean 2024-05-11T08:06:52.315 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 39, 'state': 'creating+peering', 'last_fresh': '2024-05-11T08:06:47.586184+0000', 'last_change': '2024-05-11T08:06:47.586184+0000', 'last_active': '2024-05-11T08:06:47.573468+0000', 'last_peered': '2024-05-11T08:06:47.573468+0000', 'last_clean': '2024-05-11T08:06:47.573468+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T08:06:47.586184+0000', 'last_undegraded': '2024-05-11T08:06:47.586184+0000', 'last_fullsized': '2024-05-11T08:06:47.586184+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 39, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'last_clean_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-11T08:06:52.315 INFO:tasks.ceph.ceph_manager.ceph:PG 6.25 is not active+clean 2024-05-11T08:06:52.315 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 39, 'state': 'creating+peering', 'last_fresh': '2024-05-11T08:06:47.587324+0000', 'last_change': '2024-05-11T08:06:47.587324+0000', 'last_active': '2024-05-11T08:06:47.573468+0000', 'last_peered': '2024-05-11T08:06:47.573468+0000', 'last_clean': '2024-05-11T08:06:47.573468+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T08:06:47.587324+0000', 'last_undegraded': '2024-05-11T08:06:47.587324+0000', 'last_fullsized': '2024-05-11T08:06:47.587324+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 39, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'last_clean_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-11T08:06:52.315 INFO:tasks.ceph.ceph_manager.ceph:PG 6.24 is not active+clean 2024-05-11T08:06:52.315 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 39, 'state': 'creating+peering', 'last_fresh': '2024-05-11T08:06:47.578757+0000', 'last_change': '2024-05-11T08:06:47.578757+0000', 'last_active': '2024-05-11T08:06:47.573468+0000', 'last_peered': '2024-05-11T08:06:47.573468+0000', 'last_clean': '2024-05-11T08:06:47.573468+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T08:06:47.578757+0000', 'last_undegraded': '2024-05-11T08:06:47.578757+0000', 'last_fullsized': '2024-05-11T08:06:47.578757+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 39, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'last_clean_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-11T08:06:52.315 INFO:tasks.ceph.ceph_manager.ceph:PG 6.23 is not active+clean 2024-05-11T08:06:52.315 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 39, 'state': 'creating+peering', 'last_fresh': '2024-05-11T08:06:47.588841+0000', 'last_change': '2024-05-11T08:06:47.588841+0000', 'last_active': '2024-05-11T08:06:47.573468+0000', 'last_peered': '2024-05-11T08:06:47.573468+0000', 'last_clean': '2024-05-11T08:06:47.573468+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T08:06:47.588841+0000', 'last_undegraded': '2024-05-11T08:06:47.588841+0000', 'last_fullsized': '2024-05-11T08:06:47.588841+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 39, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'last_clean_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-11T08:06:52.316 INFO:tasks.ceph.ceph_manager.ceph:PG 6.22 is not active+clean 2024-05-11T08:06:52.316 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 39, 'state': 'creating+peering', 'last_fresh': '2024-05-11T08:06:47.585718+0000', 'last_change': '2024-05-11T08:06:47.585718+0000', 'last_active': '2024-05-11T08:06:47.573468+0000', 'last_peered': '2024-05-11T08:06:47.573468+0000', 'last_clean': '2024-05-11T08:06:47.573468+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T08:06:47.585718+0000', 'last_undegraded': '2024-05-11T08:06:47.585718+0000', 'last_fullsized': '2024-05-11T08:06:47.585718+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 39, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'last_clean_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-11T08:06:52.316 INFO:tasks.ceph.ceph_manager.ceph:PG 6.21 is not active+clean 2024-05-11T08:06:52.316 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 39, 'state': 'creating+peering', 'last_fresh': '2024-05-11T08:06:47.583214+0000', 'last_change': '2024-05-11T08:06:47.583214+0000', 'last_active': '2024-05-11T08:06:47.573468+0000', 'last_peered': '2024-05-11T08:06:47.573468+0000', 'last_clean': '2024-05-11T08:06:47.573468+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T08:06:47.583214+0000', 'last_undegraded': '2024-05-11T08:06:47.583214+0000', 'last_fullsized': '2024-05-11T08:06:47.583214+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 39, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'last_clean_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-11T08:06:52.316 INFO:tasks.ceph.ceph_manager.ceph:PG 6.20 is not active+clean 2024-05-11T08:06:52.316 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 39, 'state': 'creating+peering', 'last_fresh': '2024-05-11T08:06:47.587217+0000', 'last_change': '2024-05-11T08:06:47.587217+0000', 'last_active': '2024-05-11T08:06:47.573468+0000', 'last_peered': '2024-05-11T08:06:47.573468+0000', 'last_clean': '2024-05-11T08:06:47.573468+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T08:06:47.587217+0000', 'last_undegraded': '2024-05-11T08:06:47.587217+0000', 'last_fullsized': '2024-05-11T08:06:47.587217+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 39, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'last_clean_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-11T08:06:52.316 INFO:tasks.ceph.ceph_manager.ceph:PG 6.1c is not active+clean 2024-05-11T08:06:52.316 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 39, 'state': 'creating+peering', 'last_fresh': '2024-05-11T08:06:47.584847+0000', 'last_change': '2024-05-11T08:06:47.584847+0000', 'last_active': '2024-05-11T08:06:47.573468+0000', 'last_peered': '2024-05-11T08:06:47.573468+0000', 'last_clean': '2024-05-11T08:06:47.573468+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T08:06:47.584847+0000', 'last_undegraded': '2024-05-11T08:06:47.584847+0000', 'last_fullsized': '2024-05-11T08:06:47.584847+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 39, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'last_clean_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-11T08:06:52.316 INFO:tasks.ceph.ceph_manager.ceph:PG 6.1d is not active+clean 2024-05-11T08:06:52.316 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 39, 'state': 'creating+peering', 'last_fresh': '2024-05-11T08:06:47.585679+0000', 'last_change': '2024-05-11T08:06:47.585679+0000', 'last_active': '2024-05-11T08:06:47.573468+0000', 'last_peered': '2024-05-11T08:06:47.573468+0000', 'last_clean': '2024-05-11T08:06:47.573468+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T08:06:47.585679+0000', 'last_undegraded': '2024-05-11T08:06:47.585679+0000', 'last_fullsized': '2024-05-11T08:06:47.585679+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 39, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'last_clean_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-11T08:06:52.316 INFO:tasks.ceph.ceph_manager.ceph:PG 6.1e is not active+clean 2024-05-11T08:06:52.317 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 39, 'state': 'creating+peering', 'last_fresh': '2024-05-11T08:06:47.584309+0000', 'last_change': '2024-05-11T08:06:47.584309+0000', 'last_active': '2024-05-11T08:06:47.573468+0000', 'last_peered': '2024-05-11T08:06:47.573468+0000', 'last_clean': '2024-05-11T08:06:47.573468+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T08:06:47.584309+0000', 'last_undegraded': '2024-05-11T08:06:47.584309+0000', 'last_fullsized': '2024-05-11T08:06:47.584309+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 39, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'last_clean_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-11T08:06:52.317 INFO:tasks.ceph.ceph_manager.ceph:PG 6.1f is not active+clean 2024-05-11T08:06:52.317 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 39, 'state': 'creating+peering', 'last_fresh': '2024-05-11T08:06:47.580834+0000', 'last_change': '2024-05-11T08:06:47.580834+0000', 'last_active': '2024-05-11T08:06:47.573468+0000', 'last_peered': '2024-05-11T08:06:47.573468+0000', 'last_clean': '2024-05-11T08:06:47.573468+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T08:06:47.580834+0000', 'last_undegraded': '2024-05-11T08:06:47.580834+0000', 'last_fullsized': '2024-05-11T08:06:47.580834+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 39, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'last_clean_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-11T08:06:52.317 INFO:tasks.ceph.ceph_manager.ceph:PG 6.18 is not active+clean 2024-05-11T08:06:52.317 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 39, 'state': 'creating+peering', 'last_fresh': '2024-05-11T08:06:47.586542+0000', 'last_change': '2024-05-11T08:06:47.586542+0000', 'last_active': '2024-05-11T08:06:47.573468+0000', 'last_peered': '2024-05-11T08:06:47.573468+0000', 'last_clean': '2024-05-11T08:06:47.573468+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T08:06:47.586542+0000', 'last_undegraded': '2024-05-11T08:06:47.586542+0000', 'last_fullsized': '2024-05-11T08:06:47.586542+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 39, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'last_clean_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-11T08:06:52.317 INFO:tasks.ceph.ceph_manager.ceph:PG 6.19 is not active+clean 2024-05-11T08:06:52.317 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 39, 'state': 'creating+peering', 'last_fresh': '2024-05-11T08:06:47.578355+0000', 'last_change': '2024-05-11T08:06:47.578355+0000', 'last_active': '2024-05-11T08:06:47.573468+0000', 'last_peered': '2024-05-11T08:06:47.573468+0000', 'last_clean': '2024-05-11T08:06:47.573468+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T08:06:47.578355+0000', 'last_undegraded': '2024-05-11T08:06:47.578355+0000', 'last_fullsized': '2024-05-11T08:06:47.578355+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 39, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'last_clean_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-11T08:06:52.317 INFO:tasks.ceph.ceph_manager.ceph:PG 6.1a is not active+clean 2024-05-11T08:06:52.317 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 39, 'state': 'creating+peering', 'last_fresh': '2024-05-11T08:06:47.583760+0000', 'last_change': '2024-05-11T08:06:47.583760+0000', 'last_active': '2024-05-11T08:06:47.573468+0000', 'last_peered': '2024-05-11T08:06:47.573468+0000', 'last_clean': '2024-05-11T08:06:47.573468+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T08:06:47.583760+0000', 'last_undegraded': '2024-05-11T08:06:47.583760+0000', 'last_fullsized': '2024-05-11T08:06:47.583760+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 39, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'last_clean_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-11T08:06:52.317 INFO:tasks.ceph.ceph_manager.ceph:PG 6.1b is not active+clean 2024-05-11T08:06:52.317 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 39, 'state': 'creating+peering', 'last_fresh': '2024-05-11T08:06:47.581488+0000', 'last_change': '2024-05-11T08:06:47.581488+0000', 'last_active': '2024-05-11T08:06:47.573468+0000', 'last_peered': '2024-05-11T08:06:47.573468+0000', 'last_clean': '2024-05-11T08:06:47.573468+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T08:06:47.581488+0000', 'last_undegraded': '2024-05-11T08:06:47.581488+0000', 'last_fullsized': '2024-05-11T08:06:47.581488+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 39, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'last_clean_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-11T08:06:52.318 INFO:tasks.ceph.ceph_manager.ceph:PG 6.15 is not active+clean 2024-05-11T08:06:52.318 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 39, 'state': 'creating+peering', 'last_fresh': '2024-05-11T08:06:47.584566+0000', 'last_change': '2024-05-11T08:06:47.584566+0000', 'last_active': '2024-05-11T08:06:47.573468+0000', 'last_peered': '2024-05-11T08:06:47.573468+0000', 'last_clean': '2024-05-11T08:06:47.573468+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T08:06:47.584566+0000', 'last_undegraded': '2024-05-11T08:06:47.584566+0000', 'last_fullsized': '2024-05-11T08:06:47.584566+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 39, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'last_clean_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-11T08:06:52.318 INFO:tasks.ceph.ceph_manager.ceph:PG 6.16 is not active+clean 2024-05-11T08:06:52.318 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 39, 'state': 'creating+peering', 'last_fresh': '2024-05-11T08:06:47.584770+0000', 'last_change': '2024-05-11T08:06:47.584770+0000', 'last_active': '2024-05-11T08:06:47.573468+0000', 'last_peered': '2024-05-11T08:06:47.573468+0000', 'last_clean': '2024-05-11T08:06:47.573468+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T08:06:47.584770+0000', 'last_undegraded': '2024-05-11T08:06:47.584770+0000', 'last_fullsized': '2024-05-11T08:06:47.584770+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 39, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'last_clean_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-11T08:06:52.318 INFO:tasks.ceph.ceph_manager.ceph:PG 6.17 is not active+clean 2024-05-11T08:06:52.318 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 39, 'state': 'creating+peering', 'last_fresh': '2024-05-11T08:06:47.584560+0000', 'last_change': '2024-05-11T08:06:47.584560+0000', 'last_active': '2024-05-11T08:06:47.573468+0000', 'last_peered': '2024-05-11T08:06:47.573468+0000', 'last_clean': '2024-05-11T08:06:47.573468+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T08:06:47.584560+0000', 'last_undegraded': '2024-05-11T08:06:47.584560+0000', 'last_fullsized': '2024-05-11T08:06:47.584560+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 39, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'last_clean_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-11T08:06:52.319 INFO:tasks.ceph.ceph_manager.ceph:PG 6.10 is not active+clean 2024-05-11T08:06:52.319 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 39, 'state': 'creating+peering', 'last_fresh': '2024-05-11T08:06:47.585804+0000', 'last_change': '2024-05-11T08:06:47.585804+0000', 'last_active': '2024-05-11T08:06:47.573468+0000', 'last_peered': '2024-05-11T08:06:47.573468+0000', 'last_clean': '2024-05-11T08:06:47.573468+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T08:06:47.585804+0000', 'last_undegraded': '2024-05-11T08:06:47.585804+0000', 'last_fullsized': '2024-05-11T08:06:47.585804+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 39, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'last_clean_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-11T08:06:52.319 INFO:tasks.ceph.ceph_manager.ceph:PG 6.11 is not active+clean 2024-05-11T08:06:52.319 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 39, 'state': 'creating+peering', 'last_fresh': '2024-05-11T08:06:47.579895+0000', 'last_change': '2024-05-11T08:06:47.579895+0000', 'last_active': '2024-05-11T08:06:47.573468+0000', 'last_peered': '2024-05-11T08:06:47.573468+0000', 'last_clean': '2024-05-11T08:06:47.573468+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T08:06:47.579895+0000', 'last_undegraded': '2024-05-11T08:06:47.579895+0000', 'last_fullsized': '2024-05-11T08:06:47.579895+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 39, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'last_clean_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-11T08:06:52.319 INFO:tasks.ceph.ceph_manager.ceph:PG 6.12 is not active+clean 2024-05-11T08:06:52.319 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 39, 'state': 'creating+peering', 'last_fresh': '2024-05-11T08:06:47.584152+0000', 'last_change': '2024-05-11T08:06:47.584152+0000', 'last_active': '2024-05-11T08:06:47.573468+0000', 'last_peered': '2024-05-11T08:06:47.573468+0000', 'last_clean': '2024-05-11T08:06:47.573468+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T08:06:47.584152+0000', 'last_undegraded': '2024-05-11T08:06:47.584152+0000', 'last_fullsized': '2024-05-11T08:06:47.584152+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 39, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'last_clean_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-11T08:06:52.319 INFO:tasks.ceph.ceph_manager.ceph:PG 6.13 is not active+clean 2024-05-11T08:06:52.319 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 39, 'state': 'creating+peering', 'last_fresh': '2024-05-11T08:06:47.580872+0000', 'last_change': '2024-05-11T08:06:47.580872+0000', 'last_active': '2024-05-11T08:06:47.573468+0000', 'last_peered': '2024-05-11T08:06:47.573468+0000', 'last_clean': '2024-05-11T08:06:47.573468+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T08:06:47.580872+0000', 'last_undegraded': '2024-05-11T08:06:47.580872+0000', 'last_fullsized': '2024-05-11T08:06:47.580872+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 39, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'last_clean_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-11T08:06:52.319 INFO:tasks.ceph.ceph_manager.ceph:PG 6.c is not active+clean 2024-05-11T08:06:52.319 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 39, 'state': 'creating+peering', 'last_fresh': '2024-05-11T08:06:47.580505+0000', 'last_change': '2024-05-11T08:06:47.580505+0000', 'last_active': '2024-05-11T08:06:47.573468+0000', 'last_peered': '2024-05-11T08:06:47.573468+0000', 'last_clean': '2024-05-11T08:06:47.573468+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T08:06:47.580505+0000', 'last_undegraded': '2024-05-11T08:06:47.580505+0000', 'last_fullsized': '2024-05-11T08:06:47.580505+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 39, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'last_clean_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-11T08:06:52.319 INFO:tasks.ceph.ceph_manager.ceph:PG 6.d is not active+clean 2024-05-11T08:06:52.319 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 39, 'state': 'creating+peering', 'last_fresh': '2024-05-11T08:06:47.580040+0000', 'last_change': '2024-05-11T08:06:47.580040+0000', 'last_active': '2024-05-11T08:06:47.573468+0000', 'last_peered': '2024-05-11T08:06:47.573468+0000', 'last_clean': '2024-05-11T08:06:47.573468+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T08:06:47.580040+0000', 'last_undegraded': '2024-05-11T08:06:47.580040+0000', 'last_fullsized': '2024-05-11T08:06:47.580040+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 39, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'last_clean_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-11T08:06:52.320 INFO:tasks.ceph.ceph_manager.ceph:PG 6.e is not active+clean 2024-05-11T08:06:52.320 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 39, 'state': 'creating+peering', 'last_fresh': '2024-05-11T08:06:47.585260+0000', 'last_change': '2024-05-11T08:06:47.585260+0000', 'last_active': '2024-05-11T08:06:47.573468+0000', 'last_peered': '2024-05-11T08:06:47.573468+0000', 'last_clean': '2024-05-11T08:06:47.573468+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T08:06:47.585260+0000', 'last_undegraded': '2024-05-11T08:06:47.585260+0000', 'last_fullsized': '2024-05-11T08:06:47.585260+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 39, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'last_clean_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-11T08:06:52.320 INFO:tasks.ceph.ceph_manager.ceph:PG 6.8 is not active+clean 2024-05-11T08:06:52.320 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 39, 'state': 'creating+peering', 'last_fresh': '2024-05-11T08:06:47.585274+0000', 'last_change': '2024-05-11T08:06:47.585274+0000', 'last_active': '2024-05-11T08:06:47.573468+0000', 'last_peered': '2024-05-11T08:06:47.573468+0000', 'last_clean': '2024-05-11T08:06:47.573468+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T08:06:47.585274+0000', 'last_undegraded': '2024-05-11T08:06:47.585274+0000', 'last_fullsized': '2024-05-11T08:06:47.585274+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 39, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'last_clean_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-11T08:06:52.320 INFO:tasks.ceph.ceph_manager.ceph:PG 6.9 is not active+clean 2024-05-11T08:06:52.320 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 39, 'state': 'creating+peering', 'last_fresh': '2024-05-11T08:06:47.585232+0000', 'last_change': '2024-05-11T08:06:47.585232+0000', 'last_active': '2024-05-11T08:06:47.573468+0000', 'last_peered': '2024-05-11T08:06:47.573468+0000', 'last_clean': '2024-05-11T08:06:47.573468+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T08:06:47.585232+0000', 'last_undegraded': '2024-05-11T08:06:47.585232+0000', 'last_fullsized': '2024-05-11T08:06:47.585232+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 39, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'last_clean_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-11T08:06:52.320 INFO:tasks.ceph.ceph_manager.ceph:PG 6.a is not active+clean 2024-05-11T08:06:52.320 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 39, 'state': 'creating+peering', 'last_fresh': '2024-05-11T08:06:47.578838+0000', 'last_change': '2024-05-11T08:06:47.578838+0000', 'last_active': '2024-05-11T08:06:47.573468+0000', 'last_peered': '2024-05-11T08:06:47.573468+0000', 'last_clean': '2024-05-11T08:06:47.573468+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T08:06:47.578838+0000', 'last_undegraded': '2024-05-11T08:06:47.578838+0000', 'last_fullsized': '2024-05-11T08:06:47.578838+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 39, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'last_clean_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-11T08:06:52.320 INFO:tasks.ceph.ceph_manager.ceph:PG 6.b is not active+clean 2024-05-11T08:06:52.320 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 39, 'state': 'creating+peering', 'last_fresh': '2024-05-11T08:06:47.580042+0000', 'last_change': '2024-05-11T08:06:47.580042+0000', 'last_active': '2024-05-11T08:06:47.573468+0000', 'last_peered': '2024-05-11T08:06:47.573468+0000', 'last_clean': '2024-05-11T08:06:47.573468+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T08:06:47.580042+0000', 'last_undegraded': '2024-05-11T08:06:47.580042+0000', 'last_fullsized': '2024-05-11T08:06:47.580042+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 39, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'last_clean_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-11T08:06:52.320 INFO:tasks.ceph.ceph_manager.ceph:PG 6.4 is not active+clean 2024-05-11T08:06:52.320 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 39, 'state': 'creating+peering', 'last_fresh': '2024-05-11T08:06:47.585245+0000', 'last_change': '2024-05-11T08:06:47.585245+0000', 'last_active': '2024-05-11T08:06:47.573468+0000', 'last_peered': '2024-05-11T08:06:47.573468+0000', 'last_clean': '2024-05-11T08:06:47.573468+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T08:06:47.585245+0000', 'last_undegraded': '2024-05-11T08:06:47.585245+0000', 'last_fullsized': '2024-05-11T08:06:47.585245+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 39, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'last_clean_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-11T08:06:52.321 INFO:tasks.ceph.ceph_manager.ceph:PG 6.5 is not active+clean 2024-05-11T08:06:52.321 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 39, 'state': 'creating+peering', 'last_fresh': '2024-05-11T08:06:47.584127+0000', 'last_change': '2024-05-11T08:06:47.584127+0000', 'last_active': '2024-05-11T08:06:47.573468+0000', 'last_peered': '2024-05-11T08:06:47.573468+0000', 'last_clean': '2024-05-11T08:06:47.573468+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T08:06:47.584127+0000', 'last_undegraded': '2024-05-11T08:06:47.584127+0000', 'last_fullsized': '2024-05-11T08:06:47.584127+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 39, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'last_clean_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-11T08:06:52.321 INFO:tasks.ceph.ceph_manager.ceph:PG 6.6 is not active+clean 2024-05-11T08:06:52.321 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 39, 'state': 'creating+peering', 'last_fresh': '2024-05-11T08:06:47.579855+0000', 'last_change': '2024-05-11T08:06:47.579855+0000', 'last_active': '2024-05-11T08:06:47.573468+0000', 'last_peered': '2024-05-11T08:06:47.573468+0000', 'last_clean': '2024-05-11T08:06:47.573468+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T08:06:47.579855+0000', 'last_undegraded': '2024-05-11T08:06:47.579855+0000', 'last_fullsized': '2024-05-11T08:06:47.579855+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 39, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'last_clean_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-11T08:06:52.321 INFO:tasks.ceph.ceph_manager.ceph:PG 6.0 is not active+clean 2024-05-11T08:06:52.321 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 39, 'state': 'creating+peering', 'last_fresh': '2024-05-11T08:06:47.584668+0000', 'last_change': '2024-05-11T08:06:47.584668+0000', 'last_active': '2024-05-11T08:06:47.573468+0000', 'last_peered': '2024-05-11T08:06:47.573468+0000', 'last_clean': '2024-05-11T08:06:47.573468+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T08:06:47.584668+0000', 'last_undegraded': '2024-05-11T08:06:47.584668+0000', 'last_fullsized': '2024-05-11T08:06:47.584668+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 39, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'last_clean_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-11T08:06:52.321 INFO:tasks.ceph.ceph_manager.ceph:PG 6.1 is not active+clean 2024-05-11T08:06:52.321 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 39, 'state': 'creating+peering', 'last_fresh': '2024-05-11T08:06:47.584762+0000', 'last_change': '2024-05-11T08:06:47.584762+0000', 'last_active': '2024-05-11T08:06:47.573468+0000', 'last_peered': '2024-05-11T08:06:47.573468+0000', 'last_clean': '2024-05-11T08:06:47.573468+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T08:06:47.584762+0000', 'last_undegraded': '2024-05-11T08:06:47.584762+0000', 'last_fullsized': '2024-05-11T08:06:47.584762+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 39, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'last_clean_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-11T08:06:52.321 INFO:tasks.ceph.ceph_manager.ceph:PG 6.2 is not active+clean 2024-05-11T08:06:52.321 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 39, 'state': 'creating+peering', 'last_fresh': '2024-05-11T08:06:47.585048+0000', 'last_change': '2024-05-11T08:06:47.585048+0000', 'last_active': '2024-05-11T08:06:47.573468+0000', 'last_peered': '2024-05-11T08:06:47.573468+0000', 'last_clean': '2024-05-11T08:06:47.573468+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T08:06:47.585048+0000', 'last_undegraded': '2024-05-11T08:06:47.585048+0000', 'last_fullsized': '2024-05-11T08:06:47.585048+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 39, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'last_clean_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-11T08:06:52.321 INFO:tasks.ceph.ceph_manager.ceph:PG 6.3 is not active+clean 2024-05-11T08:06:52.321 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 39, 'state': 'creating+peering', 'last_fresh': '2024-05-11T08:06:47.583351+0000', 'last_change': '2024-05-11T08:06:47.583351+0000', 'last_active': '2024-05-11T08:06:47.573468+0000', 'last_peered': '2024-05-11T08:06:47.573468+0000', 'last_clean': '2024-05-11T08:06:47.573468+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T08:06:47.583351+0000', 'last_undegraded': '2024-05-11T08:06:47.583351+0000', 'last_fullsized': '2024-05-11T08:06:47.583351+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 39, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'last_clean_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-11T08:06:52.321 INFO:tasks.ceph.ceph_manager.ceph:PG 6.7 is not active+clean 2024-05-11T08:06:52.322 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 39, 'state': 'creating+peering', 'last_fresh': '2024-05-11T08:06:47.580419+0000', 'last_change': '2024-05-11T08:06:47.580419+0000', 'last_active': '2024-05-11T08:06:47.573468+0000', 'last_peered': '2024-05-11T08:06:47.573468+0000', 'last_clean': '2024-05-11T08:06:47.573468+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-11T08:06:47.580419+0000', 'last_undegraded': '2024-05-11T08:06:47.580419+0000', 'last_fullsized': '2024-05-11T08:06:47.580419+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 39, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'last_clean_scrub_stamp': '2024-05-11T08:06:47.573468+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-11T08:06:52.322 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-05-11T08:06:52.448 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.448+0000 7f7d74a9c640 1 Processor -- start 2024-05-11T08:06:52.448 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.448+0000 7f7d74a9c640 1 -- start start 2024-05-11T08:06:52.450 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.449+0000 7f7d74a9c640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f7d70154410 0x7f7d7015e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:52.450 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.449+0000 7f7d74a9c640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f7d700b4820 0x7f7d700b4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:52.450 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.449+0000 7f7d74a9c640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d700cda10 con 0x7f7d7006ca50 2024-05-11T08:06:52.450 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.449+0000 7f7d74a9c640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f7d7015e6f0 con 0x7f7d700b4820 2024-05-11T08:06:52.450 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.449+0000 7f7d74a9c640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f7d700d1160 con 0x7f7d70154410 2024-05-11T08:06:52.450 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.449+0000 7f7d6dd9b640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f7d700b4820 0x7f7d700b4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:52.450 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.449+0000 7f7d6dd9b640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f7d700b4820 0x7f7d700b4cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:44080/0 (socket says 172.21.15.167:44080) 2024-05-11T08:06:52.450 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.449+0000 7f7d6dd9b640 1 -- 172.21.15.167:0/3269910774 learned_addr learned my addr 172.21.15.167:0/3269910774 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:52.451 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.449+0000 7f7d6e59c640 1 --2- 172.21.15.167:0/3269910774 >> v2:172.21.15.204:3300/0 conn(0x7f7d70154410 0x7f7d7015e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:52.451 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.449+0000 7f7d6dd9b640 1 -- 172.21.15.167:0/3269910774 >> v2:172.21.15.204:3300/0 conn(0x7f7d70154410 msgr2=0x7f7d7015e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:52.451 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.449+0000 7f7d6dd9b640 1 --2- 172.21.15.167:0/3269910774 >> v2:172.21.15.204:3300/0 conn(0x7f7d70154410 0x7f7d7015e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:52.451 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.449+0000 7f7d6dd9b640 1 -- 172.21.15.167:0/3269910774 >> v1:172.21.15.204:6789/0 conn(0x7f7d7006ca50 legacy=0x7f7d7006cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:52.451 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.449+0000 7f7d6dd9b640 1 -- 172.21.15.167:0/3269910774 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d700cdcb0 con 0x7f7d700b4820 2024-05-11T08:06:52.451 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.449+0000 7f7d6e59c640 1 --2- 172.21.15.167:0/3269910774 >> v2:172.21.15.204:3300/0 conn(0x7f7d70154410 0x7f7d7015e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T08:06:52.451 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.450+0000 7f7d6dd9b640 1 --2- 172.21.15.167:0/3269910774 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d700b4820 0x7f7d700b4cb0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f7d7015e8d0 tx=0x7f7d5802a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=439aac8ab1020b7a server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:52.451 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.450+0000 7f7d6d59a640 1 -- 172.21.15.167:0/3269910774 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d58037070 con 0x7f7d700b4820 2024-05-11T08:06:52.451 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.450+0000 7f7d6d59a640 1 -- 172.21.15.167:0/3269910774 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d58032440 con 0x7f7d700b4820 2024-05-11T08:06:52.451 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.450+0000 7f7d6d59a640 1 -- 172.21.15.167:0/3269910774 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d58032810 con 0x7f7d700b4820 2024-05-11T08:06:52.452 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.451+0000 7f7d74a9c640 1 -- 172.21.15.167:0/3269910774 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d700b4820 msgr2=0x7f7d700b4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:52.452 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.451+0000 7f7d74a9c640 1 --2- 172.21.15.167:0/3269910774 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d700b4820 0x7f7d700b4cb0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f7d7015e8d0 tx=0x7f7d5802a130 comp rx=0 tx=0).stop 2024-05-11T08:06:52.452 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.451+0000 7f7d74a9c640 1 -- 172.21.15.167:0/3269910774 shutdown_connections 2024-05-11T08:06:52.452 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.451+0000 7f7d74a9c640 1 --2- 172.21.15.167:0/3269910774 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d700b4820 0x7f7d700b4cb0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:52.452 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.451+0000 7f7d74a9c640 1 --2- 172.21.15.167:0/3269910774 >> v2:172.21.15.204:3300/0 conn(0x7f7d70154410 0x7f7d7015e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:52.452 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.451+0000 7f7d74a9c640 1 -- 172.21.15.167:0/3269910774 >> 172.21.15.167:0/3269910774 conn(0x7f7d70070290 msgr2=0x7f7d70070670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:52.452 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.451+0000 7f7d74a9c640 1 -- 172.21.15.167:0/3269910774 shutdown_connections 2024-05-11T08:06:52.453 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.451+0000 7f7d74a9c640 1 -- 172.21.15.167:0/3269910774 wait complete. 2024-05-11T08:06:52.453 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.452+0000 7f7d74a9c640 1 Processor -- start 2024-05-11T08:06:52.453 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.452+0000 7f7d74a9c640 1 -- start start 2024-05-11T08:06:52.455 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.453+0000 7f7d74a9c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d7006ca50 0x7f7d7014a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:52.455 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.453+0000 7f7d74a9c640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7d700b4820 0x7f7d701f32d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:52.455 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.453+0000 7f7d74a9c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d70154410 0x7f7d70155920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:52.455 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.453+0000 7f7d74a9c640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7d700cf950 con 0x7f7d7006ca50 2024-05-11T08:06:52.455 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.453+0000 7f7d74a9c640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7d700cf670 con 0x7f7d70154410 2024-05-11T08:06:52.455 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.453+0000 7f7d74a9c640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f7d700cfc30 con 0x7f7d700b4820 2024-05-11T08:06:52.455 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.453+0000 7f7d6e59c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d7006ca50 0x7f7d7014a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:52.455 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.453+0000 7f7d6e59c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d7006ca50 0x7f7d7014a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:44088/0 (socket says 172.21.15.167:44088) 2024-05-11T08:06:52.455 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.453+0000 7f7d6e59c640 1 -- 172.21.15.167:0/4015830111 learned_addr learned my addr 172.21.15.167:0/4015830111 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:52.455 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.453+0000 7f7d6dd9b640 1 --2- 172.21.15.167:0/4015830111 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7d700b4820 0x7f7d701f32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:52.455 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.453+0000 7f7d6e59c640 1 -- 172.21.15.167:0/4015830111 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7d700b4820 msgr2=0x7f7d701f32d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:52.455 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.453+0000 7f7d6e59c640 1 --2- 172.21.15.167:0/4015830111 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7d700b4820 0x7f7d701f32d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:52.455 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.453+0000 7f7d6e59c640 1 -- 172.21.15.167:0/4015830111 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d70154410 msgr2=0x7f7d70155920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:52.456 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.453+0000 7f7d6e59c640 1 --2- 172.21.15.167:0/4015830111 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d70154410 0x7f7d70155920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:52.456 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.453+0000 7f7d6e59c640 1 -- 172.21.15.167:0/4015830111 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d70154880 con 0x7f7d7006ca50 2024-05-11T08:06:52.456 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.453+0000 7f7d6e59c640 1 --2- 172.21.15.167:0/4015830111 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d7006ca50 0x7f7d7014a9c0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f7d6000c7f0 tx=0x7f7d6000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:52.456 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.454+0000 7f7d66ffd640 1 -- 172.21.15.167:0/4015830111 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d60015070 con 0x7f7d7006ca50 2024-05-11T08:06:52.456 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.454+0000 7f7d66ffd640 1 -- 172.21.15.167:0/4015830111 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d60007820 con 0x7f7d7006ca50 2024-05-11T08:06:52.456 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.454+0000 7f7d66ffd640 1 -- 172.21.15.167:0/4015830111 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d60007bd0 con 0x7f7d7006ca50 2024-05-11T08:06:52.456 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.454+0000 7f7d74a9c640 1 -- 172.21.15.167:0/4015830111 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d70154b60 con 0x7f7d7006ca50 2024-05-11T08:06:52.456 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.454+0000 7f7d74a9c640 1 -- 172.21.15.167:0/4015830111 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d70155120 con 0x7f7d7006ca50 2024-05-11T08:06:52.456 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.455+0000 7f7d74a9c640 1 -- 172.21.15.167:0/4015830111 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d70069590 con 0x7f7d7006ca50 2024-05-11T08:06:52.456 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.455+0000 7f7d66ffd640 1 -- 172.21.15.167:0/4015830111 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f7d60018070 con 0x7f7d7006ca50 2024-05-11T08:06:52.457 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.456+0000 7f7d66ffd640 1 --2- 172.21.15.167:0/4015830111 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f7d50076300 0x7f7d500787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:52.457 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.456+0000 7f7d66ffd640 1 -- 172.21.15.167:0/4015830111 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6101+0+0 (secure 0 0 0) 0x7f7d60097870 con 0x7f7d7006ca50 2024-05-11T08:06:52.457 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.456+0000 7f7d6dd9b640 1 --2- 172.21.15.167:0/4015830111 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f7d50076300 0x7f7d500787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:52.457 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.456+0000 7f7d6dd9b640 1 --2- 172.21.15.167:0/4015830111 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f7d50076300 0x7f7d500787c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f7d58034000 tx=0x7f7d5802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:52.461 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.461+0000 7f7d66ffd640 1 -- 172.21.15.167:0/4015830111 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f7d60061040 con 0x7f7d7006ca50 2024-05-11T08:06:52.714 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.713+0000 7f7d74a9c640 1 -- 172.21.15.167:0/4015830111 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f7d7006d0d0 con 0x7f7d7006ca50 2024-05-11T08:06:52.714 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.714+0000 7f7d66ffd640 1 -- 172.21.15.167:0/4015830111 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1644 (secure 0 0 0) 0x7f7d60060940 con 0x7f7d7006ca50 2024-05-11T08:06:52.714 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T08:06:52.714 INFO:teuthology.orchestra.run.smithi167.stdout:{"fsid":"2034778d-8ff1-4823-aa44-ded32b518241","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":81,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":41,"num_osds":8,"num_up_osds":8,"osd_up_since":1715414742,"num_in_osds":8,"osd_in_since":1715414733,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":58},{"state_name":"active+clean","count":39}],"num_pgs":97,"num_pools":3,"num_objects":5,"data_bytes":590438,"bytes_used":226041856,"bytes_avail":772868071424,"bytes_total":773094113280,"inactive_pgs_ratio":0.59793812036514282,"write_bytes_sec":191,"read_op_per_sec":0,"write_op_per_sec":0},"fsmap":{"epoch":16,"id":2,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":2,"rank":0,"name":"b","status":"up:active","gid":4750}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":4,"modified":"2024-05-11T08:06:43.927040+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-05-11T08:06:52.716 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.716+0000 7f7d74a9c640 1 -- 172.21.15.167:0/4015830111 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f7d50076300 msgr2=0x7f7d500787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:52.716 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.716+0000 7f7d74a9c640 1 --2- 172.21.15.167:0/4015830111 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f7d50076300 0x7f7d500787c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f7d58034000 tx=0x7f7d5802e000 comp rx=0 tx=0).stop 2024-05-11T08:06:52.716 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.716+0000 7f7d74a9c640 1 -- 172.21.15.167:0/4015830111 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d7006ca50 msgr2=0x7f7d7014a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:52.717 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.716+0000 7f7d74a9c640 1 --2- 172.21.15.167:0/4015830111 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d7006ca50 0x7f7d7014a9c0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f7d6000c7f0 tx=0x7f7d6000ccb0 comp rx=0 tx=0).stop 2024-05-11T08:06:52.717 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.716+0000 7f7d74a9c640 1 -- 172.21.15.167:0/4015830111 shutdown_connections 2024-05-11T08:06:52.717 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.716+0000 7f7d74a9c640 1 --2- 172.21.15.167:0/4015830111 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f7d50076300 0x7f7d500787c0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:52.717 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.716+0000 7f7d74a9c640 1 --2- 172.21.15.167:0/4015830111 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d70154410 0x7f7d70155920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:52.717 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.716+0000 7f7d74a9c640 1 --2- 172.21.15.167:0/4015830111 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7d700b4820 0x7f7d701f32d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:52.718 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.716+0000 7f7d74a9c640 1 --2- 172.21.15.167:0/4015830111 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d7006ca50 0x7f7d7014a9c0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:52.718 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.716+0000 7f7d74a9c640 1 -- 172.21.15.167:0/4015830111 >> 172.21.15.167:0/4015830111 conn(0x7f7d70070290 msgr2=0x7f7d7006d2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:52.718 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.716+0000 7f7d74a9c640 1 -- 172.21.15.167:0/4015830111 shutdown_connections 2024-05-11T08:06:52.718 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.716+0000 7f7d74a9c640 1 -- 172.21.15.167:0/4015830111 wait complete. 2024-05-11T08:06:52.727 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-05-11T08:06:52.727 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-11T08:06:52.828 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.827+0000 7f21e142a640 1 Processor -- start 2024-05-11T08:06:52.828 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.827+0000 7f21e142a640 1 -- start start 2024-05-11T08:06:52.829 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.828+0000 7f21e142a640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f21dc071800 0x7f21dc0d0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:52.829 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.828+0000 7f21e142a640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21dc0b4f50 con 0x7f21dc0b5d10 2024-05-11T08:06:52.829 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.828+0000 7f21e142a640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21dc0b5230 con 0x7f21dc148990 2024-05-11T08:06:52.829 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.828+0000 7f21e142a640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f21dc0b5510 con 0x7f21dc071800 2024-05-11T08:06:52.829 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.828+0000 7f21da7fc640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f21dc071800 0x7f21dc0d0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:52.829 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.828+0000 7f21db7fe640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f21dc0b5d10 0x7f21dc0ce870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:56004/0 (socket says 172.21.15.167:56004) 2024-05-11T08:06:52.829 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.829+0000 7f21db7fe640 1 -- 172.21.15.167:0/1593561751 learned_addr learned my addr 172.21.15.167:0/1593561751 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T08:06:52.829 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.829+0000 7f21da7fc640 1 -- 172.21.15.167:0/1593561751 >> v1:172.21.15.204:6789/0 conn(0x7f21dc148990 legacy=0x7f21dc0710f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:52.829 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.829+0000 7f21da7fc640 1 -- 172.21.15.167:0/1593561751 >> v1:172.21.15.167:6789/0 conn(0x7f21dc0b5d10 legacy=0x7f21dc0ce870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:52.829 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.829+0000 7f21da7fc640 1 -- 172.21.15.167:0/1593561751 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21dc0b57f0 con 0x7f21dc071800 2024-05-11T08:06:52.829 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.829+0000 7f21d9ffb640 1 -- 172.21.15.167:0/1593561751 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1056423738 0 0) 0x7f21dc0b4f50 con 0x7f21dc0b5d10 2024-05-11T08:06:52.830 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.830+0000 7f21da7fc640 1 --2- 172.21.15.167:0/1593561751 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f21dc071800 0x7f21dc0d0700 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f21d00097a0 tx=0x7f21d002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80d622bc51ae8e0a server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:52.831 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.830+0000 7f21d9ffb640 1 -- 172.21.15.167:0/1593561751 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21d0037070 con 0x7f21dc071800 2024-05-11T08:06:52.831 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.830+0000 7f21d9ffb640 1 -- 172.21.15.167:0/1593561751 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21d0032440 con 0x7f21dc071800 2024-05-11T08:06:52.831 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.830+0000 7f21d9ffb640 1 -- 172.21.15.167:0/1593561751 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21d0032810 con 0x7f21dc071800 2024-05-11T08:06:52.831 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.830+0000 7f21e142a640 1 -- 172.21.15.167:0/1593561751 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f21dc071800 msgr2=0x7f21dc0d0700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:52.831 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.830+0000 7f21e142a640 1 --2- 172.21.15.167:0/1593561751 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f21dc071800 0x7f21dc0d0700 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f21d00097a0 tx=0x7f21d002a230 comp rx=0 tx=0).stop 2024-05-11T08:06:52.831 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.830+0000 7f21e142a640 1 -- 172.21.15.167:0/1593561751 shutdown_connections 2024-05-11T08:06:52.831 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.830+0000 7f21e142a640 1 --2- 172.21.15.167:0/1593561751 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f21dc071800 0x7f21dc0d0700 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:52.831 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.830+0000 7f21e142a640 1 -- 172.21.15.167:0/1593561751 >> 172.21.15.167:0/1593561751 conn(0x7f21dc0d1e40 msgr2=0x7f21dc0d2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:52.831 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.830+0000 7f21e142a640 1 -- 172.21.15.167:0/1593561751 shutdown_connections 2024-05-11T08:06:52.831 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.830+0000 7f21e142a640 1 -- 172.21.15.167:0/1593561751 wait complete. 2024-05-11T08:06:52.832 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.832+0000 7f21e142a640 1 Processor -- start 2024-05-11T08:06:52.832 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.832+0000 7f21e142a640 1 -- start start 2024-05-11T08:06:52.834 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.832+0000 7f21e142a640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f21dc071800 0x7f21dc142b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:52.834 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.832+0000 7f21e142a640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f21dc0b5d10 0x7f21dc1f5420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:52.834 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.832+0000 7f21e142a640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f21dc148990 0x7f21dc1623d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:52.834 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.832+0000 7f21e142a640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f21dc148ef0 con 0x7f21dc071800 2024-05-11T08:06:52.834 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.832+0000 7f21e142a640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f21dc148d70 con 0x7f21dc0b5d10 2024-05-11T08:06:52.834 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.832+0000 7f21e142a640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f21dc0d0c40 con 0x7f21dc148990 2024-05-11T08:06:52.834 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.833+0000 7f21daffd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f21dc071800 0x7f21dc142b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:52.834 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.833+0000 7f21daffd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f21dc071800 0x7f21dc142b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:44102/0 (socket says 172.21.15.167:44102) 2024-05-11T08:06:52.834 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.833+0000 7f21daffd640 1 -- 172.21.15.167:0/4226154367 learned_addr learned my addr 172.21.15.167:0/4226154367 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:52.835 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.833+0000 7f21da7fc640 1 --2- 172.21.15.167:0/4226154367 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f21dc0b5d10 0x7f21dc1f5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:52.835 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.833+0000 7f21db7fe640 1 --2- 172.21.15.167:0/4226154367 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f21dc148990 0x7f21dc1623d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:52.835 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.833+0000 7f21daffd640 1 -- 172.21.15.167:0/4226154367 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f21dc148990 msgr2=0x7f21dc1623d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:52.835 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.833+0000 7f21daffd640 1 --2- 172.21.15.167:0/4226154367 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f21dc148990 0x7f21dc1623d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:52.835 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.833+0000 7f21daffd640 1 -- 172.21.15.167:0/4226154367 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f21dc0b5d10 msgr2=0x7f21dc1f5420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:52.835 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.833+0000 7f21daffd640 1 --2- 172.21.15.167:0/4226154367 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f21dc0b5d10 0x7f21dc1f5420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:52.835 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.833+0000 7f21daffd640 1 -- 172.21.15.167:0/4226154367 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21dc13e3b0 con 0x7f21dc071800 2024-05-11T08:06:52.835 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.833+0000 7f21daffd640 1 --2- 172.21.15.167:0/4226154367 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f21dc071800 0x7f21dc142b20 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f21cc00c7b0 tx=0x7f21cc00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:52.835 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.833+0000 7f21c37fe640 1 -- 172.21.15.167:0/4226154367 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21cc003b70 con 0x7f21dc071800 2024-05-11T08:06:52.835 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.833+0000 7f21c37fe640 1 -- 172.21.15.167:0/4226154367 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21cc003d10 con 0x7f21dc071800 2024-05-11T08:06:52.835 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.833+0000 7f21e142a640 1 -- 172.21.15.167:0/4226154367 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21dc13e690 con 0x7f21dc071800 2024-05-11T08:06:52.835 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.833+0000 7f21c37fe640 1 -- 172.21.15.167:0/4226154367 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21cc00aa70 con 0x7f21dc071800 2024-05-11T08:06:52.835 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.833+0000 7f21e142a640 1 -- 172.21.15.167:0/4226154367 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21dc13ec50 con 0x7f21dc071800 2024-05-11T08:06:52.835 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.834+0000 7f21e142a640 1 -- 172.21.15.167:0/4226154367 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21dc069bc0 con 0x7f21dc071800 2024-05-11T08:06:52.836 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.835+0000 7f21c37fe640 1 -- 172.21.15.167:0/4226154367 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f21cc00ac10 con 0x7f21dc071800 2024-05-11T08:06:52.836 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.835+0000 7f21c37fe640 1 --2- 172.21.15.167:0/4226154367 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f21c80762d0 0x7f21c8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:52.836 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.835+0000 7f21c37fe640 1 -- 172.21.15.167:0/4226154367 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6101+0+0 (secure 0 0 0) 0x7f21cc0976f0 con 0x7f21dc071800 2024-05-11T08:06:52.836 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.835+0000 7f21da7fc640 1 --2- 172.21.15.167:0/4226154367 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f21c80762d0 0x7f21c8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:52.836 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.836+0000 7f21da7fc640 1 --2- 172.21.15.167:0/4226154367 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f21c80762d0 0x7f21c8078790 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f21d0034000 tx=0x7f21d002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:52.839 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:52.840+0000 7f21c37fe640 1 -- 172.21.15.167:0/4226154367 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f21cc060ec0 con 0x7f21dc071800 2024-05-11T08:06:53.017 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:53.017+0000 7f21e142a640 1 -- 172.21.15.167:0/4226154367 --> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f21dc0b2200 con 0x7f21c80762d0 2024-05-11T08:06:53.023 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:53.023+0000 7f21c37fe640 1 -- 172.21.15.167:0/4226154367 <== mgr.4113 v2:172.21.15.167:6832/253569496 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247131 (secure 0 0 0) 0x7f21dc0b2200 con 0x7f21c80762d0 2024-05-11T08:06:53.025 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T08:06:53.026 INFO:teuthology.orchestra.run.smithi167.stderr:dumped all 2024-05-11T08:06:53.028 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:53.026+0000 7f21e142a640 1 -- 172.21.15.167:0/4226154367 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f21c80762d0 msgr2=0x7f21c8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:53.028 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:53.026+0000 7f21e142a640 1 --2- 172.21.15.167:0/4226154367 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f21c80762d0 0x7f21c8078790 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f21d0034000 tx=0x7f21d002e000 comp rx=0 tx=0).stop 2024-05-11T08:06:53.028 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:53.026+0000 7f21e142a640 1 -- 172.21.15.167:0/4226154367 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f21dc071800 msgr2=0x7f21dc142b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:53.028 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:53.026+0000 7f21e142a640 1 --2- 172.21.15.167:0/4226154367 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f21dc071800 0x7f21dc142b20 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f21cc00c7b0 tx=0x7f21cc00cc70 comp rx=0 tx=0).stop 2024-05-11T08:06:53.028 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:53.026+0000 7f21e142a640 1 -- 172.21.15.167:0/4226154367 shutdown_connections 2024-05-11T08:06:53.028 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:53.026+0000 7f21e142a640 1 --2- 172.21.15.167:0/4226154367 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f21c80762d0 0x7f21c8078790 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:53.028 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:53.026+0000 7f21e142a640 1 --2- 172.21.15.167:0/4226154367 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f21dc148990 0x7f21dc1623d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:53.028 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:53.026+0000 7f21e142a640 1 --2- 172.21.15.167:0/4226154367 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f21dc0b5d10 0x7f21dc1f5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:53.028 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:53.026+0000 7f21e142a640 1 --2- 172.21.15.167:0/4226154367 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f21dc071800 0x7f21dc142b20 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:53.028 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:53.026+0000 7f21e142a640 1 -- 172.21.15.167:0/4226154367 >> 172.21.15.167:0/4226154367 conn(0x7f21dc0d1e40 msgr2=0x7f21dc0d2220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:53.028 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:53.026+0000 7f21e142a640 1 -- 172.21.15.167:0/4226154367 shutdown_connections 2024-05-11T08:06:53.029 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:53.026+0000 7f21e142a640 1 -- 172.21.15.167:0/4226154367 wait complete. 2024-05-11T08:06:53.037 INFO:teuthology.orchestra.run.smithi167.stdout:{"pg_ready":true,"pg_map":{"version":70,"stamp":"2024-05-11T08:06:51.928061+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590438,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":104,"num_write_kb":1629,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":46,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220744,"kb_used_data":4584,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753976,"statfs":{"total":773094113280,"available":772868071424,"internally_reserved":0,"allocated":4694016,"data_stored":2653090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":70,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.351869"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.580160+0000","last_change":"2024-05-11T08:06:47.580160+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.580160+0000","last_undegraded":"2024-05-11T08:06:47.580160+0000","last_fullsized":"2024-05-11T08:06:47.580160+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.580981+0000","last_change":"2024-05-11T08:06:47.580981+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.580981+0000","last_undegraded":"2024-05-11T08:06:47.580981+0000","last_fullsized":"2024-05-11T08:06:47.580981+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.587939+0000","last_change":"2024-05-11T08:06:47.587939+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.587939+0000","last_undegraded":"2024-05-11T08:06:47.587939+0000","last_fullsized":"2024-05-11T08:06:47.587939+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T08:06:49.613242+0000","last_change":"2024-05-11T08:06:48.616894+0000","last_active":"2024-05-11T08:06:49.613242+0000","last_peered":"2024-05-11T08:06:49.613242+0000","last_clean":"2024-05-11T08:06:49.613242+0000","last_became_active":"2024-05-11T08:06:48.616597+0000","last_became_peered":"2024-05-11T08:06:48.616597+0000","last_unstale":"2024-05-11T08:06:49.613242+0000","last_undegraded":"2024-05-11T08:06:49.613242+0000","last_fullsized":"2024-05-11T08:06:49.613242+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:47:37.702924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T08:06:49.613143+0000","last_change":"2024-05-11T08:06:48.616535+0000","last_active":"2024-05-11T08:06:49.613143+0000","last_peered":"2024-05-11T08:06:49.613143+0000","last_clean":"2024-05-11T08:06:49.613143+0000","last_became_active":"2024-05-11T08:06:48.616283+0000","last_became_peered":"2024-05-11T08:06:48.616283+0000","last_unstale":"2024-05-11T08:06:49.613143+0000","last_undegraded":"2024-05-11T08:06:49.613143+0000","last_fullsized":"2024-05-11T08:06:49.613143+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:12:46.331254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.586507+0000","last_change":"2024-05-11T08:06:47.586507+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.586507+0000","last_undegraded":"2024-05-11T08:06:47.586507+0000","last_fullsized":"2024-05-11T08:06:47.586507+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.580887+0000","last_change":"2024-05-11T08:06:47.580887+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.580887+0000","last_undegraded":"2024-05-11T08:06:47.580887+0000","last_fullsized":"2024-05-11T08:06:47.580887+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.579778+0000","last_change":"2024-05-11T08:06:47.579778+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.579778+0000","last_undegraded":"2024-05-11T08:06:47.579778+0000","last_fullsized":"2024-05-11T08:06:47.579778+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.585551+0000","last_change":"2024-05-11T08:06:47.585551+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.585551+0000","last_undegraded":"2024-05-11T08:06:47.585551+0000","last_fullsized":"2024-05-11T08:06:47.585551+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.579247+0000","last_change":"2024-05-11T08:06:47.579247+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.579247+0000","last_undegraded":"2024-05-11T08:06:47.579247+0000","last_fullsized":"2024-05-11T08:06:47.579247+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.580547+0000","last_change":"2024-05-11T08:06:47.580547+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.580547+0000","last_undegraded":"2024-05-11T08:06:47.580547+0000","last_fullsized":"2024-05-11T08:06:47.580547+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.584544+0000","last_change":"2024-05-11T08:06:47.584544+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.584544+0000","last_undegraded":"2024-05-11T08:06:47.584544+0000","last_fullsized":"2024-05-11T08:06:47.584544+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.580055+0000","last_change":"2024-05-11T08:06:47.580055+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.580055+0000","last_undegraded":"2024-05-11T08:06:47.580055+0000","last_fullsized":"2024-05-11T08:06:47.580055+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.585515+0000","last_change":"2024-05-11T08:06:47.585515+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.585515+0000","last_undegraded":"2024-05-11T08:06:47.585515+0000","last_fullsized":"2024-05-11T08:06:47.585515+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.579888+0000","last_change":"2024-05-11T08:06:47.579888+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.579888+0000","last_undegraded":"2024-05-11T08:06:47.579888+0000","last_fullsized":"2024-05-11T08:06:47.579888+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.585758+0000","last_change":"2024-05-11T08:06:47.585758+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.585758+0000","last_undegraded":"2024-05-11T08:06:47.585758+0000","last_fullsized":"2024-05-11T08:06:47.585758+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.587418+0000","last_change":"2024-05-11T08:06:47.587418+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.587418+0000","last_undegraded":"2024-05-11T08:06:47.587418+0000","last_fullsized":"2024-05-11T08:06:47.587418+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.588413+0000","last_change":"2024-05-11T08:06:47.588413+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.588413+0000","last_undegraded":"2024-05-11T08:06:47.588413+0000","last_fullsized":"2024-05-11T08:06:47.588413+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T08:06:49.613351+0000","last_change":"2024-05-11T08:06:48.606813+0000","last_active":"2024-05-11T08:06:49.613351+0000","last_peered":"2024-05-11T08:06:49.613351+0000","last_clean":"2024-05-11T08:06:49.613351+0000","last_became_active":"2024-05-11T08:06:48.606548+0000","last_became_peered":"2024-05-11T08:06:48.606548+0000","last_unstale":"2024-05-11T08:06:49.613351+0000","last_undegraded":"2024-05-11T08:06:49.613351+0000","last_fullsized":"2024-05-11T08:06:49.613351+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:40:53.505484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.587907+0000","last_change":"2024-05-11T08:06:47.587907+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.587907+0000","last_undegraded":"2024-05-11T08:06:47.587907+0000","last_fullsized":"2024-05-11T08:06:47.587907+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.579095+0000","last_change":"2024-05-11T08:06:47.579095+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.579095+0000","last_undegraded":"2024-05-11T08:06:47.579095+0000","last_fullsized":"2024-05-11T08:06:47.579095+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.583273+0000","last_change":"2024-05-11T08:06:47.583273+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.583273+0000","last_undegraded":"2024-05-11T08:06:47.583273+0000","last_fullsized":"2024-05-11T08:06:47.583273+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.583780+0000","last_change":"2024-05-11T08:06:47.583780+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.583780+0000","last_undegraded":"2024-05-11T08:06:47.583780+0000","last_fullsized":"2024-05-11T08:06:47.583780+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T08:06:49.613144+0000","last_change":"2024-05-11T08:06:48.615211+0000","last_active":"2024-05-11T08:06:49.613144+0000","last_peered":"2024-05-11T08:06:49.613144+0000","last_clean":"2024-05-11T08:06:49.613144+0000","last_became_active":"2024-05-11T08:06:48.614902+0000","last_became_peered":"2024-05-11T08:06:48.614902+0000","last_unstale":"2024-05-11T08:06:49.613144+0000","last_undegraded":"2024-05-11T08:06:49.613144+0000","last_fullsized":"2024-05-11T08:06:49.613144+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:20:23.234025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.589116+0000","last_change":"2024-05-11T08:06:47.589116+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.589116+0000","last_undegraded":"2024-05-11T08:06:47.589116+0000","last_fullsized":"2024-05-11T08:06:47.589116+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.586184+0000","last_change":"2024-05-11T08:06:47.586184+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.586184+0000","last_undegraded":"2024-05-11T08:06:47.586184+0000","last_fullsized":"2024-05-11T08:06:47.586184+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.587324+0000","last_change":"2024-05-11T08:06:47.587324+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.587324+0000","last_undegraded":"2024-05-11T08:06:47.587324+0000","last_fullsized":"2024-05-11T08:06:47.587324+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.578757+0000","last_change":"2024-05-11T08:06:47.578757+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.578757+0000","last_undegraded":"2024-05-11T08:06:47.578757+0000","last_fullsized":"2024-05-11T08:06:47.578757+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.588841+0000","last_change":"2024-05-11T08:06:47.588841+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.588841+0000","last_undegraded":"2024-05-11T08:06:47.588841+0000","last_fullsized":"2024-05-11T08:06:47.588841+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.585718+0000","last_change":"2024-05-11T08:06:47.585718+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.585718+0000","last_undegraded":"2024-05-11T08:06:47.585718+0000","last_fullsized":"2024-05-11T08:06:47.585718+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.583214+0000","last_change":"2024-05-11T08:06:47.583214+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.583214+0000","last_undegraded":"2024-05-11T08:06:47.583214+0000","last_fullsized":"2024-05-11T08:06:47.583214+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.587217+0000","last_change":"2024-05-11T08:06:47.587217+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.587217+0000","last_undegraded":"2024-05-11T08:06:47.587217+0000","last_fullsized":"2024-05-11T08:06:47.587217+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.584847+0000","last_change":"2024-05-11T08:06:47.584847+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.584847+0000","last_undegraded":"2024-05-11T08:06:47.584847+0000","last_fullsized":"2024-05-11T08:06:47.584847+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T08:06:47.594702+0000","last_change":"2024-05-11T08:06:47.591074+0000","last_active":"2024-05-11T08:06:47.594702+0000","last_peered":"2024-05-11T08:06:47.594702+0000","last_clean":"2024-05-11T08:06:47.594702+0000","last_became_active":"2024-05-11T08:06:47.590747+0000","last_became_peered":"2024-05-11T08:06:47.590747+0000","last_unstale":"2024-05-11T08:06:47.594702+0000","last_undegraded":"2024-05-11T08:06:47.594702+0000","last_fullsized":"2024-05-11T08:06:47.594702+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:11:53.548277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.585679+0000","last_change":"2024-05-11T08:06:47.585679+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.585679+0000","last_undegraded":"2024-05-11T08:06:47.585679+0000","last_fullsized":"2024-05-11T08:06:47.585679+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T08:06:47.601797+0000","last_change":"2024-05-11T08:06:47.597547+0000","last_active":"2024-05-11T08:06:47.601797+0000","last_peered":"2024-05-11T08:06:47.601797+0000","last_clean":"2024-05-11T08:06:47.601797+0000","last_became_active":"2024-05-11T08:06:47.597260+0000","last_became_peered":"2024-05-11T08:06:47.597260+0000","last_unstale":"2024-05-11T08:06:47.601797+0000","last_undegraded":"2024-05-11T08:06:47.601797+0000","last_fullsized":"2024-05-11T08:06:47.601797+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:08:22.364810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.584309+0000","last_change":"2024-05-11T08:06:47.584309+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.584309+0000","last_undegraded":"2024-05-11T08:06:47.584309+0000","last_fullsized":"2024-05-11T08:06:47.584309+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T08:06:47.601604+0000","last_change":"2024-05-11T08:06:47.597745+0000","last_active":"2024-05-11T08:06:47.601604+0000","last_peered":"2024-05-11T08:06:47.601604+0000","last_clean":"2024-05-11T08:06:47.601604+0000","last_became_active":"2024-05-11T08:06:47.597472+0000","last_became_peered":"2024-05-11T08:06:47.597472+0000","last_unstale":"2024-05-11T08:06:47.601604+0000","last_undegraded":"2024-05-11T08:06:47.601604+0000","last_fullsized":"2024-05-11T08:06:47.601604+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:56:21.008947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.580834+0000","last_change":"2024-05-11T08:06:47.580834+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.580834+0000","last_undegraded":"2024-05-11T08:06:47.580834+0000","last_fullsized":"2024-05-11T08:06:47.580834+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T08:06:47.596539+0000","last_change":"2024-05-11T08:06:47.594580+0000","last_active":"2024-05-11T08:06:47.596539+0000","last_peered":"2024-05-11T08:06:47.596539+0000","last_clean":"2024-05-11T08:06:47.596539+0000","last_became_active":"2024-05-11T08:06:47.594315+0000","last_became_peered":"2024-05-11T08:06:47.594315+0000","last_unstale":"2024-05-11T08:06:47.596539+0000","last_undegraded":"2024-05-11T08:06:47.596539+0000","last_fullsized":"2024-05-11T08:06:47.596539+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:56:21.008947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.586542+0000","last_change":"2024-05-11T08:06:47.586542+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.586542+0000","last_undegraded":"2024-05-11T08:06:47.586542+0000","last_fullsized":"2024-05-11T08:06:47.586542+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T08:06:47.602415+0000","last_change":"2024-05-11T08:06:47.600311+0000","last_active":"2024-05-11T08:06:47.602415+0000","last_peered":"2024-05-11T08:06:47.602415+0000","last_clean":"2024-05-11T08:06:47.602415+0000","last_became_active":"2024-05-11T08:06:47.600072+0000","last_became_peered":"2024-05-11T08:06:47.600072+0000","last_unstale":"2024-05-11T08:06:47.602415+0000","last_undegraded":"2024-05-11T08:06:47.602415+0000","last_fullsized":"2024-05-11T08:06:47.602415+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:13:57.487739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.578355+0000","last_change":"2024-05-11T08:06:47.578355+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.578355+0000","last_undegraded":"2024-05-11T08:06:47.578355+0000","last_fullsized":"2024-05-11T08:06:47.578355+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T08:06:47.602942+0000","last_change":"2024-05-11T08:06:47.598857+0000","last_active":"2024-05-11T08:06:47.602942+0000","last_peered":"2024-05-11T08:06:47.602942+0000","last_clean":"2024-05-11T08:06:47.602942+0000","last_became_active":"2024-05-11T08:06:47.598102+0000","last_became_peered":"2024-05-11T08:06:47.598102+0000","last_unstale":"2024-05-11T08:06:47.602942+0000","last_undegraded":"2024-05-11T08:06:47.602942+0000","last_fullsized":"2024-05-11T08:06:47.602942+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:13:57.487739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.583760+0000","last_change":"2024-05-11T08:06:47.583760+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.583760+0000","last_undegraded":"2024-05-11T08:06:47.583760+0000","last_fullsized":"2024-05-11T08:06:47.583760+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T08:06:47.599146+0000","last_change":"2024-05-11T08:06:47.595791+0000","last_active":"2024-05-11T08:06:47.599146+0000","last_peered":"2024-05-11T08:06:47.599146+0000","last_clean":"2024-05-11T08:06:47.599146+0000","last_became_active":"2024-05-11T08:06:47.595551+0000","last_became_peered":"2024-05-11T08:06:47.595551+0000","last_unstale":"2024-05-11T08:06:47.599146+0000","last_undegraded":"2024-05-11T08:06:47.599146+0000","last_fullsized":"2024-05-11T08:06:47.599146+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:00:29.291380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.581488+0000","last_change":"2024-05-11T08:06:47.581488+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.581488+0000","last_undegraded":"2024-05-11T08:06:47.581488+0000","last_fullsized":"2024-05-11T08:06:47.581488+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T08:06:47.596271+0000","last_change":"2024-05-11T08:06:47.590670+0000","last_active":"2024-05-11T08:06:47.596271+0000","last_peered":"2024-05-11T08:06:47.596271+0000","last_clean":"2024-05-11T08:06:47.596271+0000","last_became_active":"2024-05-11T08:06:47.590380+0000","last_became_peered":"2024-05-11T08:06:47.590380+0000","last_unstale":"2024-05-11T08:06:47.596271+0000","last_undegraded":"2024-05-11T08:06:47.596271+0000","last_fullsized":"2024-05-11T08:06:47.596271+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:00:29.291380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T08:06:49.613537+0000","last_change":"2024-05-11T08:06:48.616855+0000","last_active":"2024-05-11T08:06:49.613537+0000","last_peered":"2024-05-11T08:06:49.613537+0000","last_clean":"2024-05-11T08:06:49.613537+0000","last_became_active":"2024-05-11T08:06:48.616287+0000","last_became_peered":"2024-05-11T08:06:48.616287+0000","last_unstale":"2024-05-11T08:06:49.613537+0000","last_undegraded":"2024-05-11T08:06:49.613537+0000","last_fullsized":"2024-05-11T08:06:49.613537+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:31:41.113021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T08:06:47.593461+0000","last_change":"2024-05-11T08:06:47.590630+0000","last_active":"2024-05-11T08:06:47.593461+0000","last_peered":"2024-05-11T08:06:47.593461+0000","last_clean":"2024-05-11T08:06:47.593461+0000","last_became_active":"2024-05-11T08:06:47.590361+0000","last_became_peered":"2024-05-11T08:06:47.590361+0000","last_unstale":"2024-05-11T08:06:47.593461+0000","last_undegraded":"2024-05-11T08:06:47.593461+0000","last_fullsized":"2024-05-11T08:06:47.593461+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:20:42.258886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.584566+0000","last_change":"2024-05-11T08:06:47.584566+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.584566+0000","last_undegraded":"2024-05-11T08:06:47.584566+0000","last_fullsized":"2024-05-11T08:06:47.584566+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T08:06:47.595167+0000","last_change":"2024-05-11T08:06:47.587418+0000","last_active":"2024-05-11T08:06:47.595167+0000","last_peered":"2024-05-11T08:06:47.595167+0000","last_clean":"2024-05-11T08:06:47.595167+0000","last_became_active":"2024-05-11T08:06:47.586715+0000","last_became_peered":"2024-05-11T08:06:47.586715+0000","last_unstale":"2024-05-11T08:06:47.595167+0000","last_undegraded":"2024-05-11T08:06:47.595167+0000","last_fullsized":"2024-05-11T08:06:47.595167+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:56:21.008947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.584770+0000","last_change":"2024-05-11T08:06:47.584770+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.584770+0000","last_undegraded":"2024-05-11T08:06:47.584770+0000","last_fullsized":"2024-05-11T08:06:47.584770+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T08:06:47.597011+0000","last_change":"2024-05-11T08:06:47.594319+0000","last_active":"2024-05-11T08:06:47.597011+0000","last_peered":"2024-05-11T08:06:47.597011+0000","last_clean":"2024-05-11T08:06:47.597011+0000","last_became_active":"2024-05-11T08:06:47.593840+0000","last_became_peered":"2024-05-11T08:06:47.593840+0000","last_unstale":"2024-05-11T08:06:47.597011+0000","last_undegraded":"2024-05-11T08:06:47.597011+0000","last_fullsized":"2024-05-11T08:06:47.597011+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:12:27.297983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.584560+0000","last_change":"2024-05-11T08:06:47.584560+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.584560+0000","last_undegraded":"2024-05-11T08:06:47.584560+0000","last_fullsized":"2024-05-11T08:06:47.584560+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T08:06:47.596789+0000","last_change":"2024-05-11T08:06:47.592968+0000","last_active":"2024-05-11T08:06:47.596789+0000","last_peered":"2024-05-11T08:06:47.596789+0000","last_clean":"2024-05-11T08:06:47.596789+0000","last_became_active":"2024-05-11T08:06:47.592672+0000","last_became_peered":"2024-05-11T08:06:47.592672+0000","last_unstale":"2024-05-11T08:06:47.596789+0000","last_undegraded":"2024-05-11T08:06:47.596789+0000","last_fullsized":"2024-05-11T08:06:47.596789+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:46:05.539312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.585804+0000","last_change":"2024-05-11T08:06:47.585804+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.585804+0000","last_undegraded":"2024-05-11T08:06:47.585804+0000","last_fullsized":"2024-05-11T08:06:47.585804+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T08:06:47.604642+0000","last_change":"2024-05-11T08:06:47.600533+0000","last_active":"2024-05-11T08:06:47.604642+0000","last_peered":"2024-05-11T08:06:47.604642+0000","last_clean":"2024-05-11T08:06:47.604642+0000","last_became_active":"2024-05-11T08:06:47.600226+0000","last_became_peered":"2024-05-11T08:06:47.600226+0000","last_unstale":"2024-05-11T08:06:47.604642+0000","last_undegraded":"2024-05-11T08:06:47.604642+0000","last_fullsized":"2024-05-11T08:06:47.604642+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:49:09.098705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.579895+0000","last_change":"2024-05-11T08:06:47.579895+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.579895+0000","last_undegraded":"2024-05-11T08:06:47.579895+0000","last_fullsized":"2024-05-11T08:06:47.579895+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T08:06:47.601569+0000","last_change":"2024-05-11T08:06:47.597276+0000","last_active":"2024-05-11T08:06:47.601569+0000","last_peered":"2024-05-11T08:06:47.601569+0000","last_clean":"2024-05-11T08:06:47.601569+0000","last_became_active":"2024-05-11T08:06:47.596986+0000","last_became_peered":"2024-05-11T08:06:47.596986+0000","last_unstale":"2024-05-11T08:06:47.601569+0000","last_undegraded":"2024-05-11T08:06:47.601569+0000","last_fullsized":"2024-05-11T08:06:47.601569+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:08:22.364810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.584152+0000","last_change":"2024-05-11T08:06:47.584152+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.584152+0000","last_undegraded":"2024-05-11T08:06:47.584152+0000","last_fullsized":"2024-05-11T08:06:47.584152+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T08:06:47.594681+0000","last_change":"2024-05-11T08:06:47.591052+0000","last_active":"2024-05-11T08:06:47.594681+0000","last_peered":"2024-05-11T08:06:47.594681+0000","last_clean":"2024-05-11T08:06:47.594681+0000","last_became_active":"2024-05-11T08:06:47.590726+0000","last_became_peered":"2024-05-11T08:06:47.590726+0000","last_unstale":"2024-05-11T08:06:47.594681+0000","last_undegraded":"2024-05-11T08:06:47.594681+0000","last_fullsized":"2024-05-11T08:06:47.594681+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:30:44.079909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.580872+0000","last_change":"2024-05-11T08:06:47.580872+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.580872+0000","last_undegraded":"2024-05-11T08:06:47.580872+0000","last_fullsized":"2024-05-11T08:06:47.580872+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T08:06:47.598143+0000","last_change":"2024-05-11T08:06:47.597077+0000","last_active":"2024-05-11T08:06:47.598143+0000","last_peered":"2024-05-11T08:06:47.598143+0000","last_clean":"2024-05-11T08:06:47.598143+0000","last_became_active":"2024-05-11T08:06:47.596753+0000","last_became_peered":"2024-05-11T08:06:47.596753+0000","last_unstale":"2024-05-11T08:06:47.598143+0000","last_undegraded":"2024-05-11T08:06:47.598143+0000","last_fullsized":"2024-05-11T08:06:47.598143+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:19:14.446185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.580505+0000","last_change":"2024-05-11T08:06:47.580505+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.580505+0000","last_undegraded":"2024-05-11T08:06:47.580505+0000","last_fullsized":"2024-05-11T08:06:47.580505+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T08:06:47.596912+0000","last_change":"2024-05-11T08:06:47.592140+0000","last_active":"2024-05-11T08:06:47.596912+0000","last_peered":"2024-05-11T08:06:47.596912+0000","last_clean":"2024-05-11T08:06:47.596912+0000","last_became_active":"2024-05-11T08:06:47.591867+0000","last_became_peered":"2024-05-11T08:06:47.591867+0000","last_unstale":"2024-05-11T08:06:47.596912+0000","last_undegraded":"2024-05-11T08:06:47.596912+0000","last_fullsized":"2024-05-11T08:06:47.596912+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:19:14.446185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.580040+0000","last_change":"2024-05-11T08:06:47.580040+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.580040+0000","last_undegraded":"2024-05-11T08:06:47.580040+0000","last_fullsized":"2024-05-11T08:06:47.580040+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T08:06:47.597734+0000","last_change":"2024-05-11T08:06:47.595937+0000","last_active":"2024-05-11T08:06:47.597734+0000","last_peered":"2024-05-11T08:06:47.597734+0000","last_clean":"2024-05-11T08:06:47.597734+0000","last_became_active":"2024-05-11T08:06:47.595642+0000","last_became_peered":"2024-05-11T08:06:47.595642+0000","last_unstale":"2024-05-11T08:06:47.597734+0000","last_undegraded":"2024-05-11T08:06:47.597734+0000","last_fullsized":"2024-05-11T08:06:47.597734+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:38:12.898297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.585260+0000","last_change":"2024-05-11T08:06:47.585260+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.585260+0000","last_undegraded":"2024-05-11T08:06:47.585260+0000","last_fullsized":"2024-05-11T08:06:47.585260+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"40'1","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T08:06:49.613353+0000","last_change":"2024-05-11T08:06:47.599057+0000","last_active":"2024-05-11T08:06:49.613353+0000","last_peered":"2024-05-11T08:06:49.613353+0000","last_clean":"2024-05-11T08:06:49.613353+0000","last_became_active":"2024-05-11T08:06:47.598647+0000","last_became_peered":"2024-05-11T08:06:47.598647+0000","last_unstale":"2024-05-11T08:06:49.613353+0000","last_undegraded":"2024-05-11T08:06:49.613353+0000","last_fullsized":"2024-05-11T08:06:49.613353+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:40:41.542604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T08:06:49.613419+0000","last_change":"2024-05-11T08:06:48.606563+0000","last_active":"2024-05-11T08:06:49.613419+0000","last_peered":"2024-05-11T08:06:49.613419+0000","last_clean":"2024-05-11T08:06:49.613419+0000","last_became_active":"2024-05-11T08:06:48.606287+0000","last_became_peered":"2024-05-11T08:06:48.606287+0000","last_unstale":"2024-05-11T08:06:49.613419+0000","last_undegraded":"2024-05-11T08:06:49.613419+0000","last_fullsized":"2024-05-11T08:06:49.613419+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:23:43.027012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T08:06:47.596598+0000","last_change":"2024-05-11T08:06:47.592090+0000","last_active":"2024-05-11T08:06:47.596598+0000","last_peered":"2024-05-11T08:06:47.596598+0000","last_clean":"2024-05-11T08:06:47.596598+0000","last_became_active":"2024-05-11T08:06:47.591813+0000","last_became_peered":"2024-05-11T08:06:47.591813+0000","last_unstale":"2024-05-11T08:06:47.596598+0000","last_undegraded":"2024-05-11T08:06:47.596598+0000","last_fullsized":"2024-05-11T08:06:47.596598+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:27:11.749333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.585274+0000","last_change":"2024-05-11T08:06:47.585274+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.585274+0000","last_undegraded":"2024-05-11T08:06:47.585274+0000","last_fullsized":"2024-05-11T08:06:47.585274+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"40'2","reported_seq":16,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T08:06:49.613224+0000","last_change":"2024-05-11T08:06:47.600614+0000","last_active":"2024-05-11T08:06:49.613224+0000","last_peered":"2024-05-11T08:06:49.613224+0000","last_clean":"2024-05-11T08:06:49.613224+0000","last_became_active":"2024-05-11T08:06:47.600104+0000","last_became_peered":"2024-05-11T08:06:47.600104+0000","last_unstale":"2024-05-11T08:06:49.613224+0000","last_undegraded":"2024-05-11T08:06:49.613224+0000","last_fullsized":"2024-05-11T08:06:49.613224+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:25:19.853657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.585232+0000","last_change":"2024-05-11T08:06:47.585232+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.585232+0000","last_undegraded":"2024-05-11T08:06:47.585232+0000","last_fullsized":"2024-05-11T08:06:47.585232+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":14,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T08:06:49.613139+0000","last_change":"2024-05-11T08:06:47.597958+0000","last_active":"2024-05-11T08:06:49.613139+0000","last_peered":"2024-05-11T08:06:49.613139+0000","last_clean":"2024-05-11T08:06:49.613139+0000","last_became_active":"2024-05-11T08:06:47.597638+0000","last_became_peered":"2024-05-11T08:06:47.597638+0000","last_unstale":"2024-05-11T08:06:49.613139+0000","last_undegraded":"2024-05-11T08:06:49.613139+0000","last_fullsized":"2024-05-11T08:06:49.613139+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:02:41.089263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.578838+0000","last_change":"2024-05-11T08:06:47.578838+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.578838+0000","last_undegraded":"2024-05-11T08:06:47.578838+0000","last_fullsized":"2024-05-11T08:06:47.578838+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T08:06:47.598256+0000","last_change":"2024-05-11T08:06:47.595398+0000","last_active":"2024-05-11T08:06:47.598256+0000","last_peered":"2024-05-11T08:06:47.598256+0000","last_clean":"2024-05-11T08:06:47.598256+0000","last_became_active":"2024-05-11T08:06:47.595129+0000","last_became_peered":"2024-05-11T08:06:47.595129+0000","last_unstale":"2024-05-11T08:06:47.598256+0000","last_undegraded":"2024-05-11T08:06:47.598256+0000","last_fullsized":"2024-05-11T08:06:47.598256+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:38:12.898297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.580042+0000","last_change":"2024-05-11T08:06:47.580042+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.580042+0000","last_undegraded":"2024-05-11T08:06:47.580042+0000","last_fullsized":"2024-05-11T08:06:47.580042+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":14,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T08:06:49.613084+0000","last_change":"2024-05-11T08:06:47.601435+0000","last_active":"2024-05-11T08:06:49.613084+0000","last_peered":"2024-05-11T08:06:49.613084+0000","last_clean":"2024-05-11T08:06:49.613084+0000","last_became_active":"2024-05-11T08:06:47.601171+0000","last_became_peered":"2024-05-11T08:06:47.601171+0000","last_unstale":"2024-05-11T08:06:49.613084+0000","last_undegraded":"2024-05-11T08:06:49.613084+0000","last_fullsized":"2024-05-11T08:06:49.613084+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:39:48.026570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.585245+0000","last_change":"2024-05-11T08:06:47.585245+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.585245+0000","last_undegraded":"2024-05-11T08:06:47.585245+0000","last_fullsized":"2024-05-11T08:06:47.585245+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T08:06:47.596881+0000","last_change":"2024-05-11T08:06:47.588756+0000","last_active":"2024-05-11T08:06:47.596881+0000","last_peered":"2024-05-11T08:06:47.596881+0000","last_clean":"2024-05-11T08:06:47.596881+0000","last_became_active":"2024-05-11T08:06:47.588460+0000","last_became_peered":"2024-05-11T08:06:47.588460+0000","last_unstale":"2024-05-11T08:06:47.596881+0000","last_undegraded":"2024-05-11T08:06:47.596881+0000","last_fullsized":"2024-05-11T08:06:47.596881+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:38:12.898297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.584127+0000","last_change":"2024-05-11T08:06:47.584127+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.584127+0000","last_undegraded":"2024-05-11T08:06:47.584127+0000","last_fullsized":"2024-05-11T08:06:47.584127+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"41'1","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T08:06:49.614712+0000","last_change":"2024-05-11T08:06:47.598474+0000","last_active":"2024-05-11T08:06:49.614712+0000","last_peered":"2024-05-11T08:06:49.614712+0000","last_clean":"2024-05-11T08:06:49.614712+0000","last_became_active":"2024-05-11T08:06:47.598238+0000","last_became_peered":"2024-05-11T08:06:47.598238+0000","last_unstale":"2024-05-11T08:06:49.614712+0000","last_undegraded":"2024-05-11T08:06:49.614712+0000","last_fullsized":"2024-05-11T08:06:49.614712+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:06:17.262543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.579855+0000","last_change":"2024-05-11T08:06:47.579855+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.579855+0000","last_undegraded":"2024-05-11T08:06:47.579855+0000","last_fullsized":"2024-05-11T08:06:47.579855+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T08:06:47.604460+0000","last_change":"2024-05-11T08:06:47.599942+0000","last_active":"2024-05-11T08:06:47.604460+0000","last_peered":"2024-05-11T08:06:47.604460+0000","last_clean":"2024-05-11T08:06:47.604460+0000","last_became_active":"2024-05-11T08:06:47.599513+0000","last_became_peered":"2024-05-11T08:06:47.599513+0000","last_unstale":"2024-05-11T08:06:47.604460+0000","last_undegraded":"2024-05-11T08:06:47.604460+0000","last_fullsized":"2024-05-11T08:06:47.604460+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:56:21.008947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.584668+0000","last_change":"2024-05-11T08:06:47.584668+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.584668+0000","last_undegraded":"2024-05-11T08:06:47.584668+0000","last_fullsized":"2024-05-11T08:06:47.584668+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T08:06:47.604497+0000","last_change":"2024-05-11T08:06:47.600002+0000","last_active":"2024-05-11T08:06:47.604497+0000","last_peered":"2024-05-11T08:06:47.604497+0000","last_clean":"2024-05-11T08:06:47.604497+0000","last_became_active":"2024-05-11T08:06:47.599594+0000","last_became_peered":"2024-05-11T08:06:47.599594+0000","last_unstale":"2024-05-11T08:06:47.604497+0000","last_undegraded":"2024-05-11T08:06:47.604497+0000","last_fullsized":"2024-05-11T08:06:47.604497+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:38:12.898297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.584762+0000","last_change":"2024-05-11T08:06:47.584762+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.584762+0000","last_undegraded":"2024-05-11T08:06:47.584762+0000","last_fullsized":"2024-05-11T08:06:47.584762+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T08:06:47.602973+0000","last_change":"2024-05-11T08:06:47.600101+0000","last_active":"2024-05-11T08:06:47.602973+0000","last_peered":"2024-05-11T08:06:47.602973+0000","last_clean":"2024-05-11T08:06:47.602973+0000","last_became_active":"2024-05-11T08:06:47.599765+0000","last_became_peered":"2024-05-11T08:06:47.599765+0000","last_unstale":"2024-05-11T08:06:47.602973+0000","last_undegraded":"2024-05-11T08:06:47.602973+0000","last_fullsized":"2024-05-11T08:06:47.602973+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:27:11.749333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.585048+0000","last_change":"2024-05-11T08:06:47.585048+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.585048+0000","last_undegraded":"2024-05-11T08:06:47.585048+0000","last_fullsized":"2024-05-11T08:06:47.585048+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T08:06:47.596399+0000","last_change":"2024-05-11T08:06:47.592149+0000","last_active":"2024-05-11T08:06:47.596399+0000","last_peered":"2024-05-11T08:06:47.596399+0000","last_clean":"2024-05-11T08:06:47.596399+0000","last_became_active":"2024-05-11T08:06:47.591790+0000","last_became_peered":"2024-05-11T08:06:47.591790+0000","last_unstale":"2024-05-11T08:06:47.596399+0000","last_undegraded":"2024-05-11T08:06:47.596399+0000","last_fullsized":"2024-05-11T08:06:47.596399+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:08:22.364810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.583351+0000","last_change":"2024-05-11T08:06:47.583351+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.583351+0000","last_undegraded":"2024-05-11T08:06:47.583351+0000","last_fullsized":"2024-05-11T08:06:47.583351+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T08:06:47.591117+0000","last_change":"2024-05-11T08:06:47.589050+0000","last_active":"2024-05-11T08:06:47.591117+0000","last_peered":"2024-05-11T08:06:47.591117+0000","last_clean":"2024-05-11T08:06:47.591117+0000","last_became_active":"2024-05-11T08:06:47.588780+0000","last_became_peered":"2024-05-11T08:06:47.588780+0000","last_unstale":"2024-05-11T08:06:47.591117+0000","last_undegraded":"2024-05-11T08:06:47.591117+0000","last_fullsized":"2024-05-11T08:06:47.591117+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:58:03.492050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-05-11T08:06:47.580419+0000","last_change":"2024-05-11T08:06:47.580419+0000","last_active":"2024-05-11T08:06:47.573468+0000","last_peered":"2024-05-11T08:06:47.573468+0000","last_clean":"2024-05-11T08:06:47.573468+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-11T08:06:47.580419+0000","last_undegraded":"2024-05-11T08:06:47.580419+0000","last_fullsized":"2024-05-11T08:06:47.580419+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T08:06:47.602453+0000","last_change":"2024-05-11T08:06:47.597382+0000","last_active":"2024-05-11T08:06:47.602453+0000","last_peered":"2024-05-11T08:06:47.602453+0000","last_clean":"2024-05-11T08:06:47.602453+0000","last_became_active":"2024-05-11T08:06:47.597062+0000","last_became_peered":"2024-05-11T08:06:47.597062+0000","last_unstale":"2024-05-11T08:06:47.602453+0000","last_undegraded":"2024-05-11T08:06:47.602453+0000","last_fullsized":"2024-05-11T08:06:47.602453+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:12:27.297983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":142,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-11T08:06:47.587680+0000","last_change":"2024-05-11T08:05:45.772368+0000","last_active":"2024-05-11T08:06:47.587680+0000","last_peered":"2024-05-11T08:06:47.587680+0000","last_clean":"2024-05-11T08:06:47.587680+0000","last_became_active":"2024-05-11T08:05:45.771373+0000","last_became_peered":"2024-05-11T08:05:45.771373+0000","last_unstale":"2024-05-11T08:06:47.587680+0000","last_undegraded":"2024-05-11T08:06:47.587680+0000","last_fullsized":"2024-05-11T08:06:47.587680+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:44.616617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:44.616617+0000","last_clean_scrub_stamp":"2024-05-11T08:05:44.616617+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:02:08.078748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":70,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509457,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28032,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343808,"statfs":{"total":96636764160,"available":96608059392,"internally_reserved":0,"allocated":1036288,"data_stored":776838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509457,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27452,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344388,"statfs":{"total":96636764160,"available":96608653312,"internally_reserved":0,"allocated":442368,"data_stored":186470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509457,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27440,"kb_used_data":420,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":430080,"data_stored":179803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509457,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27440,"kb_used_data":420,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":430080,"data_stored":179803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509458,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27440,"kb_used_data":420,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":430080,"data_stored":179803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509458,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27440,"kb_used_data":420,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":430080,"data_stored":179803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509457,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28020,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343820,"statfs":{"total":96636764160,"available":96608071680,"internally_reserved":0,"allocated":1024000,"data_stored":770171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574866,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27480,"kb_used_data":460,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344360,"statfs":{"total":96636764160,"available":96608624640,"internally_reserved":0,"allocated":471040,"data_stored":200399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-11T08:06:53.565 INFO:tasks.ceph.mds.b.smithi204.stderr:2024-05-11T08:06:53.566+0000 7f39458ab640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-11T08:06:55.960 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-11T08:06:55.963 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-11T08:06:56.041 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-11T08:06:56.154 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.154+0000 7fcbb67c1640 1 Processor -- start 2024-05-11T08:06:56.154 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.154+0000 7fcbb67c1640 1 -- start start 2024-05-11T08:06:56.155 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.155+0000 7fcbb67c1640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fcbb01598d0 0x7fcbb00d2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:56.156 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.155+0000 7fcbb67c1640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcbb00cb6c0 0x7fcbb00d0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:56.156 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.155+0000 7fcbb67c1640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcbb00d1190 con 0x7fcbb00d0640 2024-05-11T08:06:56.156 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.155+0000 7fcbb67c1640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fcbb006d500 con 0x7fcbb01598d0 2024-05-11T08:06:56.156 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.155+0000 7fcbb67c1640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fcbb006d7e0 con 0x7fcbb00cb6c0 2024-05-11T08:06:56.156 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.155+0000 7fcbaffff640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fcbb01598d0 0x7fcbb00d2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:56.156 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.155+0000 7fcbb4d5e640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcbb00cb6c0 0x7fcbb00d0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:56.156 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.155+0000 7fcbaf7fe640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7fcbb00d0640 0x7fcbb00cafb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.167:37740/0 (socket says 172.21.15.167:37740) 2024-05-11T08:06:56.156 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.155+0000 7fcbaffff640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fcbb01598d0 0x7fcbb00d2cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.167:40830/0 (socket says 172.21.15.167:40830) 2024-05-11T08:06:56.156 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.155+0000 7fcbaf7fe640 1 -- 172.21.15.167:0/1003899358 learned_addr learned my addr 172.21.15.167:0/1003899358 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T08:06:56.156 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.156+0000 7fcbb4d5e640 1 -- 172.21.15.167:0/1003899358 >> v2:172.21.15.204:3300/0 conn(0x7fcbb01598d0 msgr2=0x7fcbb00d2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:56.156 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.156+0000 7fcbb4d5e640 1 --2- 172.21.15.167:0/1003899358 >> v2:172.21.15.204:3300/0 conn(0x7fcbb01598d0 0x7fcbb00d2cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:56.156 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.156+0000 7fcbb4d5e640 1 -- 172.21.15.167:0/1003899358 >> v1:172.21.15.204:6789/0 conn(0x7fcbb00d0640 legacy=0x7fcbb00cafb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:56.156 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.156+0000 7fcbb4d5e640 1 -- 172.21.15.167:0/1003899358 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbb006dac0 con 0x7fcbb00cb6c0 2024-05-11T08:06:56.156 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.156+0000 7fcbaffff640 1 --2- 172.21.15.167:0/1003899358 >> v2:172.21.15.204:3300/0 conn(0x7fcbb01598d0 0x7fcbb00d2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T08:06:56.158 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.156+0000 7fcbb4d5e640 1 --2- 172.21.15.167:0/1003899358 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcbb00cb6c0 0x7fcbb00d0ad0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fcba8009d90 tx=0x7fcba802a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=55bd4ec32807ebe9 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:56.158 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.157+0000 7fcbaeffd640 1 -- 172.21.15.167:0/1003899358 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcba8007740 con 0x7fcbb00cb6c0 2024-05-11T08:06:56.158 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.157+0000 7fcbaeffd640 1 -- 172.21.15.167:0/1003899358 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcba80078e0 con 0x7fcbb00cb6c0 2024-05-11T08:06:56.158 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.157+0000 7fcbaeffd640 1 -- 172.21.15.167:0/1003899358 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcba8007cb0 con 0x7fcbb00cb6c0 2024-05-11T08:06:56.158 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.157+0000 7fcbb67c1640 1 -- 172.21.15.167:0/1003899358 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcbb00cb6c0 msgr2=0x7fcbb00d0ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:56.158 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.157+0000 7fcbb67c1640 1 --2- 172.21.15.167:0/1003899358 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcbb00cb6c0 0x7fcbb00d0ad0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fcba8009d90 tx=0x7fcba802a660 comp rx=0 tx=0).stop 2024-05-11T08:06:56.158 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.157+0000 7fcbb67c1640 1 -- 172.21.15.167:0/1003899358 shutdown_connections 2024-05-11T08:06:56.158 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.157+0000 7fcbb67c1640 1 --2- 172.21.15.167:0/1003899358 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcbb00cb6c0 0x7fcbb00d0ad0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:56.158 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.157+0000 7fcbb67c1640 1 --2- 172.21.15.167:0/1003899358 >> v2:172.21.15.204:3300/0 conn(0x7fcbb01598d0 0x7fcbb00d2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:56.158 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.157+0000 7fcbb67c1640 1 -- 172.21.15.167:0/1003899358 >> 172.21.15.167:0/1003899358 conn(0x7fcbb00d1ba0 msgr2=0x7fcbb00d1fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:56.158 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.157+0000 7fcbb67c1640 1 -- 172.21.15.167:0/1003899358 shutdown_connections 2024-05-11T08:06:56.158 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.157+0000 7fcbb67c1640 1 -- 172.21.15.167:0/1003899358 wait complete. 2024-05-11T08:06:56.159 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.159+0000 7fcbb67c1640 1 Processor -- start 2024-05-11T08:06:56.159 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.159+0000 7fcbb67c1640 1 -- start start 2024-05-11T08:06:56.161 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.160+0000 7fcbb67c1640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcbb00cb6c0 0x7fcbb0148da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:56.161 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.160+0000 7fcbb67c1640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcbb00d0640 0x7fcbb0154f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:56.161 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.160+0000 7fcbb67c1640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcbb01598d0 0x7fcbb0153720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:56.161 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.160+0000 7fcbb67c1640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fcbb006d1b0 con 0x7fcbb00d0640 2024-05-11T08:06:56.161 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.160+0000 7fcbb67c1640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fcbb0159cb0 con 0x7fcbb00cb6c0 2024-05-11T08:06:56.161 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.160+0000 7fcbb67c1640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fcbb006dfe0 con 0x7fcbb01598d0 2024-05-11T08:06:56.161 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.160+0000 7fcbaf7fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcbb00d0640 0x7fcbb0154f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:56.161 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.160+0000 7fcbaffff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcbb00cb6c0 0x7fcbb0148da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:56.161 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.160+0000 7fcbaf7fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcbb00d0640 0x7fcbb0154f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:42966/0 (socket says 172.21.15.167:42966) 2024-05-11T08:06:56.161 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.160+0000 7fcbaf7fe640 1 -- 172.21.15.167:0/2736973280 learned_addr learned my addr 172.21.15.167:0/2736973280 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:56.161 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.160+0000 7fcbb4d5e640 1 --2- 172.21.15.167:0/2736973280 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcbb01598d0 0x7fcbb0153720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:56.161 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.160+0000 7fcbaf7fe640 1 -- 172.21.15.167:0/2736973280 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcbb01598d0 msgr2=0x7fcbb0153720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:56.162 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.160+0000 7fcbaf7fe640 1 --2- 172.21.15.167:0/2736973280 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcbb01598d0 0x7fcbb0153720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:56.162 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.160+0000 7fcbaf7fe640 1 -- 172.21.15.167:0/2736973280 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcbb00cb6c0 msgr2=0x7fcbb0148da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:56.162 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.160+0000 7fcbaf7fe640 1 --2- 172.21.15.167:0/2736973280 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcbb00cb6c0 0x7fcbb0148da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:56.162 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.160+0000 7fcbaf7fe640 1 -- 172.21.15.167:0/2736973280 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbb0153ea0 con 0x7fcbb00d0640 2024-05-11T08:06:56.162 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.160+0000 7fcbaf7fe640 1 --2- 172.21.15.167:0/2736973280 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcbb00d0640 0x7fcbb0154f40 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fcba000c4d0 tx=0x7fcba000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:56.162 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.161+0000 7fcbacff9640 1 -- 172.21.15.167:0/2736973280 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcba0017070 con 0x7fcbb00d0640 2024-05-11T08:06:56.162 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.161+0000 7fcbacff9640 1 -- 172.21.15.167:0/2736973280 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcba0005d00 con 0x7fcbb00d0640 2024-05-11T08:06:56.162 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.161+0000 7fcbb67c1640 1 -- 172.21.15.167:0/2736973280 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbb0154180 con 0x7fcbb00d0640 2024-05-11T08:06:56.162 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.161+0000 7fcbacff9640 1 -- 172.21.15.167:0/2736973280 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcba00072b0 con 0x7fcbb00d0640 2024-05-11T08:06:56.162 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.161+0000 7fcbb67c1640 1 -- 172.21.15.167:0/2736973280 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbb0154740 con 0x7fcbb00d0640 2024-05-11T08:06:56.163 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.162+0000 7fcbb67c1640 1 -- 172.21.15.167:0/2736973280 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcbb0069590 con 0x7fcbb00d0640 2024-05-11T08:06:56.163 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.162+0000 7fcbacff9640 1 -- 172.21.15.167:0/2736973280 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fcba001a070 con 0x7fcbb00d0640 2024-05-11T08:06:56.163 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.163+0000 7fcbacff9640 1 --2- 172.21.15.167:0/2736973280 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fcb900762d0 0x7fcb90078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:56.163 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.163+0000 7fcbacff9640 1 -- 172.21.15.167:0/2736973280 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6101+0+0 (secure 0 0 0) 0x7fcba00969e0 con 0x7fcbb00d0640 2024-05-11T08:06:56.164 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.163+0000 7fcbaffff640 1 --2- 172.21.15.167:0/2736973280 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fcb900762d0 0x7fcb90078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:56.164 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.163+0000 7fcbaffff640 1 --2- 172.21.15.167:0/2736973280 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fcb900762d0 0x7fcb90078790 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fcb98009560 tx=0x7fcb98009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:56.167 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.167+0000 7fcbacff9640 1 -- 172.21.15.167:0/2736973280 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fcba005fb60 con 0x7fcbb00d0640 2024-05-11T08:06:56.347 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.348+0000 7fcbb67c1640 1 -- 172.21.15.167:0/2736973280 --> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fcbb00bb1f0 con 0x7fcb900762d0 2024-05-11T08:06:56.353 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.354+0000 7fcbacff9640 1 -- 172.21.15.167:0/2736973280 <== mgr.4113 v2:172.21.15.167:6832/253569496 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251948 (secure 0 0 0) 0x7fcbb00bb1f0 con 0x7fcb900762d0 2024-05-11T08:06:56.356 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T08:06:56.357 INFO:teuthology.orchestra.run.smithi167.stderr:dumped all 2024-05-11T08:06:56.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.357+0000 7fcbb67c1640 1 -- 172.21.15.167:0/2736973280 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fcb900762d0 msgr2=0x7fcb90078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:56.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.357+0000 7fcbb67c1640 1 --2- 172.21.15.167:0/2736973280 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fcb900762d0 0x7fcb90078790 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fcb98009560 tx=0x7fcb98009190 comp rx=0 tx=0).stop 2024-05-11T08:06:56.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.357+0000 7fcbb67c1640 1 -- 172.21.15.167:0/2736973280 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcbb00d0640 msgr2=0x7fcbb0154f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:56.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.357+0000 7fcbb67c1640 1 --2- 172.21.15.167:0/2736973280 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcbb00d0640 0x7fcbb0154f40 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fcba000c4d0 tx=0x7fcba000c990 comp rx=0 tx=0).stop 2024-05-11T08:06:56.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.357+0000 7fcbb67c1640 1 -- 172.21.15.167:0/2736973280 shutdown_connections 2024-05-11T08:06:56.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.357+0000 7fcbb67c1640 1 --2- 172.21.15.167:0/2736973280 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fcb900762d0 0x7fcb90078790 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:56.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.357+0000 7fcbb67c1640 1 --2- 172.21.15.167:0/2736973280 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcbb01598d0 0x7fcbb0153720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:56.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.357+0000 7fcbb67c1640 1 --2- 172.21.15.167:0/2736973280 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcbb00d0640 0x7fcbb0154f40 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:56.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.357+0000 7fcbb67c1640 1 --2- 172.21.15.167:0/2736973280 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcbb00cb6c0 0x7fcbb0148da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:56.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.357+0000 7fcbb67c1640 1 -- 172.21.15.167:0/2736973280 >> 172.21.15.167:0/2736973280 conn(0x7fcbb00d1ba0 msgr2=0x7fcbb00d2cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:56.360 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.357+0000 7fcbb67c1640 1 -- 172.21.15.167:0/2736973280 shutdown_connections 2024-05-11T08:06:56.360 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.357+0000 7fcbb67c1640 1 -- 172.21.15.167:0/2736973280 wait complete. 2024-05-11T08:06:56.368 INFO:teuthology.orchestra.run.smithi167.stdout:{"pg_ready":true,"pg_map":{"version":72,"stamp":"2024-05-11T08:06:55.928884+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":190,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":221088,"kb_used_data":4872,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753632,"statfs":{"total":773094113280,"available":772867719168,"internally_reserved":0,"allocated":4988928,"data_stored":2788188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001399"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T08:06:49.777575+0000","last_change":"2024-05-11T08:06:48.602205+0000","last_active":"2024-05-11T08:06:49.777575+0000","last_peered":"2024-05-11T08:06:49.777575+0000","last_clean":"2024-05-11T08:06:49.777575+0000","last_became_active":"2024-05-11T08:06:48.601905+0000","last_became_peered":"2024-05-11T08:06:48.601905+0000","last_unstale":"2024-05-11T08:06:49.777575+0000","last_undegraded":"2024-05-11T08:06:49.777575+0000","last_fullsized":"2024-05-11T08:06:49.777575+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:17:13.974448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T08:06:49.777160+0000","last_change":"2024-05-11T08:06:48.606760+0000","last_active":"2024-05-11T08:06:49.777160+0000","last_peered":"2024-05-11T08:06:49.777160+0000","last_clean":"2024-05-11T08:06:49.777160+0000","last_became_active":"2024-05-11T08:06:48.606450+0000","last_became_peered":"2024-05-11T08:06:48.606450+0000","last_unstale":"2024-05-11T08:06:49.777160+0000","last_undegraded":"2024-05-11T08:06:49.777160+0000","last_fullsized":"2024-05-11T08:06:49.777160+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:06:02.254447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T08:06:49.776398+0000","last_change":"2024-05-11T08:06:48.611469+0000","last_active":"2024-05-11T08:06:49.776398+0000","last_peered":"2024-05-11T08:06:49.776398+0000","last_clean":"2024-05-11T08:06:49.776398+0000","last_became_active":"2024-05-11T08:06:48.611198+0000","last_became_peered":"2024-05-11T08:06:48.611198+0000","last_unstale":"2024-05-11T08:06:49.776398+0000","last_undegraded":"2024-05-11T08:06:49.776398+0000","last_fullsized":"2024-05-11T08:06:49.776398+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:08:06.733168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T08:06:49.613242+0000","last_change":"2024-05-11T08:06:48.616894+0000","last_active":"2024-05-11T08:06:49.613242+0000","last_peered":"2024-05-11T08:06:49.613242+0000","last_clean":"2024-05-11T08:06:49.613242+0000","last_became_active":"2024-05-11T08:06:48.616597+0000","last_became_peered":"2024-05-11T08:06:48.616597+0000","last_unstale":"2024-05-11T08:06:49.613242+0000","last_undegraded":"2024-05-11T08:06:49.613242+0000","last_fullsized":"2024-05-11T08:06:49.613242+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:47:37.702924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T08:06:49.613143+0000","last_change":"2024-05-11T08:06:48.616535+0000","last_active":"2024-05-11T08:06:49.613143+0000","last_peered":"2024-05-11T08:06:49.613143+0000","last_clean":"2024-05-11T08:06:49.613143+0000","last_became_active":"2024-05-11T08:06:48.616283+0000","last_became_peered":"2024-05-11T08:06:48.616283+0000","last_unstale":"2024-05-11T08:06:49.613143+0000","last_undegraded":"2024-05-11T08:06:49.613143+0000","last_fullsized":"2024-05-11T08:06:49.613143+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:12:46.331254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T08:06:49.776671+0000","last_change":"2024-05-11T08:06:48.597463+0000","last_active":"2024-05-11T08:06:49.776671+0000","last_peered":"2024-05-11T08:06:49.776671+0000","last_clean":"2024-05-11T08:06:49.776671+0000","last_became_active":"2024-05-11T08:06:48.597210+0000","last_became_peered":"2024-05-11T08:06:48.597210+0000","last_unstale":"2024-05-11T08:06:49.776671+0000","last_undegraded":"2024-05-11T08:06:49.776671+0000","last_fullsized":"2024-05-11T08:06:49.776671+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:47:47.699231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T08:06:49.598666+0000","last_change":"2024-05-11T08:06:48.600350+0000","last_active":"2024-05-11T08:06:49.598666+0000","last_peered":"2024-05-11T08:06:49.598666+0000","last_clean":"2024-05-11T08:06:49.598666+0000","last_became_active":"2024-05-11T08:06:48.600051+0000","last_became_peered":"2024-05-11T08:06:48.600051+0000","last_unstale":"2024-05-11T08:06:49.598666+0000","last_undegraded":"2024-05-11T08:06:49.598666+0000","last_fullsized":"2024-05-11T08:06:49.598666+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:42:04.739319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T08:06:49.777285+0000","last_change":"2024-05-11T08:06:48.612999+0000","last_active":"2024-05-11T08:06:49.777285+0000","last_peered":"2024-05-11T08:06:49.777285+0000","last_clean":"2024-05-11T08:06:49.777285+0000","last_became_active":"2024-05-11T08:06:48.612670+0000","last_became_peered":"2024-05-11T08:06:48.612670+0000","last_unstale":"2024-05-11T08:06:49.777285+0000","last_undegraded":"2024-05-11T08:06:49.777285+0000","last_fullsized":"2024-05-11T08:06:49.777285+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:03:10.292955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T08:06:49.777123+0000","last_change":"2024-05-11T08:06:48.618247+0000","last_active":"2024-05-11T08:06:49.777123+0000","last_peered":"2024-05-11T08:06:49.777123+0000","last_clean":"2024-05-11T08:06:49.777123+0000","last_became_active":"2024-05-11T08:06:48.617610+0000","last_became_peered":"2024-05-11T08:06:48.617610+0000","last_unstale":"2024-05-11T08:06:49.777123+0000","last_undegraded":"2024-05-11T08:06:49.777123+0000","last_fullsized":"2024-05-11T08:06:49.777123+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:25:20.050736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T08:06:49.599064+0000","last_change":"2024-05-11T08:06:48.616462+0000","last_active":"2024-05-11T08:06:49.599064+0000","last_peered":"2024-05-11T08:06:49.599064+0000","last_clean":"2024-05-11T08:06:49.599064+0000","last_became_active":"2024-05-11T08:06:48.616163+0000","last_became_peered":"2024-05-11T08:06:48.616163+0000","last_unstale":"2024-05-11T08:06:49.599064+0000","last_undegraded":"2024-05-11T08:06:49.599064+0000","last_fullsized":"2024-05-11T08:06:49.599064+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:59:44.981066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T08:06:49.599033+0000","last_change":"2024-05-11T08:06:48.617983+0000","last_active":"2024-05-11T08:06:49.599033+0000","last_peered":"2024-05-11T08:06:49.599033+0000","last_clean":"2024-05-11T08:06:49.599033+0000","last_became_active":"2024-05-11T08:06:48.617720+0000","last_became_peered":"2024-05-11T08:06:48.617720+0000","last_unstale":"2024-05-11T08:06:49.599033+0000","last_undegraded":"2024-05-11T08:06:49.599033+0000","last_fullsized":"2024-05-11T08:06:49.599033+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:45:33.813046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T08:06:49.776231+0000","last_change":"2024-05-11T08:06:48.607575+0000","last_active":"2024-05-11T08:06:49.776231+0000","last_peered":"2024-05-11T08:06:49.776231+0000","last_clean":"2024-05-11T08:06:49.776231+0000","last_became_active":"2024-05-11T08:06:48.607274+0000","last_became_peered":"2024-05-11T08:06:48.607274+0000","last_unstale":"2024-05-11T08:06:49.776231+0000","last_undegraded":"2024-05-11T08:06:49.776231+0000","last_fullsized":"2024-05-11T08:06:49.776231+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:19:11.149837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T08:06:49.598565+0000","last_change":"2024-05-11T08:06:48.617158+0000","last_active":"2024-05-11T08:06:49.598565+0000","last_peered":"2024-05-11T08:06:49.598565+0000","last_clean":"2024-05-11T08:06:49.598565+0000","last_became_active":"2024-05-11T08:06:48.616871+0000","last_became_peered":"2024-05-11T08:06:48.616871+0000","last_unstale":"2024-05-11T08:06:49.598565+0000","last_undegraded":"2024-05-11T08:06:49.598565+0000","last_fullsized":"2024-05-11T08:06:49.598565+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:31:37.112684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T08:06:49.776340+0000","last_change":"2024-05-11T08:06:48.618448+0000","last_active":"2024-05-11T08:06:49.776340+0000","last_peered":"2024-05-11T08:06:49.776340+0000","last_clean":"2024-05-11T08:06:49.776340+0000","last_became_active":"2024-05-11T08:06:48.618144+0000","last_became_peered":"2024-05-11T08:06:48.618144+0000","last_unstale":"2024-05-11T08:06:49.776340+0000","last_undegraded":"2024-05-11T08:06:49.776340+0000","last_fullsized":"2024-05-11T08:06:49.776340+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:30:16.719412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T08:06:49.598568+0000","last_change":"2024-05-11T08:06:48.606485+0000","last_active":"2024-05-11T08:06:49.598568+0000","last_peered":"2024-05-11T08:06:49.598568+0000","last_clean":"2024-05-11T08:06:49.598568+0000","last_became_active":"2024-05-11T08:06:48.606206+0000","last_became_peered":"2024-05-11T08:06:48.606206+0000","last_unstale":"2024-05-11T08:06:49.598568+0000","last_undegraded":"2024-05-11T08:06:49.598568+0000","last_fullsized":"2024-05-11T08:06:49.598568+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:25:20.050736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T08:06:49.596997+0000","last_change":"2024-05-11T08:06:48.620516+0000","last_active":"2024-05-11T08:06:49.596997+0000","last_peered":"2024-05-11T08:06:49.596997+0000","last_clean":"2024-05-11T08:06:49.596997+0000","last_became_active":"2024-05-11T08:06:48.619655+0000","last_became_peered":"2024-05-11T08:06:48.619655+0000","last_unstale":"2024-05-11T08:06:49.596997+0000","last_undegraded":"2024-05-11T08:06:49.596997+0000","last_fullsized":"2024-05-11T08:06:49.596997+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:25:20.050736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T08:06:49.597188+0000","last_change":"2024-05-11T08:06:48.623749+0000","last_active":"2024-05-11T08:06:49.597188+0000","last_peered":"2024-05-11T08:06:49.597188+0000","last_clean":"2024-05-11T08:06:49.597188+0000","last_became_active":"2024-05-11T08:06:48.623479+0000","last_became_peered":"2024-05-11T08:06:48.623479+0000","last_unstale":"2024-05-11T08:06:49.597188+0000","last_undegraded":"2024-05-11T08:06:49.597188+0000","last_fullsized":"2024-05-11T08:06:49.597188+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:04:46.093003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T08:06:49.597121+0000","last_change":"2024-05-11T08:06:48.619400+0000","last_active":"2024-05-11T08:06:49.597121+0000","last_peered":"2024-05-11T08:06:49.597121+0000","last_clean":"2024-05-11T08:06:49.597121+0000","last_became_active":"2024-05-11T08:06:48.619125+0000","last_became_peered":"2024-05-11T08:06:48.619125+0000","last_unstale":"2024-05-11T08:06:49.597121+0000","last_undegraded":"2024-05-11T08:06:49.597121+0000","last_fullsized":"2024-05-11T08:06:49.597121+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:30:16.719412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T08:06:49.613351+0000","last_change":"2024-05-11T08:06:48.606813+0000","last_active":"2024-05-11T08:06:49.613351+0000","last_peered":"2024-05-11T08:06:49.613351+0000","last_clean":"2024-05-11T08:06:49.613351+0000","last_became_active":"2024-05-11T08:06:48.606548+0000","last_became_peered":"2024-05-11T08:06:48.606548+0000","last_unstale":"2024-05-11T08:06:49.613351+0000","last_undegraded":"2024-05-11T08:06:49.613351+0000","last_fullsized":"2024-05-11T08:06:49.613351+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:40:53.505484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T08:06:49.596816+0000","last_change":"2024-05-11T08:06:48.618288+0000","last_active":"2024-05-11T08:06:49.596816+0000","last_peered":"2024-05-11T08:06:49.596816+0000","last_clean":"2024-05-11T08:06:49.596816+0000","last_became_active":"2024-05-11T08:06:48.618026+0000","last_became_peered":"2024-05-11T08:06:48.618026+0000","last_unstale":"2024-05-11T08:06:49.596816+0000","last_undegraded":"2024-05-11T08:06:49.596816+0000","last_fullsized":"2024-05-11T08:06:49.596816+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:34:42.615373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T08:06:49.598674+0000","last_change":"2024-05-11T08:06:48.600497+0000","last_active":"2024-05-11T08:06:49.598674+0000","last_peered":"2024-05-11T08:06:49.598674+0000","last_clean":"2024-05-11T08:06:49.598674+0000","last_became_active":"2024-05-11T08:06:48.599938+0000","last_became_peered":"2024-05-11T08:06:48.599938+0000","last_unstale":"2024-05-11T08:06:49.598674+0000","last_undegraded":"2024-05-11T08:06:49.598674+0000","last_fullsized":"2024-05-11T08:06:49.598674+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:06:02.254447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T08:06:49.776727+0000","last_change":"2024-05-11T08:06:48.614321+0000","last_active":"2024-05-11T08:06:49.776727+0000","last_peered":"2024-05-11T08:06:49.776727+0000","last_clean":"2024-05-11T08:06:49.776727+0000","last_became_active":"2024-05-11T08:06:48.613949+0000","last_became_peered":"2024-05-11T08:06:48.613949+0000","last_unstale":"2024-05-11T08:06:49.776727+0000","last_undegraded":"2024-05-11T08:06:49.776727+0000","last_fullsized":"2024-05-11T08:06:49.776727+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:58:24.559024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T08:06:49.597253+0000","last_change":"2024-05-11T08:06:48.613879+0000","last_active":"2024-05-11T08:06:49.597253+0000","last_peered":"2024-05-11T08:06:49.597253+0000","last_clean":"2024-05-11T08:06:49.597253+0000","last_became_active":"2024-05-11T08:06:48.613172+0000","last_became_peered":"2024-05-11T08:06:48.613172+0000","last_unstale":"2024-05-11T08:06:49.597253+0000","last_undegraded":"2024-05-11T08:06:49.597253+0000","last_fullsized":"2024-05-11T08:06:49.597253+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:31:37.112684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T08:06:49.613144+0000","last_change":"2024-05-11T08:06:48.615211+0000","last_active":"2024-05-11T08:06:49.613144+0000","last_peered":"2024-05-11T08:06:49.613144+0000","last_clean":"2024-05-11T08:06:49.613144+0000","last_became_active":"2024-05-11T08:06:48.614902+0000","last_became_peered":"2024-05-11T08:06:48.614902+0000","last_unstale":"2024-05-11T08:06:49.613144+0000","last_undegraded":"2024-05-11T08:06:49.613144+0000","last_fullsized":"2024-05-11T08:06:49.613144+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:20:23.234025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T08:06:49.597132+0000","last_change":"2024-05-11T08:06:48.611481+0000","last_active":"2024-05-11T08:06:49.597132+0000","last_peered":"2024-05-11T08:06:49.597132+0000","last_clean":"2024-05-11T08:06:49.597132+0000","last_became_active":"2024-05-11T08:06:48.611159+0000","last_became_peered":"2024-05-11T08:06:48.611159+0000","last_unstale":"2024-05-11T08:06:49.597132+0000","last_undegraded":"2024-05-11T08:06:49.597132+0000","last_fullsized":"2024-05-11T08:06:49.597132+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:47:47.699231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T08:06:49.776833+0000","last_change":"2024-05-11T08:06:48.612158+0000","last_active":"2024-05-11T08:06:49.776833+0000","last_peered":"2024-05-11T08:06:49.776833+0000","last_clean":"2024-05-11T08:06:49.776833+0000","last_became_active":"2024-05-11T08:06:48.611173+0000","last_became_peered":"2024-05-11T08:06:48.611173+0000","last_unstale":"2024-05-11T08:06:49.776833+0000","last_undegraded":"2024-05-11T08:06:49.776833+0000","last_fullsized":"2024-05-11T08:06:49.776833+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:58:19.327537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T08:06:49.597088+0000","last_change":"2024-05-11T08:06:48.609497+0000","last_active":"2024-05-11T08:06:49.597088+0000","last_peered":"2024-05-11T08:06:49.597088+0000","last_clean":"2024-05-11T08:06:49.597088+0000","last_became_active":"2024-05-11T08:06:48.609168+0000","last_became_peered":"2024-05-11T08:06:48.609168+0000","last_unstale":"2024-05-11T08:06:49.597088+0000","last_undegraded":"2024-05-11T08:06:49.597088+0000","last_fullsized":"2024-05-11T08:06:49.597088+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:22:43.875847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T08:06:49.598926+0000","last_change":"2024-05-11T08:06:48.600257+0000","last_active":"2024-05-11T08:06:49.598926+0000","last_peered":"2024-05-11T08:06:49.598926+0000","last_clean":"2024-05-11T08:06:49.598926+0000","last_became_active":"2024-05-11T08:06:48.599935+0000","last_became_peered":"2024-05-11T08:06:48.599935+0000","last_unstale":"2024-05-11T08:06:49.598926+0000","last_undegraded":"2024-05-11T08:06:49.598926+0000","last_fullsized":"2024-05-11T08:06:49.598926+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:03:10.292955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T08:06:49.596653+0000","last_change":"2024-05-11T08:06:48.610080+0000","last_active":"2024-05-11T08:06:49.596653+0000","last_peered":"2024-05-11T08:06:49.596653+0000","last_clean":"2024-05-11T08:06:49.596653+0000","last_became_active":"2024-05-11T08:06:48.609811+0000","last_became_peered":"2024-05-11T08:06:48.609811+0000","last_unstale":"2024-05-11T08:06:49.596653+0000","last_undegraded":"2024-05-11T08:06:49.596653+0000","last_fullsized":"2024-05-11T08:06:49.596653+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:43:13.666354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T08:06:49.596732+0000","last_change":"2024-05-11T08:06:48.610852+0000","last_active":"2024-05-11T08:06:49.596732+0000","last_peered":"2024-05-11T08:06:49.596732+0000","last_clean":"2024-05-11T08:06:49.596732+0000","last_became_active":"2024-05-11T08:06:48.610601+0000","last_became_peered":"2024-05-11T08:06:48.610601+0000","last_unstale":"2024-05-11T08:06:49.596732+0000","last_undegraded":"2024-05-11T08:06:49.596732+0000","last_fullsized":"2024-05-11T08:06:49.596732+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:52:51.037946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T08:06:49.597444+0000","last_change":"2024-05-11T08:06:48.614099+0000","last_active":"2024-05-11T08:06:49.597444+0000","last_peered":"2024-05-11T08:06:49.597444+0000","last_clean":"2024-05-11T08:06:49.597444+0000","last_became_active":"2024-05-11T08:06:48.613538+0000","last_became_peered":"2024-05-11T08:06:48.613538+0000","last_unstale":"2024-05-11T08:06:49.597444+0000","last_undegraded":"2024-05-11T08:06:49.597444+0000","last_fullsized":"2024-05-11T08:06:49.597444+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:25:08.628454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T08:06:49.776419+0000","last_change":"2024-05-11T08:06:48.603886+0000","last_active":"2024-05-11T08:06:49.776419+0000","last_peered":"2024-05-11T08:06:49.776419+0000","last_clean":"2024-05-11T08:06:49.776419+0000","last_became_active":"2024-05-11T08:06:48.603624+0000","last_became_peered":"2024-05-11T08:06:48.603624+0000","last_unstale":"2024-05-11T08:06:49.776419+0000","last_undegraded":"2024-05-11T08:06:49.776419+0000","last_fullsized":"2024-05-11T08:06:49.776419+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:22:43.875847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T08:06:49.597666+0000","last_change":"2024-05-11T08:06:48.618092+0000","last_active":"2024-05-11T08:06:49.597666+0000","last_peered":"2024-05-11T08:06:49.597666+0000","last_clean":"2024-05-11T08:06:49.597666+0000","last_became_active":"2024-05-11T08:06:48.617801+0000","last_became_peered":"2024-05-11T08:06:48.617801+0000","last_unstale":"2024-05-11T08:06:49.597666+0000","last_undegraded":"2024-05-11T08:06:49.597666+0000","last_fullsized":"2024-05-11T08:06:49.597666+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:06:02.254447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"40'4","reported_seq":18,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T08:06:49.597302+0000","last_change":"2024-05-11T08:06:47.591074+0000","last_active":"2024-05-11T08:06:49.597302+0000","last_peered":"2024-05-11T08:06:49.597302+0000","last_clean":"2024-05-11T08:06:49.597302+0000","last_became_active":"2024-05-11T08:06:47.590747+0000","last_became_peered":"2024-05-11T08:06:47.590747+0000","last_unstale":"2024-05-11T08:06:49.597302+0000","last_undegraded":"2024-05-11T08:06:49.597302+0000","last_fullsized":"2024-05-11T08:06:49.597302+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:38:12.898297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T08:06:49.777121+0000","last_change":"2024-05-11T08:06:48.611441+0000","last_active":"2024-05-11T08:06:49.777121+0000","last_peered":"2024-05-11T08:06:49.777121+0000","last_clean":"2024-05-11T08:06:49.777121+0000","last_became_active":"2024-05-11T08:06:48.611090+0000","last_became_peered":"2024-05-11T08:06:48.611090+0000","last_unstale":"2024-05-11T08:06:49.777121+0000","last_undegraded":"2024-05-11T08:06:49.777121+0000","last_fullsized":"2024-05-11T08:06:49.777121+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:19:11.149837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"40'1","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T08:06:49.776988+0000","last_change":"2024-05-11T08:06:47.597547+0000","last_active":"2024-05-11T08:06:49.776988+0000","last_peered":"2024-05-11T08:06:49.776988+0000","last_clean":"2024-05-11T08:06:49.776988+0000","last_became_active":"2024-05-11T08:06:47.597260+0000","last_became_peered":"2024-05-11T08:06:47.597260+0000","last_unstale":"2024-05-11T08:06:49.776988+0000","last_undegraded":"2024-05-11T08:06:49.776988+0000","last_fullsized":"2024-05-11T08:06:49.776988+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:58:03.492050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T08:06:49.776723+0000","last_change":"2024-05-11T08:06:48.620646+0000","last_active":"2024-05-11T08:06:49.776723+0000","last_peered":"2024-05-11T08:06:49.776723+0000","last_clean":"2024-05-11T08:06:49.776723+0000","last_became_active":"2024-05-11T08:06:48.617771+0000","last_became_peered":"2024-05-11T08:06:48.617771+0000","last_unstale":"2024-05-11T08:06:49.776723+0000","last_undegraded":"2024-05-11T08:06:49.776723+0000","last_fullsized":"2024-05-11T08:06:49.776723+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T08:08:06.733168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"40'2","reported_seq":16,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T08:06:49.777269+0000","last_change":"2024-05-11T08:06:47.597745+0000","last_active":"2024-05-11T08:06:49.777269+0000","last_peered":"2024-05-11T08:06:49.777269+0000","last_clean":"2024-05-11T08:06:49.777269+0000","last_became_active":"2024-05-11T08:06:47.597472+0000","last_became_peered":"2024-05-11T08:06:47.597472+0000","last_unstale":"2024-05-11T08:06:49.777269+0000","last_undegraded":"2024-05-11T08:06:49.777269+0000","last_fullsized":"2024-05-11T08:06:49.777269+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:13:57.487739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T08:06:49.777551+0000","last_change":"2024-05-11T08:06:48.613655+0000","last_active":"2024-05-11T08:06:49.777551+0000","last_peered":"2024-05-11T08:06:49.777551+0000","last_clean":"2024-05-11T08:06:49.777551+0000","last_became_active":"2024-05-11T08:06:48.613360+0000","last_became_peered":"2024-05-11T08:06:48.613360+0000","last_unstale":"2024-05-11T08:06:49.777551+0000","last_undegraded":"2024-05-11T08:06:49.777551+0000","last_fullsized":"2024-05-11T08:06:49.777551+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:31:11.314582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"40'2","reported_seq":16,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T08:06:49.776571+0000","last_change":"2024-05-11T08:06:47.594580+0000","last_active":"2024-05-11T08:06:49.776571+0000","last_peered":"2024-05-11T08:06:49.776571+0000","last_clean":"2024-05-11T08:06:49.776571+0000","last_became_active":"2024-05-11T08:06:47.594315+0000","last_became_peered":"2024-05-11T08:06:47.594315+0000","last_unstale":"2024-05-11T08:06:49.776571+0000","last_undegraded":"2024-05-11T08:06:49.776571+0000","last_fullsized":"2024-05-11T08:06:49.776571+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:12:27.297983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T08:06:49.776501+0000","last_change":"2024-05-11T08:06:48.604680+0000","last_active":"2024-05-11T08:06:49.776501+0000","last_peered":"2024-05-11T08:06:49.776501+0000","last_clean":"2024-05-11T08:06:49.776501+0000","last_became_active":"2024-05-11T08:06:48.604399+0000","last_became_peered":"2024-05-11T08:06:48.604399+0000","last_unstale":"2024-05-11T08:06:49.776501+0000","last_undegraded":"2024-05-11T08:06:49.776501+0000","last_fullsized":"2024-05-11T08:06:49.776501+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:32:56.714452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"40'1","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T08:06:49.598796+0000","last_change":"2024-05-11T08:06:47.600311+0000","last_active":"2024-05-11T08:06:49.598796+0000","last_peered":"2024-05-11T08:06:49.598796+0000","last_clean":"2024-05-11T08:06:49.598796+0000","last_became_active":"2024-05-11T08:06:47.600072+0000","last_became_peered":"2024-05-11T08:06:47.600072+0000","last_unstale":"2024-05-11T08:06:49.598796+0000","last_undegraded":"2024-05-11T08:06:49.598796+0000","last_fullsized":"2024-05-11T08:06:49.598796+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:58:03.492050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T08:06:49.598813+0000","last_change":"2024-05-11T08:06:48.602490+0000","last_active":"2024-05-11T08:06:49.598813+0000","last_peered":"2024-05-11T08:06:49.598813+0000","last_clean":"2024-05-11T08:06:49.598813+0000","last_became_active":"2024-05-11T08:06:48.602062+0000","last_became_peered":"2024-05-11T08:06:48.602062+0000","last_unstale":"2024-05-11T08:06:49.598813+0000","last_undegraded":"2024-05-11T08:06:49.598813+0000","last_fullsized":"2024-05-11T08:06:49.598813+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:25:08.628454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"40'1","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T08:06:49.597497+0000","last_change":"2024-05-11T08:06:47.598857+0000","last_active":"2024-05-11T08:06:49.597497+0000","last_peered":"2024-05-11T08:06:49.597497+0000","last_clean":"2024-05-11T08:06:49.597497+0000","last_became_active":"2024-05-11T08:06:47.598102+0000","last_became_peered":"2024-05-11T08:06:47.598102+0000","last_unstale":"2024-05-11T08:06:49.597497+0000","last_undegraded":"2024-05-11T08:06:49.597497+0000","last_fullsized":"2024-05-11T08:06:49.597497+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:58:03.492050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T08:06:49.776687+0000","last_change":"2024-05-11T08:06:48.614661+0000","last_active":"2024-05-11T08:06:49.776687+0000","last_peered":"2024-05-11T08:06:49.776687+0000","last_clean":"2024-05-11T08:06:49.776687+0000","last_became_active":"2024-05-11T08:06:48.614351+0000","last_became_peered":"2024-05-11T08:06:48.614351+0000","last_unstale":"2024-05-11T08:06:49.776687+0000","last_undegraded":"2024-05-11T08:06:49.776687+0000","last_fullsized":"2024-05-11T08:06:49.776687+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:19:11.149837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":14,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T08:06:49.776345+0000","last_change":"2024-05-11T08:06:47.595791+0000","last_active":"2024-05-11T08:06:49.776345+0000","last_peered":"2024-05-11T08:06:49.776345+0000","last_clean":"2024-05-11T08:06:49.776345+0000","last_became_active":"2024-05-11T08:06:47.595551+0000","last_became_peered":"2024-05-11T08:06:47.595551+0000","last_unstale":"2024-05-11T08:06:49.776345+0000","last_undegraded":"2024-05-11T08:06:49.776345+0000","last_fullsized":"2024-05-11T08:06:49.776345+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:19:14.446185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T08:06:49.777316+0000","last_change":"2024-05-11T08:06:48.612558+0000","last_active":"2024-05-11T08:06:49.777316+0000","last_peered":"2024-05-11T08:06:49.777316+0000","last_clean":"2024-05-11T08:06:49.777316+0000","last_became_active":"2024-05-11T08:06:48.611937+0000","last_became_peered":"2024-05-11T08:06:48.611937+0000","last_unstale":"2024-05-11T08:06:49.777316+0000","last_undegraded":"2024-05-11T08:06:49.777316+0000","last_fullsized":"2024-05-11T08:06:49.777316+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:59:44.981066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":14,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T08:06:49.776245+0000","last_change":"2024-05-11T08:06:47.590670+0000","last_active":"2024-05-11T08:06:49.776245+0000","last_peered":"2024-05-11T08:06:49.776245+0000","last_clean":"2024-05-11T08:06:49.776245+0000","last_became_active":"2024-05-11T08:06:47.590380+0000","last_became_peered":"2024-05-11T08:06:47.590380+0000","last_unstale":"2024-05-11T08:06:49.776245+0000","last_undegraded":"2024-05-11T08:06:49.776245+0000","last_fullsized":"2024-05-11T08:06:49.776245+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:19:14.446185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T08:06:49.613537+0000","last_change":"2024-05-11T08:06:48.616855+0000","last_active":"2024-05-11T08:06:49.613537+0000","last_peered":"2024-05-11T08:06:49.613537+0000","last_clean":"2024-05-11T08:06:49.613537+0000","last_became_active":"2024-05-11T08:06:48.616287+0000","last_became_peered":"2024-05-11T08:06:48.616287+0000","last_unstale":"2024-05-11T08:06:49.613537+0000","last_undegraded":"2024-05-11T08:06:49.613537+0000","last_fullsized":"2024-05-11T08:06:49.613537+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:31:41.113021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":14,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T08:06:49.777599+0000","last_change":"2024-05-11T08:06:47.590630+0000","last_active":"2024-05-11T08:06:49.777599+0000","last_peered":"2024-05-11T08:06:49.777599+0000","last_clean":"2024-05-11T08:06:49.777599+0000","last_became_active":"2024-05-11T08:06:47.590361+0000","last_became_peered":"2024-05-11T08:06:47.590361+0000","last_unstale":"2024-05-11T08:06:49.777599+0000","last_undegraded":"2024-05-11T08:06:49.777599+0000","last_fullsized":"2024-05-11T08:06:49.777599+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:43:12.657647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T08:06:49.597916+0000","last_change":"2024-05-11T08:06:48.617983+0000","last_active":"2024-05-11T08:06:49.597916+0000","last_peered":"2024-05-11T08:06:49.597916+0000","last_clean":"2024-05-11T08:06:49.597916+0000","last_became_active":"2024-05-11T08:06:48.617701+0000","last_became_peered":"2024-05-11T08:06:48.617701+0000","last_unstale":"2024-05-11T08:06:49.597916+0000","last_undegraded":"2024-05-11T08:06:49.597916+0000","last_fullsized":"2024-05-11T08:06:49.597916+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:03:10.292955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":14,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T08:06:49.599141+0000","last_change":"2024-05-11T08:06:47.587418+0000","last_active":"2024-05-11T08:06:49.599141+0000","last_peered":"2024-05-11T08:06:49.599141+0000","last_clean":"2024-05-11T08:06:49.599141+0000","last_became_active":"2024-05-11T08:06:47.586715+0000","last_became_peered":"2024-05-11T08:06:47.586715+0000","last_unstale":"2024-05-11T08:06:49.599141+0000","last_undegraded":"2024-05-11T08:06:49.599141+0000","last_fullsized":"2024-05-11T08:06:49.599141+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:49:09.098705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T08:06:49.776189+0000","last_change":"2024-05-11T08:06:48.612026+0000","last_active":"2024-05-11T08:06:49.776189+0000","last_peered":"2024-05-11T08:06:49.776189+0000","last_clean":"2024-05-11T08:06:49.776189+0000","last_became_active":"2024-05-11T08:06:48.611657+0000","last_became_peered":"2024-05-11T08:06:48.611657+0000","last_unstale":"2024-05-11T08:06:49.776189+0000","last_undegraded":"2024-05-11T08:06:49.776189+0000","last_fullsized":"2024-05-11T08:06:49.776189+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:47:47.699231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"40'3","reported_seq":17,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T08:06:49.599120+0000","last_change":"2024-05-11T08:06:47.594319+0000","last_active":"2024-05-11T08:06:49.599120+0000","last_peered":"2024-05-11T08:06:49.599120+0000","last_clean":"2024-05-11T08:06:49.599120+0000","last_became_active":"2024-05-11T08:06:47.593840+0000","last_became_peered":"2024-05-11T08:06:47.593840+0000","last_unstale":"2024-05-11T08:06:49.599120+0000","last_undegraded":"2024-05-11T08:06:49.599120+0000","last_fullsized":"2024-05-11T08:06:49.599120+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:43:12.657647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T08:06:49.776694+0000","last_change":"2024-05-11T08:06:48.614547+0000","last_active":"2024-05-11T08:06:49.776694+0000","last_peered":"2024-05-11T08:06:49.776694+0000","last_clean":"2024-05-11T08:06:49.776694+0000","last_became_active":"2024-05-11T08:06:48.614269+0000","last_became_peered":"2024-05-11T08:06:48.614269+0000","last_unstale":"2024-05-11T08:06:49.776694+0000","last_undegraded":"2024-05-11T08:06:49.776694+0000","last_fullsized":"2024-05-11T08:06:49.776694+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:48:10.064056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"40'6","reported_seq":20,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T08:06:49.777131+0000","last_change":"2024-05-11T08:06:47.592968+0000","last_active":"2024-05-11T08:06:49.777131+0000","last_peered":"2024-05-11T08:06:49.777131+0000","last_clean":"2024-05-11T08:06:49.777131+0000","last_became_active":"2024-05-11T08:06:47.592672+0000","last_became_peered":"2024-05-11T08:06:47.592672+0000","last_unstale":"2024-05-11T08:06:49.777131+0000","last_undegraded":"2024-05-11T08:06:49.777131+0000","last_fullsized":"2024-05-11T08:06:49.777131+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:34:41.606665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T08:06:49.776213+0000","last_change":"2024-05-11T08:06:48.617943+0000","last_active":"2024-05-11T08:06:49.776213+0000","last_peered":"2024-05-11T08:06:49.776213+0000","last_clean":"2024-05-11T08:06:49.776213+0000","last_became_active":"2024-05-11T08:06:48.617621+0000","last_became_peered":"2024-05-11T08:06:48.617621+0000","last_unstale":"2024-05-11T08:06:49.776213+0000","last_undegraded":"2024-05-11T08:06:49.776213+0000","last_fullsized":"2024-05-11T08:06:49.776213+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:58:19.327537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"40'2","reported_seq":16,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T08:06:49.777472+0000","last_change":"2024-05-11T08:06:47.600533+0000","last_active":"2024-05-11T08:06:49.777472+0000","last_peered":"2024-05-11T08:06:49.777472+0000","last_clean":"2024-05-11T08:06:49.777472+0000","last_became_active":"2024-05-11T08:06:47.600226+0000","last_became_peered":"2024-05-11T08:06:47.600226+0000","last_unstale":"2024-05-11T08:06:49.777472+0000","last_undegraded":"2024-05-11T08:06:49.777472+0000","last_fullsized":"2024-05-11T08:06:49.777472+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:52:50.029239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T08:06:49.776877+0000","last_change":"2024-05-11T08:06:48.602152+0000","last_active":"2024-05-11T08:06:49.776877+0000","last_peered":"2024-05-11T08:06:49.776877+0000","last_clean":"2024-05-11T08:06:49.776877+0000","last_became_active":"2024-05-11T08:06:48.601878+0000","last_became_peered":"2024-05-11T08:06:48.601878+0000","last_unstale":"2024-05-11T08:06:49.776877+0000","last_undegraded":"2024-05-11T08:06:49.776877+0000","last_fullsized":"2024-05-11T08:06:49.776877+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:58:03.309459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":14,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T08:06:49.776425+0000","last_change":"2024-05-11T08:06:47.597276+0000","last_active":"2024-05-11T08:06:49.776425+0000","last_peered":"2024-05-11T08:06:49.776425+0000","last_clean":"2024-05-11T08:06:49.776425+0000","last_became_active":"2024-05-11T08:06:47.596986+0000","last_became_peered":"2024-05-11T08:06:47.596986+0000","last_unstale":"2024-05-11T08:06:49.776425+0000","last_undegraded":"2024-05-11T08:06:49.776425+0000","last_fullsized":"2024-05-11T08:06:49.776425+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:38:12.898297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T08:06:49.597321+0000","last_change":"2024-05-11T08:06:48.613220+0000","last_active":"2024-05-11T08:06:49.597321+0000","last_peered":"2024-05-11T08:06:49.597321+0000","last_clean":"2024-05-11T08:06:49.597321+0000","last_became_active":"2024-05-11T08:06:48.612892+0000","last_became_peered":"2024-05-11T08:06:48.612892+0000","last_unstale":"2024-05-11T08:06:49.597321+0000","last_undegraded":"2024-05-11T08:06:49.597321+0000","last_fullsized":"2024-05-11T08:06:49.597321+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:42:04.739319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":14,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T08:06:49.596439+0000","last_change":"2024-05-11T08:06:47.591052+0000","last_active":"2024-05-11T08:06:49.596439+0000","last_peered":"2024-05-11T08:06:49.596439+0000","last_clean":"2024-05-11T08:06:49.596439+0000","last_became_active":"2024-05-11T08:06:47.590726+0000","last_became_peered":"2024-05-11T08:06:47.590726+0000","last_unstale":"2024-05-11T08:06:49.596439+0000","last_undegraded":"2024-05-11T08:06:49.596439+0000","last_fullsized":"2024-05-11T08:06:49.596439+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:08:22.364810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T08:06:49.777410+0000","last_change":"2024-05-11T08:06:48.602834+0000","last_active":"2024-05-11T08:06:49.777410+0000","last_peered":"2024-05-11T08:06:49.777410+0000","last_clean":"2024-05-11T08:06:49.777410+0000","last_became_active":"2024-05-11T08:06:48.602590+0000","last_became_peered":"2024-05-11T08:06:48.602590+0000","last_unstale":"2024-05-11T08:06:49.777410+0000","last_undegraded":"2024-05-11T08:06:49.777410+0000","last_fullsized":"2024-05-11T08:06:49.777410+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:04:46.093003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"40'8","reported_seq":22,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T08:06:49.597100+0000","last_change":"2024-05-11T08:06:47.597077+0000","last_active":"2024-05-11T08:06:49.597100+0000","last_peered":"2024-05-11T08:06:49.597100+0000","last_clean":"2024-05-11T08:06:49.597100+0000","last_became_active":"2024-05-11T08:06:47.596753+0000","last_became_peered":"2024-05-11T08:06:47.596753+0000","last_unstale":"2024-05-11T08:06:49.597100+0000","last_undegraded":"2024-05-11T08:06:49.597100+0000","last_fullsized":"2024-05-11T08:06:49.597100+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:20:42.258886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T08:06:49.777042+0000","last_change":"2024-05-11T08:06:48.605503+0000","last_active":"2024-05-11T08:06:49.777042+0000","last_peered":"2024-05-11T08:06:49.777042+0000","last_clean":"2024-05-11T08:06:49.777042+0000","last_became_active":"2024-05-11T08:06:48.605214+0000","last_became_peered":"2024-05-11T08:06:48.605214+0000","last_unstale":"2024-05-11T08:06:49.777042+0000","last_undegraded":"2024-05-11T08:06:49.777042+0000","last_fullsized":"2024-05-11T08:06:49.777042+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:25:08.628454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"40'1","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T08:06:49.598959+0000","last_change":"2024-05-11T08:06:47.592140+0000","last_active":"2024-05-11T08:06:49.598959+0000","last_peered":"2024-05-11T08:06:49.598959+0000","last_clean":"2024-05-11T08:06:49.598959+0000","last_became_active":"2024-05-11T08:06:47.591867+0000","last_became_peered":"2024-05-11T08:06:47.591867+0000","last_unstale":"2024-05-11T08:06:49.598959+0000","last_undegraded":"2024-05-11T08:06:49.598959+0000","last_fullsized":"2024-05-11T08:06:49.598959+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:20:42.258886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T08:06:49.599111+0000","last_change":"2024-05-11T08:06:48.610374+0000","last_active":"2024-05-11T08:06:49.599111+0000","last_peered":"2024-05-11T08:06:49.599111+0000","last_clean":"2024-05-11T08:06:49.599111+0000","last_became_active":"2024-05-11T08:06:48.609812+0000","last_became_peered":"2024-05-11T08:06:48.609812+0000","last_unstale":"2024-05-11T08:06:49.599111+0000","last_undegraded":"2024-05-11T08:06:49.599111+0000","last_fullsized":"2024-05-11T08:06:49.599111+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:35:08.257212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"40'2","reported_seq":16,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T08:06:49.776756+0000","last_change":"2024-05-11T08:06:47.595937+0000","last_active":"2024-05-11T08:06:49.776756+0000","last_peered":"2024-05-11T08:06:49.776756+0000","last_clean":"2024-05-11T08:06:49.776756+0000","last_became_active":"2024-05-11T08:06:47.595642+0000","last_became_peered":"2024-05-11T08:06:47.595642+0000","last_unstale":"2024-05-11T08:06:49.776756+0000","last_undegraded":"2024-05-11T08:06:49.776756+0000","last_fullsized":"2024-05-11T08:06:49.776756+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:58:03.492050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T08:06:49.776639+0000","last_change":"2024-05-11T08:06:48.615934+0000","last_active":"2024-05-11T08:06:49.776639+0000","last_peered":"2024-05-11T08:06:49.776639+0000","last_clean":"2024-05-11T08:06:49.776639+0000","last_became_active":"2024-05-11T08:06:48.615084+0000","last_became_peered":"2024-05-11T08:06:48.615084+0000","last_unstale":"2024-05-11T08:06:49.776639+0000","last_undegraded":"2024-05-11T08:06:49.776639+0000","last_fullsized":"2024-05-11T08:06:49.776639+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:58:19.327537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"40'1","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T08:06:49.613353+0000","last_change":"2024-05-11T08:06:47.599057+0000","last_active":"2024-05-11T08:06:49.613353+0000","last_peered":"2024-05-11T08:06:49.613353+0000","last_clean":"2024-05-11T08:06:49.613353+0000","last_became_active":"2024-05-11T08:06:47.598647+0000","last_became_peered":"2024-05-11T08:06:47.598647+0000","last_unstale":"2024-05-11T08:06:49.613353+0000","last_undegraded":"2024-05-11T08:06:49.613353+0000","last_fullsized":"2024-05-11T08:06:49.613353+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T11:40:41.542604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T08:06:49.613419+0000","last_change":"2024-05-11T08:06:48.606563+0000","last_active":"2024-05-11T08:06:49.613419+0000","last_peered":"2024-05-11T08:06:49.613419+0000","last_clean":"2024-05-11T08:06:49.613419+0000","last_became_active":"2024-05-11T08:06:48.606287+0000","last_became_peered":"2024-05-11T08:06:48.606287+0000","last_unstale":"2024-05-11T08:06:49.613419+0000","last_undegraded":"2024-05-11T08:06:49.613419+0000","last_fullsized":"2024-05-11T08:06:49.613419+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:23:43.027012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":14,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T08:06:49.776549+0000","last_change":"2024-05-11T08:06:47.592090+0000","last_active":"2024-05-11T08:06:49.776549+0000","last_peered":"2024-05-11T08:06:49.776549+0000","last_clean":"2024-05-11T08:06:49.776549+0000","last_became_active":"2024-05-11T08:06:47.591813+0000","last_became_peered":"2024-05-11T08:06:47.591813+0000","last_unstale":"2024-05-11T08:06:49.776549+0000","last_undegraded":"2024-05-11T08:06:49.776549+0000","last_fullsized":"2024-05-11T08:06:49.776549+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:12:27.297983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T08:06:49.597432+0000","last_change":"2024-05-11T08:06:48.613165+0000","last_active":"2024-05-11T08:06:49.597432+0000","last_peered":"2024-05-11T08:06:49.597432+0000","last_clean":"2024-05-11T08:06:49.597432+0000","last_became_active":"2024-05-11T08:06:48.612678+0000","last_became_peered":"2024-05-11T08:06:48.612678+0000","last_unstale":"2024-05-11T08:06:49.597432+0000","last_undegraded":"2024-05-11T08:06:49.597432+0000","last_fullsized":"2024-05-11T08:06:49.597432+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:58:03.309459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"40'2","reported_seq":16,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T08:06:49.613224+0000","last_change":"2024-05-11T08:06:47.600614+0000","last_active":"2024-05-11T08:06:49.613224+0000","last_peered":"2024-05-11T08:06:49.613224+0000","last_clean":"2024-05-11T08:06:49.613224+0000","last_became_active":"2024-05-11T08:06:47.600104+0000","last_became_peered":"2024-05-11T08:06:47.600104+0000","last_unstale":"2024-05-11T08:06:49.613224+0000","last_undegraded":"2024-05-11T08:06:49.613224+0000","last_fullsized":"2024-05-11T08:06:49.613224+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:25:19.853657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T08:06:49.776250+0000","last_change":"2024-05-11T08:06:48.612125+0000","last_active":"2024-05-11T08:06:49.776250+0000","last_peered":"2024-05-11T08:06:49.776250+0000","last_clean":"2024-05-11T08:06:49.776250+0000","last_became_active":"2024-05-11T08:06:48.611838+0000","last_became_peered":"2024-05-11T08:06:48.611838+0000","last_unstale":"2024-05-11T08:06:49.776250+0000","last_undegraded":"2024-05-11T08:06:49.776250+0000","last_fullsized":"2024-05-11T08:06:49.776250+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:48:10.064056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":14,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T08:06:49.613139+0000","last_change":"2024-05-11T08:06:47.597958+0000","last_active":"2024-05-11T08:06:49.613139+0000","last_peered":"2024-05-11T08:06:49.613139+0000","last_clean":"2024-05-11T08:06:49.613139+0000","last_became_active":"2024-05-11T08:06:47.597638+0000","last_became_peered":"2024-05-11T08:06:47.597638+0000","last_unstale":"2024-05-11T08:06:49.613139+0000","last_undegraded":"2024-05-11T08:06:49.613139+0000","last_fullsized":"2024-05-11T08:06:49.613139+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:02:41.089263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T08:06:49.598806+0000","last_change":"2024-05-11T08:06:48.611306+0000","last_active":"2024-05-11T08:06:49.598806+0000","last_peered":"2024-05-11T08:06:49.598806+0000","last_clean":"2024-05-11T08:06:49.598806+0000","last_became_active":"2024-05-11T08:06:48.611032+0000","last_became_peered":"2024-05-11T08:06:48.611032+0000","last_unstale":"2024-05-11T08:06:49.598806+0000","last_undegraded":"2024-05-11T08:06:49.598806+0000","last_fullsized":"2024-05-11T08:06:49.598806+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:58:03.309459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":14,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T08:06:49.597414+0000","last_change":"2024-05-11T08:06:47.595398+0000","last_active":"2024-05-11T08:06:49.597414+0000","last_peered":"2024-05-11T08:06:49.597414+0000","last_clean":"2024-05-11T08:06:49.597414+0000","last_became_active":"2024-05-11T08:06:47.595129+0000","last_became_peered":"2024-05-11T08:06:47.595129+0000","last_unstale":"2024-05-11T08:06:49.597414+0000","last_undegraded":"2024-05-11T08:06:49.597414+0000","last_fullsized":"2024-05-11T08:06:49.597414+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:49:09.098705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T08:06:49.777468+0000","last_change":"2024-05-11T08:06:48.612773+0000","last_active":"2024-05-11T08:06:49.777468+0000","last_peered":"2024-05-11T08:06:49.777468+0000","last_clean":"2024-05-11T08:06:49.777468+0000","last_became_active":"2024-05-11T08:06:48.612291+0000","last_became_peered":"2024-05-11T08:06:48.612291+0000","last_unstale":"2024-05-11T08:06:49.777468+0000","last_undegraded":"2024-05-11T08:06:49.777468+0000","last_fullsized":"2024-05-11T08:06:49.777468+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:35:08.257212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":14,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T08:06:49.613084+0000","last_change":"2024-05-11T08:06:47.601435+0000","last_active":"2024-05-11T08:06:49.613084+0000","last_peered":"2024-05-11T08:06:49.613084+0000","last_clean":"2024-05-11T08:06:49.613084+0000","last_became_active":"2024-05-11T08:06:47.601171+0000","last_became_peered":"2024-05-11T08:06:47.601171+0000","last_unstale":"2024-05-11T08:06:49.613084+0000","last_undegraded":"2024-05-11T08:06:49.613084+0000","last_fullsized":"2024-05-11T08:06:49.613084+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:39:48.026570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T08:06:49.776476+0000","last_change":"2024-05-11T08:06:48.611355+0000","last_active":"2024-05-11T08:06:49.776476+0000","last_peered":"2024-05-11T08:06:49.776476+0000","last_clean":"2024-05-11T08:06:49.776476+0000","last_became_active":"2024-05-11T08:06:48.611027+0000","last_became_peered":"2024-05-11T08:06:48.611027+0000","last_unstale":"2024-05-11T08:06:49.776476+0000","last_undegraded":"2024-05-11T08:06:49.776476+0000","last_fullsized":"2024-05-11T08:06:49.776476+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:30:16.719412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"40'2","reported_seq":16,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T08:06:49.599174+0000","last_change":"2024-05-11T08:06:47.588756+0000","last_active":"2024-05-11T08:06:49.599174+0000","last_peered":"2024-05-11T08:06:49.599174+0000","last_clean":"2024-05-11T08:06:49.599174+0000","last_became_active":"2024-05-11T08:06:47.588460+0000","last_became_peered":"2024-05-11T08:06:47.588460+0000","last_unstale":"2024-05-11T08:06:49.599174+0000","last_undegraded":"2024-05-11T08:06:49.599174+0000","last_fullsized":"2024-05-11T08:06:49.599174+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:46:05.539312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T08:06:49.597967+0000","last_change":"2024-05-11T08:06:48.618611+0000","last_active":"2024-05-11T08:06:49.597967+0000","last_peered":"2024-05-11T08:06:49.597967+0000","last_clean":"2024-05-11T08:06:49.597967+0000","last_became_active":"2024-05-11T08:06:48.618358+0000","last_became_peered":"2024-05-11T08:06:48.618358+0000","last_unstale":"2024-05-11T08:06:49.597967+0000","last_undegraded":"2024-05-11T08:06:49.597967+0000","last_fullsized":"2024-05-11T08:06:49.597967+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:59:44.981066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"41'1","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T08:06:49.614712+0000","last_change":"2024-05-11T08:06:47.598474+0000","last_active":"2024-05-11T08:06:49.614712+0000","last_peered":"2024-05-11T08:06:49.614712+0000","last_clean":"2024-05-11T08:06:49.614712+0000","last_became_active":"2024-05-11T08:06:47.598238+0000","last_became_peered":"2024-05-11T08:06:47.598238+0000","last_unstale":"2024-05-11T08:06:49.614712+0000","last_undegraded":"2024-05-11T08:06:49.614712+0000","last_fullsized":"2024-05-11T08:06:49.614712+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T10:06:17.262543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T08:06:49.777307+0000","last_change":"2024-05-11T08:06:48.619214+0000","last_active":"2024-05-11T08:06:49.777307+0000","last_peered":"2024-05-11T08:06:49.777307+0000","last_clean":"2024-05-11T08:06:49.777307+0000","last_became_active":"2024-05-11T08:06:48.618951+0000","last_became_peered":"2024-05-11T08:06:48.618951+0000","last_unstale":"2024-05-11T08:06:49.777307+0000","last_undegraded":"2024-05-11T08:06:49.777307+0000","last_fullsized":"2024-05-11T08:06:49.777307+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:45:33.813046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":14,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T08:06:49.776484+0000","last_change":"2024-05-11T08:06:47.599942+0000","last_active":"2024-05-11T08:06:49.776484+0000","last_peered":"2024-05-11T08:06:49.776484+0000","last_clean":"2024-05-11T08:06:49.776484+0000","last_became_active":"2024-05-11T08:06:47.599513+0000","last_became_peered":"2024-05-11T08:06:47.599513+0000","last_unstale":"2024-05-11T08:06:49.776484+0000","last_undegraded":"2024-05-11T08:06:49.776484+0000","last_fullsized":"2024-05-11T08:06:49.776484+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:12:27.297983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T08:06:49.776382+0000","last_change":"2024-05-11T08:06:48.602295+0000","last_active":"2024-05-11T08:06:49.776382+0000","last_peered":"2024-05-11T08:06:49.776382+0000","last_clean":"2024-05-11T08:06:49.776382+0000","last_became_active":"2024-05-11T08:06:48.601860+0000","last_became_peered":"2024-05-11T08:06:48.601860+0000","last_unstale":"2024-05-11T08:06:49.776382+0000","last_undegraded":"2024-05-11T08:06:49.776382+0000","last_fullsized":"2024-05-11T08:06:49.776382+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:58:24.559024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":14,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T08:06:49.776832+0000","last_change":"2024-05-11T08:06:47.600002+0000","last_active":"2024-05-11T08:06:49.776832+0000","last_peered":"2024-05-11T08:06:49.776832+0000","last_clean":"2024-05-11T08:06:49.776832+0000","last_became_active":"2024-05-11T08:06:47.599594+0000","last_became_peered":"2024-05-11T08:06:47.599594+0000","last_unstale":"2024-05-11T08:06:49.776832+0000","last_undegraded":"2024-05-11T08:06:49.776832+0000","last_fullsized":"2024-05-11T08:06:49.776832+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:13:57.487739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T08:06:49.776256+0000","last_change":"2024-05-11T08:06:48.605250+0000","last_active":"2024-05-11T08:06:49.776256+0000","last_peered":"2024-05-11T08:06:49.776256+0000","last_clean":"2024-05-11T08:06:49.776256+0000","last_became_active":"2024-05-11T08:06:48.604558+0000","last_became_peered":"2024-05-11T08:06:48.604558+0000","last_unstale":"2024-05-11T08:06:49.776256+0000","last_undegraded":"2024-05-11T08:06:49.776256+0000","last_fullsized":"2024-05-11T08:06:49.776256+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:34:42.615373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":14,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T08:06:49.596569+0000","last_change":"2024-05-11T08:06:47.600101+0000","last_active":"2024-05-11T08:06:49.596569+0000","last_peered":"2024-05-11T08:06:49.596569+0000","last_clean":"2024-05-11T08:06:49.596569+0000","last_became_active":"2024-05-11T08:06:47.599765+0000","last_became_peered":"2024-05-11T08:06:47.599765+0000","last_unstale":"2024-05-11T08:06:49.596569+0000","last_undegraded":"2024-05-11T08:06:49.596569+0000","last_fullsized":"2024-05-11T08:06:49.596569+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:56:21.008947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T08:06:49.776354+0000","last_change":"2024-05-11T08:06:48.615746+0000","last_active":"2024-05-11T08:06:49.776354+0000","last_peered":"2024-05-11T08:06:49.776354+0000","last_clean":"2024-05-11T08:06:49.776354+0000","last_became_active":"2024-05-11T08:06:48.615466+0000","last_became_peered":"2024-05-11T08:06:48.615466+0000","last_unstale":"2024-05-11T08:06:49.776354+0000","last_undegraded":"2024-05-11T08:06:49.776354+0000","last_fullsized":"2024-05-11T08:06:49.776354+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:47:47.699231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":14,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T08:06:49.776246+0000","last_change":"2024-05-11T08:06:47.592149+0000","last_active":"2024-05-11T08:06:49.776246+0000","last_peered":"2024-05-11T08:06:49.776246+0000","last_clean":"2024-05-11T08:06:49.776246+0000","last_became_active":"2024-05-11T08:06:47.591790+0000","last_became_peered":"2024-05-11T08:06:47.591790+0000","last_unstale":"2024-05-11T08:06:49.776246+0000","last_undegraded":"2024-05-11T08:06:49.776246+0000","last_fullsized":"2024-05-11T08:06:49.776246+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T09:13:57.487739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T08:06:49.597941+0000","last_change":"2024-05-11T08:06:48.620144+0000","last_active":"2024-05-11T08:06:49.597941+0000","last_peered":"2024-05-11T08:06:49.597941+0000","last_clean":"2024-05-11T08:06:49.597941+0000","last_became_active":"2024-05-11T08:06:48.619862+0000","last_became_peered":"2024-05-11T08:06:48.619862+0000","last_unstale":"2024-05-11T08:06:49.597941+0000","last_undegraded":"2024-05-11T08:06:49.597941+0000","last_fullsized":"2024-05-11T08:06:49.597941+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:45:33.813046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":14,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T08:06:49.777369+0000","last_change":"2024-05-11T08:06:47.589050+0000","last_active":"2024-05-11T08:06:49.777369+0000","last_peered":"2024-05-11T08:06:49.777369+0000","last_clean":"2024-05-11T08:06:49.777369+0000","last_became_active":"2024-05-11T08:06:47.588780+0000","last_became_peered":"2024-05-11T08:06:47.588780+0000","last_unstale":"2024-05-11T08:06:49.777369+0000","last_undegraded":"2024-05-11T08:06:49.777369+0000","last_fullsized":"2024-05-11T08:06:49.777369+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:22:42.867140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T08:06:49.599080+0000","last_change":"2024-05-11T08:06:48.602780+0000","last_active":"2024-05-11T08:06:49.599080+0000","last_peered":"2024-05-11T08:06:49.599080+0000","last_clean":"2024-05-11T08:06:49.599080+0000","last_became_active":"2024-05-11T08:06:48.602508+0000","last_became_peered":"2024-05-11T08:06:48.602508+0000","last_unstale":"2024-05-11T08:06:49.599080+0000","last_undegraded":"2024-05-11T08:06:49.599080+0000","last_fullsized":"2024-05-11T08:06:49.599080+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:47.573468+0000","last_clean_scrub_stamp":"2024-05-11T08:06:47.573468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:04:46.093003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":14,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T08:06:49.597788+0000","last_change":"2024-05-11T08:06:47.597382+0000","last_active":"2024-05-11T08:06:49.597788+0000","last_peered":"2024-05-11T08:06:49.597788+0000","last_clean":"2024-05-11T08:06:49.597788+0000","last_became_active":"2024-05-11T08:06:47.597062+0000","last_became_peered":"2024-05-11T08:06:47.597062+0000","last_unstale":"2024-05-11T08:06:49.597788+0000","last_undegraded":"2024-05-11T08:06:49.597788+0000","last_fullsized":"2024-05-11T08:06:49.597788+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:06:46.564761+0000","last_clean_scrub_stamp":"2024-05-11T08:06:46.564761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:46:05.539312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":146,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-11T08:06:49.597592+0000","last_change":"2024-05-11T08:05:45.772368+0000","last_active":"2024-05-11T08:06:49.597592+0000","last_peered":"2024-05-11T08:06:49.597592+0000","last_clean":"2024-05-11T08:06:49.597592+0000","last_became_active":"2024-05-11T08:05:45.771373+0000","last_became_peered":"2024-05-11T08:05:45.771373+0000","last_unstale":"2024-05-11T08:06:49.597592+0000","last_undegraded":"2024-05-11T08:06:49.597592+0000","last_fullsized":"2024-05-11T08:06:49.597592+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:44.616617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:44.616617+0000","last_clean_scrub_stamp":"2024-05-11T08:05:44.616617+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:02:08.078748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509458,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28080,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343760,"statfs":{"total":96636764160,"available":96608010240,"internally_reserved":0,"allocated":1077248,"data_stored":792379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509458,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27488,"kb_used_data":460,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344352,"statfs":{"total":96636764160,"available":96608616448,"internally_reserved":0,"allocated":471040,"data_stored":200911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509458,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27492,"kb_used_data":464,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344348,"statfs":{"total":96636764160,"available":96608612352,"internally_reserved":0,"allocated":475136,"data_stored":200441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509458,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27492,"kb_used_data":464,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344348,"statfs":{"total":96636764160,"available":96608612352,"internally_reserved":0,"allocated":475136,"data_stored":200933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509459,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27500,"kb_used_data":472,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344340,"statfs":{"total":96636764160,"available":96608604160,"internally_reserved":0,"allocated":483328,"data_stored":201975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509459,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27488,"kb_used_data":460,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344352,"statfs":{"total":96636764160,"available":96608616448,"internally_reserved":0,"allocated":471040,"data_stored":200419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509458,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28068,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343772,"statfs":{"total":96636764160,"available":96608022528,"internally_reserved":0,"allocated":1064960,"data_stored":790731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574866,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27480,"kb_used_data":460,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344360,"statfs":{"total":96636764160,"available":96608624640,"internally_reserved":0,"allocated":471040,"data_stored":200399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-11T08:06:56.372 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-11T08:06:56.372 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-05-11T08:06:56.522 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.521+0000 7f6658ae1640 1 Processor -- start 2024-05-11T08:06:56.522 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.522+0000 7f6658ae1640 1 -- start start 2024-05-11T08:06:56.523 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.523+0000 7f6658ae1640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6654145a00 0x7f6654073320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:56.523 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.523+0000 7f6658ae1640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f66541608c0 0x7f665406e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:56.523 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.523+0000 7f6658ae1640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6654145de0 con 0x7f6654073860 2024-05-11T08:06:56.524 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.523+0000 7f6658ae1640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f66540b41b0 con 0x7f66541608c0 2024-05-11T08:06:56.524 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.523+0000 7f6658ae1640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f6654073d70 con 0x7f6654145a00 2024-05-11T08:06:56.524 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.523+0000 7f665259c640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6654145a00 0x7f6654073320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:56.524 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.523+0000 7f6651d9b640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f66541608c0 0x7f665406e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:56.524 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.523+0000 7f6652d9d640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f6654073860 0x7f66540ceed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.167:37746/0 (socket says 172.21.15.167:37746) 2024-05-11T08:06:56.524 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.523+0000 7f6652d9d640 1 -- 172.21.15.167:0/1671066585 learned_addr learned my addr 172.21.15.167:0/1671066585 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T08:06:56.524 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.523+0000 7f665259c640 1 -- 172.21.15.167:0/1671066585 >> v2:172.21.15.204:3300/0 conn(0x7f66541608c0 msgr2=0x7f665406e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:56.524 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.523+0000 7f665259c640 1 --2- 172.21.15.167:0/1671066585 >> v2:172.21.15.204:3300/0 conn(0x7f66541608c0 0x7f665406e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:56.524 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.523+0000 7f665259c640 1 -- 172.21.15.167:0/1671066585 >> v1:172.21.15.204:6789/0 conn(0x7f6654073860 legacy=0x7f66540ceed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:56.524 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.523+0000 7f665259c640 1 -- 172.21.15.167:0/1671066585 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6654074050 con 0x7f6654145a00 2024-05-11T08:06:56.524 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.523+0000 7f6651d9b640 1 --2- 172.21.15.167:0/1671066585 >> v2:172.21.15.204:3300/0 conn(0x7f66541608c0 0x7f665406e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T08:06:56.525 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.524+0000 7f665259c640 1 --2- 172.21.15.167:0/1671066585 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6654145a00 0x7f6654073320 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f664c00ba40 tx=0x7f664c02c850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=92907cfccdbbfa04 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:56.525 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.524+0000 7f665159a640 1 -- 172.21.15.167:0/1671066585 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f664c032750 con 0x7f6654145a00 2024-05-11T08:06:56.526 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.524+0000 7f665159a640 1 -- 172.21.15.167:0/1671066585 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f664c030050 con 0x7f6654145a00 2024-05-11T08:06:56.526 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.524+0000 7f665159a640 1 -- 172.21.15.167:0/1671066585 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f664c032b10 con 0x7f6654145a00 2024-05-11T08:06:56.526 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.525+0000 7f6658ae1640 1 -- 172.21.15.167:0/1671066585 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6654145a00 msgr2=0x7f6654073320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:56.526 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.525+0000 7f6658ae1640 1 --2- 172.21.15.167:0/1671066585 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6654145a00 0x7f6654073320 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f664c00ba40 tx=0x7f664c02c850 comp rx=0 tx=0).stop 2024-05-11T08:06:56.526 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.525+0000 7f6658ae1640 1 -- 172.21.15.167:0/1671066585 shutdown_connections 2024-05-11T08:06:56.526 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.525+0000 7f6658ae1640 1 --2- 172.21.15.167:0/1671066585 >> v2:172.21.15.204:3300/0 conn(0x7f66541608c0 0x7f665406e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:56.526 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.525+0000 7f6658ae1640 1 --2- 172.21.15.167:0/1671066585 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6654145a00 0x7f6654073320 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:56.526 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.525+0000 7f6658ae1640 1 -- 172.21.15.167:0/1671066585 >> 172.21.15.167:0/1671066585 conn(0x7f66540b2f60 msgr2=0x7f66540b3340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:56.526 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.525+0000 7f6658ae1640 1 -- 172.21.15.167:0/1671066585 shutdown_connections 2024-05-11T08:06:56.526 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.525+0000 7f6658ae1640 1 -- 172.21.15.167:0/1671066585 wait complete. 2024-05-11T08:06:56.528 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.526+0000 7f6658ae1640 1 Processor -- start 2024-05-11T08:06:56.528 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.526+0000 7f6658ae1640 1 -- start start 2024-05-11T08:06:56.529 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.527+0000 7f6658ae1640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6654073860 0x7f66541547d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:56.529 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.527+0000 7f6658ae1640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6654145a00 0x7f66541f53b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:56.529 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.527+0000 7f6658ae1640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f66541608c0 0x7f66541f58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:56.529 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.527+0000 7f6658ae1640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f66540b5650 con 0x7f6654145a00 2024-05-11T08:06:56.529 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.527+0000 7f6658ae1640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6654074610 con 0x7f6654073860 2024-05-11T08:06:56.529 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.527+0000 7f6658ae1640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f66540b57d0 con 0x7f66541608c0 2024-05-11T08:06:56.529 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.527+0000 7f6651d9b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6654145a00 0x7f66541f53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:56.529 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.527+0000 7f6651d9b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6654145a00 0x7f66541f53b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:42980/0 (socket says 172.21.15.167:42980) 2024-05-11T08:06:56.529 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.527+0000 7f6651d9b640 1 -- 172.21.15.167:0/1792390682 learned_addr learned my addr 172.21.15.167:0/1792390682 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:56.529 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.527+0000 7f665259c640 1 --2- 172.21.15.167:0/1792390682 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6654073860 0x7f66541547d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:56.529 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.527+0000 7f6652d9d640 1 --2- 172.21.15.167:0/1792390682 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f66541608c0 0x7f66541f58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:56.529 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.528+0000 7f6651d9b640 1 -- 172.21.15.167:0/1792390682 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f66541608c0 msgr2=0x7f66541f58f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:56.529 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.528+0000 7f6651d9b640 1 --2- 172.21.15.167:0/1792390682 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f66541608c0 0x7f66541f58f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:56.529 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.528+0000 7f6651d9b640 1 -- 172.21.15.167:0/1792390682 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6654073860 msgr2=0x7f66541547d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:56.529 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.528+0000 7f6651d9b640 1 --2- 172.21.15.167:0/1792390682 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6654073860 0x7f66541547d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:56.530 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.528+0000 7f6651d9b640 1 -- 172.21.15.167:0/1792390682 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66540ca400 con 0x7f6654145a00 2024-05-11T08:06:56.530 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.528+0000 7f665259c640 1 --2- 172.21.15.167:0/1792390682 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6654073860 0x7f66541547d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T08:06:56.530 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.528+0000 7f6651d9b640 1 --2- 172.21.15.167:0/1792390682 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6654145a00 0x7f66541f53b0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f664800efc0 tx=0x7f664800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:56.530 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.528+0000 7f6642ffd640 1 -- 172.21.15.167:0/1792390682 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6648018020 con 0x7f6654145a00 2024-05-11T08:06:56.530 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.528+0000 7f6642ffd640 1 -- 172.21.15.167:0/1792390682 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6648014680 con 0x7f6654145a00 2024-05-11T08:06:56.530 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.528+0000 7f6642ffd640 1 -- 172.21.15.167:0/1792390682 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6648014a50 con 0x7f6654145a00 2024-05-11T08:06:56.530 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.528+0000 7f6658ae1640 1 -- 172.21.15.167:0/1792390682 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66541f5f20 con 0x7f6654145a00 2024-05-11T08:06:56.530 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.528+0000 7f6658ae1640 1 -- 172.21.15.167:0/1792390682 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66541f64e0 con 0x7f6654145a00 2024-05-11T08:06:56.531 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.529+0000 7f6658ae1640 1 -- 172.21.15.167:0/1792390682 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6654069630 con 0x7f6654145a00 2024-05-11T08:06:56.531 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.530+0000 7f6642ffd640 1 -- 172.21.15.167:0/1792390682 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f6648005d10 con 0x7f6654145a00 2024-05-11T08:06:56.531 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.530+0000 7f6642ffd640 1 --2- 172.21.15.167:0/1792390682 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f66340762d0 0x7f6634078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:56.531 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.530+0000 7f6642ffd640 1 -- 172.21.15.167:0/1792390682 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6101+0+0 (secure 0 0 0) 0x7f6648098330 con 0x7f6654145a00 2024-05-11T08:06:56.531 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.530+0000 7f665259c640 1 --2- 172.21.15.167:0/1792390682 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f66340762d0 0x7f6634078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:56.531 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.531+0000 7f665259c640 1 --2- 172.21.15.167:0/1792390682 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f66340762d0 0x7f6634078790 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f664c037000 tx=0x7f664c036040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:56.534 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.535+0000 7f6642ffd640 1 -- 172.21.15.167:0/1792390682 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f6648022030 con 0x7f6654145a00 2024-05-11T08:06:56.795 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.795+0000 7f6658ae1640 1 -- 172.21.15.167:0/1792390682 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f665406db00 con 0x7f6654145a00 2024-05-11T08:06:56.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.800+0000 7f6642ffd640 1 -- 172.21.15.167:0/1792390682 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v9) ==== 190+0+147 (secure 0 0 0) 0x7f6648010040 con 0x7f6654145a00 2024-05-11T08:06:56.800 INFO:teuthology.orchestra.run.smithi167.stdout:[client.0] 2024-05-11T08:06:56.800 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQCqJj9m/0ccFBAAu878v2GdClfu/qvQ/KOnvQ== 2024-05-11T08:06:56.801 INFO:teuthology.orchestra.run.smithi167.stdout: caps mds = "allow" 2024-05-11T08:06:56.801 INFO:teuthology.orchestra.run.smithi167.stdout: caps mon = "allow r" 2024-05-11T08:06:56.801 INFO:teuthology.orchestra.run.smithi167.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-05-11T08:06:56.801 INFO:teuthology.orchestra.run.smithi167.stderr:updated caps for client.0 2024-05-11T08:06:56.803 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.803+0000 7f6658ae1640 1 -- 172.21.15.167:0/1792390682 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f66340762d0 msgr2=0x7f6634078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:56.803 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.803+0000 7f6658ae1640 1 --2- 172.21.15.167:0/1792390682 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f66340762d0 0x7f6634078790 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f664c037000 tx=0x7f664c036040 comp rx=0 tx=0).stop 2024-05-11T08:06:56.803 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.803+0000 7f6658ae1640 1 -- 172.21.15.167:0/1792390682 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6654145a00 msgr2=0x7f66541f53b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:56.803 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.803+0000 7f6658ae1640 1 --2- 172.21.15.167:0/1792390682 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6654145a00 0x7f66541f53b0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f664800efc0 tx=0x7f664800c490 comp rx=0 tx=0).stop 2024-05-11T08:06:56.803 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.803+0000 7f6658ae1640 1 -- 172.21.15.167:0/1792390682 shutdown_connections 2024-05-11T08:06:56.803 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.803+0000 7f6658ae1640 1 --2- 172.21.15.167:0/1792390682 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f66340762d0 0x7f6634078790 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:56.803 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.803+0000 7f6658ae1640 1 --2- 172.21.15.167:0/1792390682 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f66541608c0 0x7f66541f58f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:56.803 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.803+0000 7f6658ae1640 1 --2- 172.21.15.167:0/1792390682 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6654145a00 0x7f66541f53b0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:56.803 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.803+0000 7f6658ae1640 1 --2- 172.21.15.167:0/1792390682 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6654073860 0x7f66541547d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:56.803 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.803+0000 7f6658ae1640 1 -- 172.21.15.167:0/1792390682 >> 172.21.15.167:0/1792390682 conn(0x7f66540b2f60 msgr2=0x7f665415fb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:56.803 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.803+0000 7f6658ae1640 1 -- 172.21.15.167:0/1792390682 shutdown_connections 2024-05-11T08:06:56.804 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.803+0000 7f6658ae1640 1 -- 172.21.15.167:0/1792390682 wait complete. 2024-05-11T08:06:56.814 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-11T08:06:56.915 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.915+0000 7fcaf3524640 1 Processor -- start 2024-05-11T08:06:56.915 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.915+0000 7fcaf3524640 1 -- start start 2024-05-11T08:06:56.917 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.916+0000 7fcaf3524640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fcaec0d2cf0 0x7fcaec0cafb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:56.917 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.916+0000 7fcaf3524640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fcaec0cb4f0 0x7fcaec06dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:56.917 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.916+0000 7fcaf3524640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcaec0d1190 con 0x7fcaec1598d0 2024-05-11T08:06:56.917 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.916+0000 7fcaf3524640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7fcaec0cb920 con 0x7fcaec0cb4f0 2024-05-11T08:06:56.917 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.916+0000 7fcaf3524640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fcaec06d500 con 0x7fcaec0d2cf0 2024-05-11T08:06:56.917 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.916+0000 7fcaf1ac1640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fcaec0cb4f0 0x7fcaec06dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:56.917 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.916+0000 7fcaf1ac1640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fcaec0cb4f0 0x7fcaec06dfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:42984/0 (socket says 172.21.15.167:42984) 2024-05-11T08:06:56.917 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.916+0000 7fcaf1ac1640 1 -- 172.21.15.167:0/2881391036 learned_addr learned my addr 172.21.15.167:0/2881391036 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:56.917 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.916+0000 7fcaf0abf640 1 --2- 172.21.15.167:0/2881391036 >> v2:172.21.15.204:3300/0 conn(0x7fcaec0d2cf0 0x7fcaec0cafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:56.917 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.917+0000 7fcaf1ac1640 1 -- 172.21.15.167:0/2881391036 >> v2:172.21.15.204:3300/0 conn(0x7fcaec0d2cf0 msgr2=0x7fcaec0cafb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:56.917 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.917+0000 7fcaf1ac1640 1 --2- 172.21.15.167:0/2881391036 >> v2:172.21.15.204:3300/0 conn(0x7fcaec0d2cf0 0x7fcaec0cafb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:56.917 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.917+0000 7fcaf1ac1640 1 -- 172.21.15.167:0/2881391036 >> v1:172.21.15.204:6789/0 conn(0x7fcaec1598d0 legacy=0x7fcaec0d0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:56.917 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.917+0000 7fcaf1ac1640 1 -- 172.21.15.167:0/2881391036 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcaec06d7e0 con 0x7fcaec0cb4f0 2024-05-11T08:06:56.919 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.917+0000 7fcaf1ac1640 1 --2- 172.21.15.167:0/2881391036 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcaec0cb4f0 0x7fcaec06dfe0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fcaec0b4c50 tx=0x7fcae402a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98a00a9e03b7a94 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:56.919 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.918+0000 7fcadffff640 1 -- 172.21.15.167:0/2881391036 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcae4036070 con 0x7fcaec0cb4f0 2024-05-11T08:06:56.919 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.918+0000 7fcadffff640 1 -- 172.21.15.167:0/2881391036 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcae4031600 con 0x7fcaec0cb4f0 2024-05-11T08:06:56.919 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.918+0000 7fcadffff640 1 -- 172.21.15.167:0/2881391036 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcae40319d0 con 0x7fcaec0cb4f0 2024-05-11T08:06:56.919 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.918+0000 7fcaf3524640 1 -- 172.21.15.167:0/2881391036 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcaec0cb4f0 msgr2=0x7fcaec06dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:56.919 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.918+0000 7fcaf3524640 1 --2- 172.21.15.167:0/2881391036 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcaec0cb4f0 0x7fcaec06dfe0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fcaec0b4c50 tx=0x7fcae402a730 comp rx=0 tx=0).stop 2024-05-11T08:06:56.919 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.918+0000 7fcaf3524640 1 -- 172.21.15.167:0/2881391036 shutdown_connections 2024-05-11T08:06:56.919 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.918+0000 7fcaf3524640 1 --2- 172.21.15.167:0/2881391036 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcaec0cb4f0 0x7fcaec06dfe0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:56.919 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.918+0000 7fcaf3524640 1 --2- 172.21.15.167:0/2881391036 >> v2:172.21.15.204:3300/0 conn(0x7fcaec0d2cf0 0x7fcaec0cafb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:56.919 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.918+0000 7fcaf3524640 1 -- 172.21.15.167:0/2881391036 >> 172.21.15.167:0/2881391036 conn(0x7fcaec0d1ba0 msgr2=0x7fcaec0d1fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:56.919 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.919+0000 7fcaf3524640 1 -- 172.21.15.167:0/2881391036 shutdown_connections 2024-05-11T08:06:56.919 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.919+0000 7fcaf3524640 1 -- 172.21.15.167:0/2881391036 wait complete. 2024-05-11T08:06:56.920 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.920+0000 7fcaf3524640 1 Processor -- start 2024-05-11T08:06:56.920 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.920+0000 7fcaf3524640 1 -- start start 2024-05-11T08:06:56.923 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.921+0000 7fcaf3524640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcaec0cb4f0 0x7fcaec156c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:56.923 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.921+0000 7fcaf3524640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcaec0d2cf0 0x7fcaec0ce7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:56.923 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.921+0000 7fcaf3524640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcaec1598d0 0x7fcaec14cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:56.923 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.921+0000 7fcaf3524640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fcaec159cb0 con 0x7fcaec0cb4f0 2024-05-11T08:06:56.923 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.922+0000 7fcaf3524640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fcaec1607c0 con 0x7fcaec0d2cf0 2024-05-11T08:06:56.923 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.922+0000 7fcaf3524640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fcaec06d1b0 con 0x7fcaec1598d0 2024-05-11T08:06:56.923 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.922+0000 7fcaf12c0640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcaec0cb4f0 0x7fcaec156c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:56.923 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.922+0000 7fcaf12c0640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcaec0cb4f0 0x7fcaec156c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:42998/0 (socket says 172.21.15.167:42998) 2024-05-11T08:06:56.923 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.922+0000 7fcaf12c0640 1 -- 172.21.15.167:0/3921811646 learned_addr learned my addr 172.21.15.167:0/3921811646 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:56.923 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.922+0000 7fcaf0abf640 1 --2- 172.21.15.167:0/3921811646 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcaec0d2cf0 0x7fcaec0ce7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:56.923 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.922+0000 7fcaf1ac1640 1 --2- 172.21.15.167:0/3921811646 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcaec1598d0 0x7fcaec14cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:56.924 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.922+0000 7fcaf12c0640 1 -- 172.21.15.167:0/3921811646 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcaec1598d0 msgr2=0x7fcaec14cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:56.924 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.922+0000 7fcaf12c0640 1 --2- 172.21.15.167:0/3921811646 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcaec1598d0 0x7fcaec14cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:56.924 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.922+0000 7fcaf12c0640 1 -- 172.21.15.167:0/3921811646 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcaec0d2cf0 msgr2=0x7fcaec0ce7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:56.924 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.922+0000 7fcaf12c0640 1 --2- 172.21.15.167:0/3921811646 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcaec0d2cf0 0x7fcaec0ce7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:56.924 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.922+0000 7fcaf12c0640 1 -- 172.21.15.167:0/3921811646 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcaec14d100 con 0x7fcaec0cb4f0 2024-05-11T08:06:56.924 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.922+0000 7fcaf12c0640 1 --2- 172.21.15.167:0/3921811646 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcaec0cb4f0 0x7fcaec156c50 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fcae000c450 tx=0x7fcae000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:56.924 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.922+0000 7fcaddffb640 1 -- 172.21.15.167:0/3921811646 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcae00052d0 con 0x7fcaec0cb4f0 2024-05-11T08:06:56.924 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.922+0000 7fcaddffb640 1 -- 172.21.15.167:0/3921811646 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcae0005470 con 0x7fcaec0cb4f0 2024-05-11T08:06:56.924 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.922+0000 7fcaf3524640 1 -- 172.21.15.167:0/3921811646 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcaec0cd740 con 0x7fcaec0cb4f0 2024-05-11T08:06:56.924 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.922+0000 7fcaddffb640 1 -- 172.21.15.167:0/3921811646 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcae0012870 con 0x7fcaec0cb4f0 2024-05-11T08:06:56.924 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.923+0000 7fcaf3524640 1 -- 172.21.15.167:0/3921811646 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcaec0cdfe0 con 0x7fcaec0cb4f0 2024-05-11T08:06:56.925 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.924+0000 7fcaf3524640 1 -- 172.21.15.167:0/3921811646 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcaec069590 con 0x7fcaec0cb4f0 2024-05-11T08:06:56.925 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.924+0000 7fcaddffb640 1 -- 172.21.15.167:0/3921811646 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fcae0012a10 con 0x7fcaec0cb4f0 2024-05-11T08:06:56.925 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.925+0000 7fcaddffb640 1 --2- 172.21.15.167:0/3921811646 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fcad00762d0 0x7fcad0078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:56.925 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.925+0000 7fcaf0abf640 1 --2- 172.21.15.167:0/3921811646 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fcad00762d0 0x7fcad0078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:56.925 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.925+0000 7fcaddffb640 1 -- 172.21.15.167:0/3921811646 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6101+0+0 (secure 0 0 0) 0x7fcae0097240 con 0x7fcaec0cb4f0 2024-05-11T08:06:56.925 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.925+0000 7fcaf0abf640 1 --2- 172.21.15.167:0/3921811646 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fcad00762d0 0x7fcad0078790 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fcaec158a20 tx=0x7fcae8009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:56.933 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:56.933+0000 7fcaddffb640 1 -- 172.21.15.167:0/3921811646 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fcaec14d100 con 0x7fcaec0cb4f0 2024-05-11T08:06:57.152 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.152+0000 7fcaf3524640 1 -- 172.21.15.167:0/3921811646 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcaec0cbfb0 con 0x7fcaec0cb4f0 2024-05-11T08:06:57.152 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.153+0000 7fcaddffb640 1 -- 172.21.15.167:0/3921811646 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5190 (secure 0 0 0) 0x7fcae00603c0 con 0x7fcaec0cb4f0 2024-05-11T08:06:57.153 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T08:06:57.153 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4604,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2674983673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":2674983673},{"type":"v1","addr":"172.21.15.167:6837","nonce":2674983673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4607,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6834/1018548066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1018548066},{"type":"v1","addr":"172.21.15.204:6834","nonce":1018548066}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4729,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3369722249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3369722249},{"type":"v1","addr":"172.21.15.167:6836","nonce":3369722249}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-11T08:06:47.943939+0000","modified":"2024-05-11T08:06:49.955430+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4750},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4750":{"gid":4750,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.204:6835/469864006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6833","nonce":469864006},{"type":"v1","addr":"172.21.15.204:6835","nonce":469864006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4750,"qdb_cluster":[4750]},"id":2}]} 2024-05-11T08:06:57.153 INFO:teuthology.orchestra.run.smithi167.stderr:dumped fsmap epoch 16 2024-05-11T08:06:57.155 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.155+0000 7fcaf3524640 1 -- 172.21.15.167:0/3921811646 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fcad00762d0 msgr2=0x7fcad0078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:57.155 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.155+0000 7fcaf3524640 1 --2- 172.21.15.167:0/3921811646 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fcad00762d0 0x7fcad0078790 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fcaec158a20 tx=0x7fcae8009190 comp rx=0 tx=0).stop 2024-05-11T08:06:57.155 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.155+0000 7fcaf3524640 1 -- 172.21.15.167:0/3921811646 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcaec0cb4f0 msgr2=0x7fcaec156c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:57.155 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.155+0000 7fcaf3524640 1 --2- 172.21.15.167:0/3921811646 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcaec0cb4f0 0x7fcaec156c50 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fcae000c450 tx=0x7fcae000c910 comp rx=0 tx=0).stop 2024-05-11T08:06:57.155 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.155+0000 7fcaf3524640 1 -- 172.21.15.167:0/3921811646 shutdown_connections 2024-05-11T08:06:57.155 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.155+0000 7fcaf3524640 1 --2- 172.21.15.167:0/3921811646 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fcad00762d0 0x7fcad0078790 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:57.155 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.155+0000 7fcaf3524640 1 --2- 172.21.15.167:0/3921811646 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcaec1598d0 0x7fcaec14cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:57.156 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.155+0000 7fcaf3524640 1 --2- 172.21.15.167:0/3921811646 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcaec0d2cf0 0x7fcaec0ce7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:57.156 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.155+0000 7fcaf3524640 1 --2- 172.21.15.167:0/3921811646 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcaec0cb4f0 0x7fcaec156c50 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:57.156 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.155+0000 7fcaf3524640 1 -- 172.21.15.167:0/3921811646 >> 172.21.15.167:0/3921811646 conn(0x7fcaec0d1ba0 msgr2=0x7fcaec1609d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:57.156 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.155+0000 7fcaf3524640 1 -- 172.21.15.167:0/3921811646 shutdown_connections 2024-05-11T08:06:57.156 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.155+0000 7fcaf3524640 1 -- 172.21.15.167:0/3921811646 wait complete. 2024-05-11T08:06:57.569 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 16, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-11T08:06:47.943939+0000', 'modified': '2024-05-11T08:06:49.955430+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4750}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4750': {'gid': 4750, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.204:6835/469864006', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.204:6833', 'nonce': 469864006}, {'type': 'v1', 'addr': '172.21.15.204:6835', 'nonce': 469864006}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4750, 'qdb_cluster': [4750]} 2024-05-11T08:06:57.569 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-11T08:06:57.570 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-05-11T08:06:57.673 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.672+0000 7fc7e486d640 1 Processor -- start 2024-05-11T08:06:57.673 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.673+0000 7fc7e486d640 1 -- start start 2024-05-11T08:06:57.674 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.674+0000 7fc7e486d640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fc7e00b74c0 0x7fc7e00b7900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:57.674 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.674+0000 7fc7e486d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc7e00cd580 0x7fc7e00cda10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:57.674 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.674+0000 7fc7e486d640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7e00d27c0 con 0x7fc7e015f220 2024-05-11T08:06:57.674 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.674+0000 7fc7e486d640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fc7e015f600 con 0x7fc7e00b74c0 2024-05-11T08:06:57.674 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.674+0000 7fc7e486d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fc7e00cf450 con 0x7fc7e00cd580 2024-05-11T08:06:57.674 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.674+0000 7fc7de59c640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7fc7e015f220 0x7fc7e00d1510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:48354/0 (socket says 172.21.15.167:48354) 2024-05-11T08:06:57.675 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.674+0000 7fc7de59c640 1 -- 172.21.15.167:0/764717990 learned_addr learned my addr 172.21.15.167:0/764717990 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T08:06:57.675 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.674+0000 7fc7ddd9b640 1 --2- 172.21.15.167:0/764717990 >> v2:172.21.15.204:3300/0 conn(0x7fc7e00b74c0 0x7fc7e00b7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:57.675 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.674+0000 7fc7ded9d640 1 --2- 172.21.15.167:0/764717990 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc7e00cd580 0x7fc7e00cda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:57.675 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.674+0000 7fc7dd59a640 1 -- 172.21.15.167:0/764717990 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1037968202 0 0) 0x7fc7e00d27c0 con 0x7fc7e015f220 2024-05-11T08:06:57.675 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.674+0000 7fc7dd59a640 1 -- 172.21.15.167:0/764717990 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc7cc003520 con 0x7fc7e015f220 2024-05-11T08:06:57.675 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.674+0000 7fc7ddd9b640 1 -- 172.21.15.167:0/764717990 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc7e00cd580 msgr2=0x7fc7e00cda10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:57.675 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.674+0000 7fc7ddd9b640 1 --2- 172.21.15.167:0/764717990 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc7e00cd580 0x7fc7e00cda10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:57.675 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.674+0000 7fc7ddd9b640 1 -- 172.21.15.167:0/764717990 >> v1:172.21.15.167:6789/0 conn(0x7fc7e015f220 legacy=0x7fc7e00d1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:57.675 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.674+0000 7fc7ddd9b640 1 -- 172.21.15.167:0/764717990 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7e00cee00 con 0x7fc7e00b74c0 2024-05-11T08:06:57.675 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.674+0000 7fc7ded9d640 1 --2- 172.21.15.167:0/764717990 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc7e00cd580 0x7fc7e00cda10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T08:06:57.676 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.675+0000 7fc7ddd9b640 1 --2- 172.21.15.167:0/764717990 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc7e00b74c0 0x7fc7e00b7900 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fc7c80097a0 tx=0x7fc7c802a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d16cb8962be584aa server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:57.676 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.675+0000 7fc7dd59a640 1 -- 172.21.15.167:0/764717990 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7c8037070 con 0x7fc7e00b74c0 2024-05-11T08:06:57.677 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.675+0000 7fc7dd59a640 1 -- 172.21.15.167:0/764717990 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7c8032440 con 0x7fc7e00b74c0 2024-05-11T08:06:57.677 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.675+0000 7fc7dd59a640 1 -- 172.21.15.167:0/764717990 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7c80327f0 con 0x7fc7e00b74c0 2024-05-11T08:06:57.677 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.676+0000 7fc7e486d640 1 -- 172.21.15.167:0/764717990 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc7e00b74c0 msgr2=0x7fc7e00b7900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:57.677 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.676+0000 7fc7e486d640 1 --2- 172.21.15.167:0/764717990 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc7e00b74c0 0x7fc7e00b7900 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fc7c80097a0 tx=0x7fc7c802a180 comp rx=0 tx=0).stop 2024-05-11T08:06:57.677 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.676+0000 7fc7e486d640 1 -- 172.21.15.167:0/764717990 shutdown_connections 2024-05-11T08:06:57.677 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.676+0000 7fc7e486d640 1 --2- 172.21.15.167:0/764717990 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc7e00cd580 0x7fc7e00cda10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:57.677 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.676+0000 7fc7e486d640 1 --2- 172.21.15.167:0/764717990 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc7e00b74c0 0x7fc7e00b7900 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:57.677 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.676+0000 7fc7e486d640 1 -- 172.21.15.167:0/764717990 >> 172.21.15.167:0/764717990 conn(0x7fc7e00b4ae0 msgr2=0x7fc7e00b4ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:57.677 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.676+0000 7fc7e486d640 1 -- 172.21.15.167:0/764717990 shutdown_connections 2024-05-11T08:06:57.677 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.676+0000 7fc7e486d640 1 -- 172.21.15.167:0/764717990 wait complete. 2024-05-11T08:06:57.679 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.677+0000 7fc7e486d640 1 Processor -- start 2024-05-11T08:06:57.680 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.677+0000 7fc7e486d640 1 -- start start 2024-05-11T08:06:57.680 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.678+0000 7fc7e486d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc7e00b74c0 0x7fc7e01f6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:57.680 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.678+0000 7fc7e486d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc7e00cd580 0x7fc7e01f5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:57.680 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.678+0000 7fc7e486d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc7e015f220 0x7fc7e0157290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:57.680 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.678+0000 7fc7e486d640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc7e00d2400 con 0x7fc7e00b74c0 2024-05-11T08:06:57.680 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.678+0000 7fc7e486d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc7e00ce2e0 con 0x7fc7e00cd580 2024-05-11T08:06:57.680 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.678+0000 7fc7e486d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fc7e00d2580 con 0x7fc7e015f220 2024-05-11T08:06:57.680 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.678+0000 7fc7de59c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc7e00b74c0 0x7fc7e01f6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:57.680 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.678+0000 7fc7de59c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc7e00b74c0 0x7fc7e01f6260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:43010/0 (socket says 172.21.15.167:43010) 2024-05-11T08:06:57.680 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.678+0000 7fc7de59c640 1 -- 172.21.15.167:0/2802076662 learned_addr learned my addr 172.21.15.167:0/2802076662 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:57.681 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.678+0000 7fc7ddd9b640 1 --2- 172.21.15.167:0/2802076662 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc7e00cd580 0x7fc7e01f5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:57.681 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.678+0000 7fc7ded9d640 1 --2- 172.21.15.167:0/2802076662 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc7e015f220 0x7fc7e0157290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:57.681 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.678+0000 7fc7de59c640 1 -- 172.21.15.167:0/2802076662 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc7e015f220 msgr2=0x7fc7e0157290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:57.681 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.678+0000 7fc7de59c640 1 --2- 172.21.15.167:0/2802076662 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc7e015f220 0x7fc7e0157290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:57.681 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.678+0000 7fc7de59c640 1 -- 172.21.15.167:0/2802076662 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc7e00cd580 msgr2=0x7fc7e01f5910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:57.681 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.678+0000 7fc7de59c640 1 --2- 172.21.15.167:0/2802076662 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc7e00cd580 0x7fc7e01f5910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:57.681 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.678+0000 7fc7de59c640 1 -- 172.21.15.167:0/2802076662 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7e00c6cb0 con 0x7fc7e00b74c0 2024-05-11T08:06:57.681 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.678+0000 7fc7ddd9b640 1 --2- 172.21.15.167:0/2802076662 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc7e00cd580 0x7fc7e01f5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T08:06:57.681 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.678+0000 7fc7ded9d640 1 --2- 172.21.15.167:0/2802076662 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc7e015f220 0x7fc7e0157290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T08:06:57.681 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.678+0000 7fc7de59c640 1 --2- 172.21.15.167:0/2802076662 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc7e00b74c0 0x7fc7e01f6260 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fc7d800c440 tx=0x7fc7d800c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:57.681 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.679+0000 7fc7d6ffd640 1 -- 172.21.15.167:0/2802076662 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7d8017070 con 0x7fc7e00b74c0 2024-05-11T08:06:57.681 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.679+0000 7fc7d6ffd640 1 -- 172.21.15.167:0/2802076662 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7d8005ce0 con 0x7fc7e00b74c0 2024-05-11T08:06:57.681 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.679+0000 7fc7e486d640 1 -- 172.21.15.167:0/2802076662 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7e00c6f90 con 0x7fc7e00b74c0 2024-05-11T08:06:57.681 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.679+0000 7fc7d6ffd640 1 -- 172.21.15.167:0/2802076662 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7d8007270 con 0x7fc7e00b74c0 2024-05-11T08:06:57.681 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.679+0000 7fc7e486d640 1 -- 172.21.15.167:0/2802076662 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7e00c7550 con 0x7fc7e00b74c0 2024-05-11T08:06:57.685 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.680+0000 7fc7d6ffd640 1 -- 172.21.15.167:0/2802076662 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fc7d801a070 con 0x7fc7e00b74c0 2024-05-11T08:06:57.685 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.681+0000 7fc7d6ffd640 1 --2- 172.21.15.167:0/2802076662 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fc7c00762d0 0x7fc7c0078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:57.685 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.681+0000 7fc7d6ffd640 1 -- 172.21.15.167:0/2802076662 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6101+0+0 (secure 0 0 0) 0x7fc7d80968a0 con 0x7fc7e00b74c0 2024-05-11T08:06:57.685 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.681+0000 7fc7ddd9b640 1 --2- 172.21.15.167:0/2802076662 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fc7c00762d0 0x7fc7c0078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:57.685 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.681+0000 7fc7ddd9b640 1 --2- 172.21.15.167:0/2802076662 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fc7c00762d0 0x7fc7c0078790 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fc7c8034000 tx=0x7fc7c802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:57.685 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.682+0000 7fc7e7582740 1 Processor -- start 2024-05-11T08:06:57.685 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.682+0000 7fc7e7582740 1 -- start start 2024-05-11T08:06:57.685 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.683+0000 7fc7e7582740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e2b33d5ff0 0x55e2b33d8800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:57.685 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.683+0000 7fc7e7582740 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55e2b33d63d0 0x55e2b33d8d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:57.685 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.683+0000 7fc7e7582740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e2b33a8c50 0x55e2b33ac460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:57.685 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.683+0000 7fc7e7582740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55e2b33128d0 con 0x55e2b33d5ff0 2024-05-11T08:06:57.685 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.683+0000 7fc7e7582740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55e2b32cbc10 con 0x55e2b33a8c50 2024-05-11T08:06:57.685 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.683+0000 7fc7e7582740 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x55e2b330e890 con 0x55e2b33d63d0 2024-05-11T08:06:57.685 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.683+0000 7fc7de59c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e2b33d5ff0 0x55e2b33d8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:57.685 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.683+0000 7fc7de59c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e2b33d5ff0 0x55e2b33d8800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:43024/0 (socket says 172.21.15.167:43024) 2024-05-11T08:06:57.686 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.683+0000 7fc7de59c640 1 -- 172.21.15.167:0/1209896235 learned_addr learned my addr 172.21.15.167:0/1209896235 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:57.686 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.683+0000 7fc7ded9d640 1 --2- 172.21.15.167:0/1209896235 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e2b33a8c50 0x55e2b33ac460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:57.686 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.683+0000 7fc7ddd9b640 1 --2- 172.21.15.167:0/1209896235 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55e2b33d63d0 0x55e2b33d8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:57.686 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.683+0000 7fc7de59c640 1 -- 172.21.15.167:0/1209896235 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55e2b33d63d0 msgr2=0x55e2b33d8d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:57.686 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.683+0000 7fc7de59c640 1 --2- 172.21.15.167:0/1209896235 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55e2b33d63d0 0x55e2b33d8d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:57.686 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.683+0000 7fc7de59c640 1 -- 172.21.15.167:0/1209896235 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e2b33a8c50 msgr2=0x55e2b33ac460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:57.686 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.683+0000 7fc7de59c640 1 --2- 172.21.15.167:0/1209896235 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e2b33a8c50 0x55e2b33ac460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:57.686 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.683+0000 7fc7de59c640 1 -- 172.21.15.167:0/1209896235 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2b32d35d0 con 0x55e2b33d5ff0 2024-05-11T08:06:57.686 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.683+0000 7fc7ddd9b640 1 --2- 172.21.15.167:0/1209896235 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55e2b33d63d0 0x55e2b33d8d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T08:06:57.686 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.683+0000 7fc7de59c640 1 --2- 172.21.15.167:0/1209896235 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e2b33d5ff0 0x55e2b33d8800 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fc7d800c410 tx=0x7fc7d805f320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c64adec3a1a5cae6 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:57.686 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.683+0000 7fc7bffff640 1 -- 172.21.15.167:0/1209896235 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7d8063db0 con 0x55e2b33d5ff0 2024-05-11T08:06:57.686 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.683+0000 7fc7bffff640 1 -- 172.21.15.167:0/1209896235 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7d80687d0 con 0x55e2b33d5ff0 2024-05-11T08:06:57.686 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.683+0000 7fc7bffff640 1 -- 172.21.15.167:0/1209896235 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7d80662b0 con 0x55e2b33d5ff0 2024-05-11T08:06:57.687 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.684+0000 7fc7e7582740 1 -- 172.21.15.167:0/1209896235 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e2b33d5ff0 msgr2=0x55e2b33d8800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:57.687 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.684+0000 7fc7e7582740 1 --2- 172.21.15.167:0/1209896235 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e2b33d5ff0 0x55e2b33d8800 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fc7d800c410 tx=0x7fc7d805f320 comp rx=0 tx=0).stop 2024-05-11T08:06:57.687 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.684+0000 7fc7e7582740 1 -- 172.21.15.167:0/1209896235 shutdown_connections 2024-05-11T08:06:57.687 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.684+0000 7fc7e7582740 1 --2- 172.21.15.167:0/1209896235 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e2b33a8c50 0x55e2b33ac460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:57.687 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.684+0000 7fc7e7582740 1 --2- 172.21.15.167:0/1209896235 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55e2b33d63d0 0x55e2b33d8d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:57.687 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.684+0000 7fc7e7582740 1 --2- 172.21.15.167:0/1209896235 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e2b33d5ff0 0x55e2b33d8800 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:57.687 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.684+0000 7fc7e7582740 1 -- 172.21.15.167:0/1209896235 >> 172.21.15.167:0/1209896235 conn(0x55e2b33aaec0 msgr2=0x55e2b33f3800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:57.687 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.684+0000 7fc7e7582740 1 -- 172.21.15.167:0/1209896235 shutdown_connections 2024-05-11T08:06:57.687 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.684+0000 7fc7e7582740 1 -- 172.21.15.167:0/1209896235 wait complete. 2024-05-11T08:06:57.687 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.685+0000 7fc7e7582740 1 -- start start 2024-05-11T08:06:57.687 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.685+0000 7fc7e7582740 1 Processor -- start 2024-05-11T08:06:57.691 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.686+0000 7fc7e7582740 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55e2b33d5ff0 0x55e2b33f5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:57.691 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.686+0000 7fc7e7582740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e2b33d63d0 0x55e2b33cc5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:57.691 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.686+0000 7fc7e7582740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e2b33f89b0 0x55e2b33f8e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:57.691 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.686+0000 7fc7e7582740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55e2b33ae070 con 0x55e2b33f89b0 2024-05-11T08:06:57.691 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.686+0000 7fc7e7582740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55e2b32cf760 con 0x55e2b33d63d0 2024-05-11T08:06:57.691 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.686+0000 7fc7e7582740 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x55e2b33ae350 con 0x55e2b33d5ff0 2024-05-11T08:06:57.691 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.686+0000 7fc7e7582740 10 client.0 _ll_register_callbacks cb 0x55e2b3308cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T08:06:57.691 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.686+0000 7fc7ded9d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e2b33f89b0 0x55e2b33f8e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:57.691 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.686+0000 7fc7ded9d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e2b33f89b0 0x55e2b33f8e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:43028/0 (socket says 172.21.15.167:43028) 2024-05-11T08:06:57.691 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.686+0000 7fc7ded9d640 1 -- 172.21.15.167:0/4109595258 learned_addr learned my addr 172.21.15.167:0/4109595258 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:57.691 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.686+0000 7fc7de59c640 1 --2- 172.21.15.167:0/4109595258 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55e2b33d5ff0 0x55e2b33f5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:57.691 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.686+0000 7fc7ddd9b640 1 --2- 172.21.15.167:0/4109595258 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e2b33d63d0 0x55e2b33cc5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:57.692 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.686+0000 7fc7ded9d640 1 -- 172.21.15.167:0/4109595258 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55e2b33d5ff0 msgr2=0x55e2b33f5ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:57.692 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.686+0000 7fc7ded9d640 1 --2- 172.21.15.167:0/4109595258 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55e2b33d5ff0 0x55e2b33f5ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:57.692 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.686+0000 7fc7ded9d640 1 -- 172.21.15.167:0/4109595258 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e2b33d63d0 msgr2=0x55e2b33cc5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:57.692 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.686+0000 7fc7ded9d640 1 --2- 172.21.15.167:0/4109595258 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e2b33d63d0 0x55e2b33cc5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:57.692 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.686+0000 7fc7ded9d640 1 -- 172.21.15.167:0/4109595258 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2b30242d0 con 0x55e2b33f89b0 2024-05-11T08:06:57.692 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.686+0000 7fc7ded9d640 1 --2- 172.21.15.167:0/4109595258 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e2b33f89b0 0x55e2b33f8e30 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fc7d0007da0 tx=0x7fc7d0013680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92ded9edfdf22afe server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:57.692 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.686+0000 7fc7bd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-05-11T08:06:57.692 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.686+0000 7fc7bd7fa640 1 -- 172.21.15.167:0/4109595258 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7d001c070 con 0x55e2b33f89b0 2024-05-11T08:06:57.692 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.686+0000 7fc7bd7fa640 1 -- 172.21.15.167:0/4109595258 <== mon.0 v2:172.21.15.167:3300/0 2 ==== osd_map(41..41 src has 1..41) ==== 6101+0+0 (secure 0 0 0) 0x7fc7d0016630 con 0x55e2b33f89b0 2024-05-11T08:06:57.692 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.686+0000 7fc7e7582740 1 -- 172.21.15.167:0/4109595258 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc7d80687d0 con 0x55e2b33f89b0 2024-05-11T08:06:57.692 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.686+0000 7fc7bd7fa640 1 -- 172.21.15.167:0/4109595258 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7fc7d0016830 con 0x55e2b33f89b0 2024-05-11T08:06:57.692 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.686+0000 7fc7e7582740 10 client.5109 fetch_fsmap learned FSMap version 16 2024-05-11T08:06:57.692 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.686+0000 7fc7e7582740 1 -- 172.21.15.167:0/4109595258 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) -- 0x55e2b30ee6b0 con 0x55e2b33f89b0 2024-05-11T08:06:57.692 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.687+0000 7fc7bd7fa640 1 -- 172.21.15.167:0/4109595258 <== mon.0 v2:172.21.15.167:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7d80687d0 con 0x55e2b33f89b0 2024-05-11T08:06:57.692 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.687+0000 7fc7bd7fa640 1 -- 172.21.15.167:0/4109595258 <== mon.0 v2:172.21.15.167:3300/0 5 ==== fsmap(e 16) ==== 3494+0+0 (secure 0 0 0) 0x7fc7d002a080 con 0x55e2b33f89b0 2024-05-11T08:06:57.693 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.687+0000 7fc7e7582740 10 client.5109 fetch_fsmap finished waiting for FSMap version 16 2024-05-11T08:06:57.693 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.687+0000 7fc7e7582740 10 client.5109 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-11T08:06:57.693 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.687+0000 7fc7e7582740 20 client.5109 populate_metadata read hostname 'smithi167' 2024-05-11T08:06:57.693 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.687+0000 7fc7e7582740 1 --2- 172.21.15.167:0/4109595258 >> [v2:172.21.15.204:6833/469864006,v1:172.21.15.204:6835/469864006] conn(0x55e2b340fe80 0x55e2b3412270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:57.693 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.687+0000 7fc7e7582740 4 client.5109 mds_command: new command op to 4750 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-11T08:06:57.693 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.687+0000 7fc7e7582740 1 -- 172.21.15.167:0/4109595258 --> [v2:172.21.15.204:6833/469864006,v1:172.21.15.204:6835/469864006] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e2b3296760 con 0x55e2b340fe80 2024-05-11T08:06:57.693 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.687+0000 7fc7de59c640 1 --2- 172.21.15.167:0/4109595258 >> [v2:172.21.15.204:6833/469864006,v1:172.21.15.204:6835/469864006] conn(0x55e2b340fe80 0x55e2b3412270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:57.693 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.687+0000 7fc7bd7fa640 10 client.5109 ms_handle_connect on v2:172.21.15.204:6833/469864006 2024-05-11T08:06:57.693 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.687+0000 7fc7de59c640 1 --2- 172.21.15.167:0/4109595258 >> [v2:172.21.15.204:6833/469864006,v1:172.21.15.204:6835/469864006] conn(0x55e2b340fe80 0x55e2b3412270 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a4dbfda03a6b1b5 server_cookie=80d5c2b1a06d6b81 in_seq=0 out_seq=0 2024-05-11T08:06:57.693 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.688+0000 7fc7bd7fa640 1 -- 172.21.15.167:0/4109595258 <== mds.0 v2:172.21.15.204:6833/469864006 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e2b3296760 con 0x55e2b340fe80 2024-05-11T08:06:57.693 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.688+0000 7fc7bd7fa640 10 client.5109 handle_command_reply: tid=0 2024-05-11T08:06:57.693 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.688+0000 7fc7e7582740 1 client.5109 shutdown 2024-05-11T08:06:57.693 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.688+0000 7fc7e7582740 1 -- 172.21.15.167:0/4109595258 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e2b33f89b0 msgr2=0x55e2b33f8e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:57.693 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.688+0000 7fc7e7582740 1 --2- 172.21.15.167:0/4109595258 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e2b33f89b0 0x55e2b33f8e30 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fc7d0007da0 tx=0x7fc7d0013680 comp rx=0 tx=0).stop 2024-05-11T08:06:57.693 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.688+0000 7fc7e7582740 1 -- 172.21.15.167:0/4109595258 shutdown_connections 2024-05-11T08:06:57.693 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.688+0000 7fc7e7582740 1 --2- 172.21.15.167:0/4109595258 >> [v2:172.21.15.204:6833/469864006,v1:172.21.15.204:6835/469864006] conn(0x55e2b340fe80 0x55e2b3412270 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:57.694 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.688+0000 7fc7e7582740 1 --2- 172.21.15.167:0/4109595258 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e2b33f89b0 0x55e2b33f8e30 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:57.694 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.688+0000 7fc7bd7fa640 0 client.5109 ms_handle_reset on v2:172.21.15.204:6833/469864006 2024-05-11T08:06:57.694 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.688+0000 7fc7e7582740 1 --2- 172.21.15.167:0/4109595258 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e2b33d63d0 0x55e2b33cc5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:57.694 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.688+0000 7fc7e7582740 1 --2- 172.21.15.167:0/4109595258 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55e2b33d5ff0 0x55e2b33f5ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:57.694 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.688+0000 7fc7e7582740 1 -- 172.21.15.167:0/4109595258 >> 172.21.15.167:0/4109595258 conn(0x55e2b33a8c50 msgr2=0x55e2b33a9a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:57.694 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.688+0000 7fc7e7582740 1 -- 172.21.15.167:0/4109595258 shutdown_connections 2024-05-11T08:06:57.694 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.688+0000 7fc7e7582740 1 -- 172.21.15.167:0/4109595258 wait complete. 2024-05-11T08:06:57.694 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.689+0000 7fc7e7582740 20 client.5109 trim_cache size 0 max 16384 2024-05-11T08:06:57.707 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.707+0000 7fc7e7582740 1 Processor -- start 2024-05-11T08:06:57.707 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.707+0000 7fc7e7582740 1 -- start start 2024-05-11T08:06:57.709 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.708+0000 7fc7e7582740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e2b33d5ff0 0x55e2b33f5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:57.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.708+0000 7fc7e7582740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e2b33d63d0 0x55e2b33a7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:57.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.708+0000 7fc7e7582740 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55e2b33f89b0 0x55e2b33a8020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:57.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.708+0000 7fc7e7582740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc7e0200c40 con 0x55e2b33d63d0 2024-05-11T08:06:57.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.708+0000 7fc7e7582740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc7e0155f90 con 0x55e2b33d5ff0 2024-05-11T08:06:57.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.708+0000 7fc7e7582740 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fc7e00c85b0 con 0x55e2b33f89b0 2024-05-11T08:06:57.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.708+0000 7fc7ddd9b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e2b33d63d0 0x55e2b33a7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:57.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.708+0000 7fc7ddd9b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e2b33d63d0 0x55e2b33a7ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:43030/0 (socket says 172.21.15.167:43030) 2024-05-11T08:06:57.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.708+0000 7fc7ddd9b640 1 -- 172.21.15.167:0/4225019739 learned_addr learned my addr 172.21.15.167:0/4225019739 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:57.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.708+0000 7fc7ddd9b640 1 -- 172.21.15.167:0/4225019739 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55e2b33f89b0 msgr2=0x55e2b33a8020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:57.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.708+0000 7fc7ddd9b640 1 --2- 172.21.15.167:0/4225019739 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55e2b33f89b0 0x55e2b33a8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:57.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.708+0000 7fc7ddd9b640 1 -- 172.21.15.167:0/4225019739 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e2b33d5ff0 msgr2=0x55e2b33f5ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:57.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.708+0000 7fc7de59c640 1 --2- 172.21.15.167:0/4225019739 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e2b33d5ff0 0x55e2b33f5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:57.711 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.708+0000 7fc7ddd9b640 1 --2- 172.21.15.167:0/4225019739 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e2b33d5ff0 0x55e2b33f5ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:57.711 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.708+0000 7fc7ddd9b640 1 -- 172.21.15.167:0/4225019739 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2b33a44f0 con 0x55e2b33d63d0 2024-05-11T08:06:57.711 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.708+0000 7fc7ddd9b640 1 --2- 172.21.15.167:0/4225019739 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e2b33d63d0 0x55e2b33a7ae0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fc7c802a770 tx=0x7fc7c802c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27a13685f38100af server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:57.711 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.709+0000 7fc7bf7fe640 1 -- 172.21.15.167:0/4225019739 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7c8037050 con 0x55e2b33d63d0 2024-05-11T08:06:57.711 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.709+0000 7fc7bf7fe640 1 -- 172.21.15.167:0/4225019739 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7c8051070 con 0x55e2b33d63d0 2024-05-11T08:06:57.711 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.709+0000 7fc7bf7fe640 1 -- 172.21.15.167:0/4225019739 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7c8003d60 con 0x55e2b33d63d0 2024-05-11T08:06:57.711 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.709+0000 7fc7e7582740 1 -- 172.21.15.167:0/4225019739 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e2b33d63d0 msgr2=0x55e2b33a7ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:57.711 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.709+0000 7fc7e7582740 1 --2- 172.21.15.167:0/4225019739 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e2b33d63d0 0x55e2b33a7ae0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fc7c802a770 tx=0x7fc7c802c000 comp rx=0 tx=0).stop 2024-05-11T08:06:57.711 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.709+0000 7fc7e7582740 1 -- 172.21.15.167:0/4225019739 shutdown_connections 2024-05-11T08:06:57.711 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.709+0000 7fc7e7582740 1 --2- 172.21.15.167:0/4225019739 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55e2b33f89b0 0x55e2b33a8020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:57.711 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.709+0000 7fc7e7582740 1 --2- 172.21.15.167:0/4225019739 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e2b33d63d0 0x55e2b33a7ae0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:57.711 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.709+0000 7fc7e7582740 1 --2- 172.21.15.167:0/4225019739 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e2b33d5ff0 0x55e2b33f5ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:57.711 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.709+0000 7fc7e7582740 1 -- 172.21.15.167:0/4225019739 >> 172.21.15.167:0/4225019739 conn(0x55e2b33a8c50 msgr2=0x55e2b33aec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:57.711 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.709+0000 7fc7e7582740 1 -- 172.21.15.167:0/4225019739 shutdown_connections 2024-05-11T08:06:57.711 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.709+0000 7fc7e7582740 1 -- 172.21.15.167:0/4225019739 wait complete. 2024-05-11T08:06:57.711 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.710+0000 7fc7e7582740 1 -- start start 2024-05-11T08:06:57.712 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.710+0000 7fc7e7582740 1 Processor -- start 2024-05-11T08:06:57.715 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.711+0000 7fc7e7582740 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55e2b33d5ff0 0x55e2b3471990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:57.715 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.711+0000 7fc7e7582740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e2b33d63d0 0x55e2b33a9900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:57.716 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.711+0000 7fc7e7582740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e2b33f89b0 0x55e2b33e9190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:57.716 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.711+0000 7fc7e7582740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55e2b33da4b0 con 0x55e2b33d63d0 2024-05-11T08:06:57.716 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.711+0000 7fc7e7582740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55e2b33f9480 con 0x55e2b33f89b0 2024-05-11T08:06:57.716 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.711+0000 7fc7e7582740 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x55e2b33db870 con 0x55e2b33d5ff0 2024-05-11T08:06:57.716 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.711+0000 7fc7e7582740 10 client.0 _ll_register_callbacks cb 0x7fc7d0009e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T08:06:57.716 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.711+0000 7fc7ddd9b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e2b33d63d0 0x55e2b33a9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:57.716 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.711+0000 7fc7ddd9b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e2b33d63d0 0x55e2b33a9900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:43038/0 (socket says 172.21.15.167:43038) 2024-05-11T08:06:57.716 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.711+0000 7fc7ddd9b640 1 -- 172.21.15.167:0/3939856048 learned_addr learned my addr 172.21.15.167:0/3939856048 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:57.716 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.711+0000 7fc7ded9d640 1 --2- 172.21.15.167:0/3939856048 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e2b33f89b0 0x55e2b33e9190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:57.716 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.711+0000 7fc7de59c640 1 --2- 172.21.15.167:0/3939856048 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55e2b33d5ff0 0x55e2b3471990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:57.716 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.711+0000 7fc7ddd9b640 1 -- 172.21.15.167:0/3939856048 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55e2b33d5ff0 msgr2=0x55e2b3471990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:57.716 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.711+0000 7fc7ddd9b640 1 --2- 172.21.15.167:0/3939856048 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55e2b33d5ff0 0x55e2b3471990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:57.716 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.711+0000 7fc7ddd9b640 1 -- 172.21.15.167:0/3939856048 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e2b33f89b0 msgr2=0x55e2b33e9190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:57.716 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.711+0000 7fc7ddd9b640 1 --2- 172.21.15.167:0/3939856048 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e2b33f89b0 0x55e2b33e9190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:57.717 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.711+0000 7fc7ddd9b640 1 -- 172.21.15.167:0/3939856048 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2b33a3600 con 0x55e2b33d63d0 2024-05-11T08:06:57.717 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.711+0000 7fc7ddd9b640 1 --2- 172.21.15.167:0/3939856048 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e2b33d63d0 0x55e2b33a9900 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fc7c804e040 tx=0x7fc7c8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78891314fc6506f4 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:57.717 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.711+0000 7fc7bd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-05-11T08:06:57.717 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.711+0000 7fc7bd7fa640 1 -- 172.21.15.167:0/3939856048 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7c8037070 con 0x55e2b33d63d0 2024-05-11T08:06:57.717 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.711+0000 7fc7bd7fa640 1 -- 172.21.15.167:0/3939856048 <== mon.0 v2:172.21.15.167:3300/0 2 ==== osd_map(41..41 src has 1..41) ==== 6101+0+0 (secure 0 0 0) 0x7fc7c8057660 con 0x55e2b33d63d0 2024-05-11T08:06:57.717 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.711+0000 7fc7e7582740 1 -- 172.21.15.167:0/3939856048 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc7c8051070 con 0x55e2b33d63d0 2024-05-11T08:06:57.717 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.711+0000 7fc7bd7fa640 1 -- 172.21.15.167:0/3939856048 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7fc7c8003e70 con 0x55e2b33d63d0 2024-05-11T08:06:57.717 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.711+0000 7fc7e7582740 10 client.5115 fetch_fsmap learned FSMap version 16 2024-05-11T08:06:57.717 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.711+0000 7fc7e7582740 1 -- 172.21.15.167:0/3939856048 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) -- 0x55e2b33d8cb0 con 0x55e2b33d63d0 2024-05-11T08:06:57.717 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.712+0000 7fc7bd7fa640 1 -- 172.21.15.167:0/3939856048 <== mon.0 v2:172.21.15.167:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7c8051070 con 0x55e2b33d63d0 2024-05-11T08:06:57.717 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.712+0000 7fc7bd7fa640 1 -- 172.21.15.167:0/3939856048 <== mon.0 v2:172.21.15.167:3300/0 5 ==== fsmap(e 16) ==== 3494+0+0 (secure 0 0 0) 0x7fc7c8057860 con 0x55e2b33d63d0 2024-05-11T08:06:57.717 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.712+0000 7fc7e7582740 10 client.5115 fetch_fsmap finished waiting for FSMap version 16 2024-05-11T08:06:57.717 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.712+0000 7fc7e7582740 10 client.5115 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-11T08:06:57.717 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.712+0000 7fc7e7582740 20 client.5115 populate_metadata read hostname 'smithi167' 2024-05-11T08:06:57.718 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.712+0000 7fc7e7582740 1 --2- 172.21.15.167:0/3939856048 >> [v2:172.21.15.204:6833/469864006,v1:172.21.15.204:6835/469864006] conn(0x55e2b33e96d0 0x55e2b33f3800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:57.718 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.712+0000 7fc7e7582740 4 client.5115 mds_command: new command op to 4750 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-11T08:06:57.718 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.712+0000 7fc7e7582740 1 -- 172.21.15.167:0/3939856048 --> [v2:172.21.15.204:6833/469864006,v1:172.21.15.204:6835/469864006] -- command(tid 0: {"prefix": "status"}) -- 0x55e2b33f8520 con 0x55e2b33e96d0 2024-05-11T08:06:57.718 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.712+0000 7fc7de59c640 1 --2- 172.21.15.167:0/3939856048 >> [v2:172.21.15.204:6833/469864006,v1:172.21.15.204:6835/469864006] conn(0x55e2b33e96d0 0x55e2b33f3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:57.718 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.712+0000 7fc7bd7fa640 10 client.5115 ms_handle_connect on v2:172.21.15.204:6833/469864006 2024-05-11T08:06:57.718 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.712+0000 7fc7de59c640 1 --2- 172.21.15.167:0/3939856048 >> [v2:172.21.15.204:6833/469864006,v1:172.21.15.204:6835/469864006] conn(0x55e2b33e96d0 0x55e2b33f3800 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e52c8ce60b07a6b server_cookie=84bfd9070b85a4bc in_seq=0 out_seq=0 2024-05-11T08:06:57.718 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.712+0000 7fc7bd7fa640 1 -- 172.21.15.167:0/3939856048 <== mds.0 v2:172.21.15.204:6833/469864006 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55e2b33f8520 con 0x55e2b33e96d0 2024-05-11T08:06:57.718 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.712+0000 7fc7bd7fa640 10 client.5115 handle_command_reply: tid=0 2024-05-11T08:06:57.718 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.712+0000 7fc7e7582740 1 client.5115 shutdown 2024-05-11T08:06:57.718 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.713+0000 7fc7e7582740 1 -- 172.21.15.167:0/3939856048 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e2b33d63d0 msgr2=0x55e2b33a9900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:57.718 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.713+0000 7fc7e7582740 1 --2- 172.21.15.167:0/3939856048 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e2b33d63d0 0x55e2b33a9900 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fc7c804e040 tx=0x7fc7c8004730 comp rx=0 tx=0).stop 2024-05-11T08:06:57.718 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.713+0000 7fc7e7582740 1 -- 172.21.15.167:0/3939856048 shutdown_connections 2024-05-11T08:06:57.718 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.713+0000 7fc7e7582740 1 --2- 172.21.15.167:0/3939856048 >> [v2:172.21.15.204:6833/469864006,v1:172.21.15.204:6835/469864006] conn(0x55e2b33e96d0 0x55e2b33f3800 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:57.718 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.713+0000 7fc7e7582740 1 --2- 172.21.15.167:0/3939856048 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e2b33f89b0 0x55e2b33e9190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:57.719 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.713+0000 7fc7bd7fa640 0 client.5115 ms_handle_reset on v2:172.21.15.204:6833/469864006 2024-05-11T08:06:57.719 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.713+0000 7fc7e7582740 1 --2- 172.21.15.167:0/3939856048 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e2b33d63d0 0x55e2b33a9900 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:57.719 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.713+0000 7fc7e7582740 1 --2- 172.21.15.167:0/3939856048 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55e2b33d5ff0 0x55e2b3471990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:57.719 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.713+0000 7fc7e7582740 1 -- 172.21.15.167:0/3939856048 >> 172.21.15.167:0/3939856048 conn(0x55e2b33a8c50 msgr2=0x55e2b33a7ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:57.719 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.713+0000 7fc7e7582740 1 -- 172.21.15.167:0/3939856048 shutdown_connections 2024-05-11T08:06:57.719 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.713+0000 7fc7e7582740 1 -- 172.21.15.167:0/3939856048 wait complete. 2024-05-11T08:06:57.719 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.713+0000 7fc7e7582740 20 client.5115 trim_cache size 0 max 16384 2024-05-11T08:06:57.719 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-05-11T08:06:57.719 INFO:teuthology.orchestra.run.smithi167.stdout: "cluster_fsid": "2034778d-8ff1-4823-aa44-ded32b518241", 2024-05-11T08:06:57.719 INFO:teuthology.orchestra.run.smithi167.stdout: "whoami": 0, 2024-05-11T08:06:57.719 INFO:teuthology.orchestra.run.smithi167.stdout: "id": 4750, 2024-05-11T08:06:57.720 INFO:teuthology.orchestra.run.smithi167.stdout: "want_state": "up:active", 2024-05-11T08:06:57.720 INFO:teuthology.orchestra.run.smithi167.stdout: "state": "up:active", 2024-05-11T08:06:57.720 INFO:teuthology.orchestra.run.smithi167.stdout: "fs_name": "cephfs", 2024-05-11T08:06:57.720 INFO:teuthology.orchestra.run.smithi167.stdout: "rank_uptime": 9.1095909689999992, 2024-05-11T08:06:57.720 INFO:teuthology.orchestra.run.smithi167.stdout: "mdsmap_epoch": 16, 2024-05-11T08:06:57.720 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_epoch": 41, 2024-05-11T08:06:57.720 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_epoch_barrier": 41, 2024-05-11T08:06:57.720 INFO:teuthology.orchestra.run.smithi167.stdout: "uptime": 16.015147893999998 2024-05-11T08:06:57.720 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-05-11T08:06:57.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.713+0000 7fc7e486d640 1 -- 172.21.15.167:0/2802076662 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fc7c00762d0 msgr2=0x7fc7c0078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:57.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.713+0000 7fc7e486d640 1 --2- 172.21.15.167:0/2802076662 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fc7c00762d0 0x7fc7c0078790 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fc7c8034000 tx=0x7fc7c802e000 comp rx=0 tx=0).stop 2024-05-11T08:06:57.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.713+0000 7fc7e486d640 1 -- 172.21.15.167:0/2802076662 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc7e00b74c0 msgr2=0x7fc7e01f6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:57.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.713+0000 7fc7e486d640 1 --2- 172.21.15.167:0/2802076662 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc7e00b74c0 0x7fc7e01f6260 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fc7d800c440 tx=0x7fc7d800c900 comp rx=0 tx=0).stop 2024-05-11T08:06:57.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.713+0000 7fc7e486d640 1 -- 172.21.15.167:0/2802076662 shutdown_connections 2024-05-11T08:06:57.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.713+0000 7fc7e486d640 1 --2- 172.21.15.167:0/2802076662 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fc7c00762d0 0x7fc7c0078790 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:57.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.713+0000 7fc7e486d640 1 --2- 172.21.15.167:0/2802076662 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc7e015f220 0x7fc7e0157290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:57.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.713+0000 7fc7e486d640 1 --2- 172.21.15.167:0/2802076662 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc7e00cd580 0x7fc7e01f5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:57.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.713+0000 7fc7e486d640 1 --2- 172.21.15.167:0/2802076662 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc7e00b74c0 0x7fc7e01f6260 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:57.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.713+0000 7fc7e486d640 1 -- 172.21.15.167:0/2802076662 >> 172.21.15.167:0/2802076662 conn(0x7fc7e00b4ae0 msgr2=0x7fc7e0154830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:57.722 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.713+0000 7fc7e486d640 1 -- 172.21.15.167:0/2802076662 shutdown_connections 2024-05-11T08:06:57.722 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.713+0000 7fc7e486d640 1 -- 172.21.15.167:0/2802076662 wait complete. 2024-05-11T08:06:57.724 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-11T08:06:57.825 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.825+0000 7fc161437640 1 Processor -- start 2024-05-11T08:06:57.825 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.825+0000 7fc161437640 1 -- start start 2024-05-11T08:06:57.827 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.826+0000 7fc161437640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc15c1598d0 0x7fc15c0d2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:57.827 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.826+0000 7fc161437640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fc15c0cb6c0 0x7fc15c0d0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:57.827 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.826+0000 7fc161437640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc15c0d1190 con 0x7fc15c0d0640 2024-05-11T08:06:57.827 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.826+0000 7fc161437640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7fc15c06d500 con 0x7fc15c0cb6c0 2024-05-11T08:06:57.827 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.826+0000 7fc161437640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fc15c06d7e0 con 0x7fc15c1598d0 2024-05-11T08:06:57.827 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.826+0000 7fc15b7fe640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fc15c0cb6c0 0x7fc15c0d0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:57.827 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.826+0000 7fc15b7fe640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fc15c0cb6c0 0x7fc15c0d0aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:43044/0 (socket says 172.21.15.167:43044) 2024-05-11T08:06:57.827 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.826+0000 7fc15b7fe640 1 -- 172.21.15.167:0/2495306503 learned_addr learned my addr 172.21.15.167:0/2495306503 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:57.828 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.826+0000 7fc15affd640 1 --2- 172.21.15.167:0/2495306503 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc15c1598d0 0x7fc15c0d2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:57.828 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.826+0000 7fc15b7fe640 1 -- 172.21.15.167:0/2495306503 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc15c1598d0 msgr2=0x7fc15c0d2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:57.828 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.826+0000 7fc15b7fe640 1 --2- 172.21.15.167:0/2495306503 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc15c1598d0 0x7fc15c0d2cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:57.828 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.826+0000 7fc15b7fe640 1 -- 172.21.15.167:0/2495306503 >> v1:172.21.15.204:6789/0 conn(0x7fc15c0d0640 legacy=0x7fc15c0cafb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:57.828 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.826+0000 7fc15b7fe640 1 -- 172.21.15.167:0/2495306503 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc15c06dac0 con 0x7fc15c0cb6c0 2024-05-11T08:06:57.828 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.827+0000 7fc15b7fe640 1 --2- 172.21.15.167:0/2495306503 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc15c0cb6c0 0x7fc15c0d0aa0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fc154009e50 tx=0x7fc15402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5488583034c6e9a server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:57.828 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.827+0000 7fc15a7fc640 1 -- 172.21.15.167:0/2495306503 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc154036070 con 0x7fc15c0cb6c0 2024-05-11T08:06:57.828 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.827+0000 7fc15a7fc640 1 -- 172.21.15.167:0/2495306503 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc154031600 con 0x7fc15c0cb6c0 2024-05-11T08:06:57.828 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.827+0000 7fc15a7fc640 1 -- 172.21.15.167:0/2495306503 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1540319d0 con 0x7fc15c0cb6c0 2024-05-11T08:06:57.829 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.828+0000 7fc161437640 1 -- 172.21.15.167:0/2495306503 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc15c0cb6c0 msgr2=0x7fc15c0d0aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:57.829 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.828+0000 7fc161437640 1 --2- 172.21.15.167:0/2495306503 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc15c0cb6c0 0x7fc15c0d0aa0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fc154009e50 tx=0x7fc15402a710 comp rx=0 tx=0).stop 2024-05-11T08:06:57.829 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.828+0000 7fc161437640 1 -- 172.21.15.167:0/2495306503 shutdown_connections 2024-05-11T08:06:57.829 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.828+0000 7fc161437640 1 --2- 172.21.15.167:0/2495306503 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc15c0cb6c0 0x7fc15c0d0aa0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:57.829 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.828+0000 7fc161437640 1 --2- 172.21.15.167:0/2495306503 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc15c1598d0 0x7fc15c0d2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:57.829 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.828+0000 7fc161437640 1 -- 172.21.15.167:0/2495306503 >> 172.21.15.167:0/2495306503 conn(0x7fc15c0d1ba0 msgr2=0x7fc15c0d1fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:57.829 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.828+0000 7fc161437640 1 -- 172.21.15.167:0/2495306503 shutdown_connections 2024-05-11T08:06:57.830 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.828+0000 7fc161437640 1 -- 172.21.15.167:0/2495306503 wait complete. 2024-05-11T08:06:57.830 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.829+0000 7fc161437640 1 Processor -- start 2024-05-11T08:06:57.830 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.829+0000 7fc161437640 1 -- start start 2024-05-11T08:06:57.832 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.830+0000 7fc161437640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc15c0cb6c0 0x7fc15c0cd0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:57.832 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.830+0000 7fc161437640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc15c0d0640 0x7fc15c14ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:57.832 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.830+0000 7fc161437640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc15c1598d0 0x7fc15c152d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:57.832 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.830+0000 7fc161437640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc15c06d1b0 con 0x7fc15c1598d0 2024-05-11T08:06:57.832 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.830+0000 7fc161437640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc15c159cb0 con 0x7fc15c0cb6c0 2024-05-11T08:06:57.832 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.830+0000 7fc161437640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fc15c06dfe0 con 0x7fc15c0d0640 2024-05-11T08:06:57.832 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.830+0000 7fc15b7fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc15c1598d0 0x7fc15c152d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:57.832 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.830+0000 7fc15b7fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc15c1598d0 0x7fc15c152d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:43046/0 (socket says 172.21.15.167:43046) 2024-05-11T08:06:57.832 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.830+0000 7fc15b7fe640 1 -- 172.21.15.167:0/3952651568 learned_addr learned my addr 172.21.15.167:0/3952651568 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:57.833 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.830+0000 7fc15affd640 1 --2- 172.21.15.167:0/3952651568 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc15c0cb6c0 0x7fc15c0cd0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:57.833 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.830+0000 7fc153fff640 1 --2- 172.21.15.167:0/3952651568 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc15c0d0640 0x7fc15c14ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:57.833 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.830+0000 7fc15b7fe640 1 -- 172.21.15.167:0/3952651568 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc15c0d0640 msgr2=0x7fc15c14ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:57.833 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.830+0000 7fc15b7fe640 1 --2- 172.21.15.167:0/3952651568 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc15c0d0640 0x7fc15c14ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:57.833 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.830+0000 7fc15b7fe640 1 -- 172.21.15.167:0/3952651568 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc15c0cb6c0 msgr2=0x7fc15c0cd0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:57.833 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.830+0000 7fc15b7fe640 1 --2- 172.21.15.167:0/3952651568 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc15c0cb6c0 0x7fc15c0cd0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:57.833 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.830+0000 7fc15b7fe640 1 -- 172.21.15.167:0/3952651568 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc15c153590 con 0x7fc15c1598d0 2024-05-11T08:06:57.833 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.830+0000 7fc153fff640 1 --2- 172.21.15.167:0/3952651568 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc15c0d0640 0x7fc15c14ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T08:06:57.833 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.830+0000 7fc15b7fe640 1 --2- 172.21.15.167:0/3952651568 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc15c1598d0 0x7fc15c152d40 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fc154009f70 tx=0x7fc154004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:57.833 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.830+0000 7fc1537fe640 1 -- 172.21.15.167:0/3952651568 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc15403f020 con 0x7fc15c1598d0 2024-05-11T08:06:57.833 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.830+0000 7fc1537fe640 1 -- 172.21.15.167:0/3952651568 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc154037040 con 0x7fc15c1598d0 2024-05-11T08:06:57.833 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.830+0000 7fc161437640 1 -- 172.21.15.167:0/3952651568 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc15c153870 con 0x7fc15c1598d0 2024-05-11T08:06:57.833 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.830+0000 7fc1537fe640 1 -- 172.21.15.167:0/3952651568 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc154036070 con 0x7fc15c1598d0 2024-05-11T08:06:57.833 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.831+0000 7fc161437640 1 -- 172.21.15.167:0/3952651568 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc15c153e30 con 0x7fc15c1598d0 2024-05-11T08:06:57.834 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.831+0000 7fc161437640 1 -- 172.21.15.167:0/3952651568 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc15c069590 con 0x7fc15c1598d0 2024-05-11T08:06:57.834 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.832+0000 7fc1537fe640 1 -- 172.21.15.167:0/3952651568 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fc154042020 con 0x7fc15c1598d0 2024-05-11T08:06:57.834 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.833+0000 7fc1537fe640 1 --2- 172.21.15.167:0/3952651568 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fc13c0762d0 0x7fc13c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:57.834 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.833+0000 7fc15affd640 1 --2- 172.21.15.167:0/3952651568 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fc13c0762d0 0x7fc13c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:57.834 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.833+0000 7fc1537fe640 1 -- 172.21.15.167:0/3952651568 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6101+0+0 (secure 0 0 0) 0x7fc1540b7b40 con 0x7fc15c1598d0 2024-05-11T08:06:57.835 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.833+0000 7fc15affd640 1 --2- 172.21.15.167:0/3952651568 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fc13c0762d0 0x7fc13c078790 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc14c009aa0 tx=0x7fc14c009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:57.837 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:57.837+0000 7fc1537fe640 1 -- 172.21.15.167:0/3952651568 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fc154049030 con 0x7fc15c1598d0 2024-05-11T08:06:58.056 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:58.056+0000 7fc161437640 1 -- 172.21.15.167:0/3952651568 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fc15c15fb70 con 0x7fc15c1598d0 2024-05-11T08:06:58.057 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:58.057+0000 7fc1537fe640 1 -- 172.21.15.167:0/3952651568 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) ==== 53+0+73 (secure 0 0 0) 0x7fc154081310 con 0x7fc15c1598d0 2024-05-11T08:06:58.057 INFO:teuthology.orchestra.run.smithi167.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-11T08:06:58.059 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:58.059+0000 7fc161437640 1 -- 172.21.15.167:0/3952651568 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fc13c0762d0 msgr2=0x7fc13c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:58.059 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:58.059+0000 7fc161437640 1 --2- 172.21.15.167:0/3952651568 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fc13c0762d0 0x7fc13c078790 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc14c009aa0 tx=0x7fc14c009190 comp rx=0 tx=0).stop 2024-05-11T08:06:58.059 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:58.059+0000 7fc161437640 1 -- 172.21.15.167:0/3952651568 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc15c1598d0 msgr2=0x7fc15c152d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:58.059 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:58.059+0000 7fc161437640 1 --2- 172.21.15.167:0/3952651568 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc15c1598d0 0x7fc15c152d40 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fc154009f70 tx=0x7fc154004a40 comp rx=0 tx=0).stop 2024-05-11T08:06:58.059 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:58.059+0000 7fc161437640 1 -- 172.21.15.167:0/3952651568 shutdown_connections 2024-05-11T08:06:58.059 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:58.059+0000 7fc161437640 1 --2- 172.21.15.167:0/3952651568 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fc13c0762d0 0x7fc13c078790 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:58.059 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:58.059+0000 7fc161437640 1 --2- 172.21.15.167:0/3952651568 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc15c1598d0 0x7fc15c152d40 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:58.060 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:58.059+0000 7fc161437640 1 --2- 172.21.15.167:0/3952651568 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc15c0d0640 0x7fc15c14ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:58.060 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:58.059+0000 7fc161437640 1 --2- 172.21.15.167:0/3952651568 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc15c0cb6c0 0x7fc15c0cd0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:58.060 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:58.059+0000 7fc161437640 1 -- 172.21.15.167:0/3952651568 >> 172.21.15.167:0/3952651568 conn(0x7fc15c0d1ba0 msgr2=0x7fc15c160ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:58.060 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:58.059+0000 7fc161437640 1 -- 172.21.15.167:0/3952651568 shutdown_connections 2024-05-11T08:06:58.060 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:58.059+0000 7fc161437640 1 -- 172.21.15.167:0/3952651568 wait complete. 2024-05-11T08:06:58.070 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-11T08:06:58.070 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-11T08:06:58.070 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi167.front.sepia.ceph.com 2024-05-11T08:06:58.070 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-11T08:06:58.070 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-11T08:06:58.070 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-11T08:06:58.070 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-05-11T08:06:58.071 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-11T08:06:58.071 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-11T08:06:58.071 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T08:06:58.071 DEBUG:teuthology.orchestra.run.smithi167:> ip addr 2024-05-11T08:06:58.087 INFO:teuthology.orchestra.run.smithi167.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-11T08:06:58.087 INFO:teuthology.orchestra.run.smithi167.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-11T08:06:58.087 INFO:teuthology.orchestra.run.smithi167.stdout: inet 127.0.0.1/8 scope host lo 2024-05-11T08:06:58.087 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2024-05-11T08:06:58.087 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 ::1/128 scope host 2024-05-11T08:06:58.087 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2024-05-11T08:06:58.087 INFO:teuthology.orchestra.run.smithi167.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-11T08:06:58.087 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:d9:35:e6 brd ff:ff:ff:ff:ff:ff 2024-05-11T08:06:58.087 INFO:teuthology.orchestra.run.smithi167.stdout: altname enp7s0f0 2024-05-11T08:06:58.087 INFO:teuthology.orchestra.run.smithi167.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-11T08:06:58.087 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:d9:35:e7 brd ff:ff:ff:ff:ff:ff 2024-05-11T08:06:58.088 INFO:teuthology.orchestra.run.smithi167.stdout: altname enp7s0f1 2024-05-11T08:06:58.088 INFO:teuthology.orchestra.run.smithi167.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-11T08:06:58.088 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:88:73:08 brd ff:ff:ff:ff:ff:ff 2024-05-11T08:06:58.088 INFO:teuthology.orchestra.run.smithi167.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-11T08:06:58.088 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:88:73:09 brd ff:ff:ff:ff:ff:ff 2024-05-11T08:06:58.088 INFO:teuthology.orchestra.run.smithi167.stdout: inet 172.21.15.167/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-05-11T08:06:58.088 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft 42665sec preferred_lft 42665sec 2024-05-11T08:06:58.088 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::ec4:7aff:fe88:7309/64 scope link 2024-05-11T08:06:58.088 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2024-05-11T08:06:58.088 INFO:teuthology.orchestra.run.smithi167.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-11T08:06:58.088 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether ba:bf:3e:5f:7c:d5 brd ff:ff:ff:ff:ff:ff 2024-05-11T08:06:58.088 INFO:teuthology.orchestra.run.smithi167.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-11T08:06:58.088 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2024-05-11T08:06:58.088 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::c0f2:32ff:fe0b:cf4f/64 scope link 2024-05-11T08:06:58.088 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2024-05-11T08:06:58.089 INFO:teuthology.orchestra.run.smithi167.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-11T08:06:58.089 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether ce:35:1f:82:16:86 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-11T08:06:58.089 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::cc35:1fff:fe82:1686/64 scope link 2024-05-11T08:06:58.089 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2024-05-11T08:06:58.089 INFO:teuthology.orchestra.run.smithi167.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-11T08:06:58.089 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether ba:bf:3e:5f:7c:d5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-11T08:06:58.089 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::b8bf:3eff:fe5f:7cd5/64 scope link 2024-05-11T08:06:58.089 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2024-05-11T08:06:58.089 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T08:06:58.089 DEBUG:teuthology.orchestra.run.smithi167:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-11T08:06:58.151 INFO:teuthology.orchestra.run.smithi167.stdout:1 2024-05-11T08:06:58.152 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T08:06:58.152 DEBUG:teuthology.orchestra.run.smithi167:> route 2024-05-11T08:06:58.208 INFO:teuthology.orchestra.run.smithi167.stdout:Kernel IP routing table 2024-05-11T08:06:58.208 INFO:teuthology.orchestra.run.smithi167.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-11T08:06:58.209 INFO:teuthology.orchestra.run.smithi167.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-11T08:06:58.209 INFO:teuthology.orchestra.run.smithi167.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-11T08:06:58.209 INFO:teuthology.orchestra.run.smithi167.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-11T08:06:58.209 DEBUG:teuthology.orchestra.run.smithi167:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-11T08:06:58.209 DEBUG:teuthology.orchestra.run.smithi167:> set -e 2024-05-11T08:06:58.209 DEBUG:teuthology.orchestra.run.smithi167:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-11T08:06:58.209 DEBUG:teuthology.orchestra.run.smithi167:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-11T08:06:58.209 DEBUG:teuthology.orchestra.run.smithi167:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-11T08:06:58.209 DEBUG:teuthology.orchestra.run.smithi167:> ') 2024-05-11T08:06:58.298 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:58 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-11T08:06:58.330 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T08:06:58.336 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T08:06:58.336 DEBUG:teuthology.orchestra.run.smithi167:> ip netns list 2024-05-11T08:06:58.352 INFO:teuthology.orchestra.run.smithi167.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-11T08:06:58.352 INFO:teuthology.orchestra.run.smithi167.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-11T08:06:58.352 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-11T08:06:58.352 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T08:06:58.352 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip link set brx.0 up 2024-05-11T08:06:58.417 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-11T08:06:58.417 DEBUG:teuthology.orchestra.run.smithi167:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-11T08:06:58.472 INFO:teuthology.orchestra.run.smithi167.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-11T08:06:58.473 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-11T08:06:58.473 DEBUG:teuthology.orchestra.run.smithi167:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-11T08:06:58.528 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-05-11T08:06:58.528 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T08:06:58.528 DEBUG:teuthology.orchestra.run.smithi167:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-11T08:06:58.611 INFO:teuthology.orchestra.run.smithi167.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-11T08:06:58.611 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-11T08:06:58.611 INFO:teuthology.orchestra.run.smithi167.stdout:invalid new device string format 2024-05-11T08:06:58.620 INFO:teuthology.orchestra.run.smithi167.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-11T08:06:58.620 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-11T08:06:58.620 INFO:teuthology.orchestra.run.smithi167.stdout:invalid new device string format 2024-05-11T08:06:58.621 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: resolved to: "172.21.15.167:3300,172.21.15.204:3300,172.21.15.204:3301" 2024-05-11T08:06:58.621 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-11T08:06:58.632 INFO:teuthology.orchestra.run.smithi167.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-05-11T08:06:58.632 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-05-11T08:06:58.632 INFO:teuthology.orchestra.run.smithi167.stdout:invalid new device string format 2024-05-11T08:06:58.632 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: resolved to: "172.21.15.167:3300,172.21.15.204:3300,172.21.15.204:3301" 2024-05-11T08:06:58.632 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: switching to using v1 address with old syntax 2024-05-11T08:06:58.632 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: resolved to: "172.21.15.167:6789,172.21.15.204:6789,172.21.15.204:6790" 2024-05-11T08:06:58.632 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.167:6789,172.21.15.204:6789,172.21.15.204:6790:/ 2024-05-11T08:06:58.632 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=2034778d-8ff1-4823-aa44-ded32b518241" will pass to kernel 2024-05-11T08:06:58.633 INFO:teuthology.orchestra.run.smithi167.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-05-11T08:06:58.633 INFO:teuthology.orchestra.run.smithi167.stdout: You just mounted a file system that supports labels which does not 2024-05-11T08:06:58.633 INFO:teuthology.orchestra.run.smithi167.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-11T08:06:58.633 INFO:teuthology.orchestra.run.smithi167.stdout: applications will generate AVC messages and not be allowed access to 2024-05-11T08:06:58.633 INFO:teuthology.orchestra.run.smithi167.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-11T08:06:58.635 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-11T08:06:58.635 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T08:06:58.635 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-11T08:06:58.701 DEBUG:teuthology.orchestra.run.smithi167:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-11T08:06:58.792 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:58 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-11T08:06:58.799 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T08:06:58.805 INFO:teuthology.orchestra.run.smithi167.stdout:client4894 2024-05-11T08:06:58.805 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-05-11T08:06:58.805 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T08:06:58.821 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_symlink_recovery false 2024-05-11T08:06:58.965 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:58.965+0000 7f690a1c9640 1 Processor -- start 2024-05-11T08:06:58.965 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:58.965+0000 7f690a1c9640 1 -- start start 2024-05-11T08:06:58.966 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:58.966+0000 7f690a1c9640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f6904073270 0x7f690415d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:58.966 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:58.966+0000 7f690a1c9640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f6904073650 0x7f690406e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:58.967 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:58.966+0000 7f690a1c9640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f69040ce8f0 0x7f69040ced80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:58.967 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:58.966+0000 7f690a1c9640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f6904160b00 con 0x7f6904073270 2024-05-11T08:06:58.967 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:58.966+0000 7f690a1c9640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f690415cce0 con 0x7f6904073650 2024-05-11T08:06:58.967 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:58.966+0000 7f690a1c9640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f690415c6d0 con 0x7f69040ce8f0 2024-05-11T08:06:58.967 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:58.966+0000 7f69037fe640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f6904073270 0x7f690415d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:58.967 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:58.966+0000 7f69037fe640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f6904073270 0x7f690415d170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:43062/0 (socket says 172.21.15.167:43062) 2024-05-11T08:06:58.967 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:58.966+0000 7f69037fe640 1 -- 172.21.15.167:0/1336687501 learned_addr learned my addr 172.21.15.167:0/1336687501 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:58.967 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:58.966+0000 7f6903fff640 1 --2- 172.21.15.167:0/1336687501 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f69040ce8f0 0x7f69040ced80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:58.967 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:58.966+0000 7f69037fe640 1 -- 172.21.15.167:0/1336687501 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f69040ce8f0 msgr2=0x7f69040ced80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:58.967 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:58.966+0000 7f6902ffd640 1 --2- 172.21.15.167:0/1336687501 >> v2:172.21.15.204:3300/0 conn(0x7f6904073650 0x7f690406e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:58.967 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:58.966+0000 7f69037fe640 1 --2- 172.21.15.167:0/1336687501 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f69040ce8f0 0x7f69040ced80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:58.967 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:58.966+0000 7f69037fe640 1 -- 172.21.15.167:0/1336687501 >> v2:172.21.15.204:3300/0 conn(0x7f6904073650 msgr2=0x7f690406e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:58.967 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:58.966+0000 7f69037fe640 1 --2- 172.21.15.167:0/1336687501 >> v2:172.21.15.204:3300/0 conn(0x7f6904073650 0x7f690406e3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:58.967 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:58.966+0000 7f69037fe640 1 -- 172.21.15.167:0/1336687501 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69040700b0 con 0x7f6904073270 2024-05-11T08:06:58.968 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:58.967+0000 7f69037fe640 1 --2- 172.21.15.167:0/1336687501 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6904073270 0x7f690415d170 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f68f4009e50 tx=0x7f68f402a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5debcbcacfa5f71 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:58.969 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:58.967+0000 7f69027fc640 1 -- 172.21.15.167:0/1336687501 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68f4036070 con 0x7f6904073270 2024-05-11T08:06:58.969 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:58.967+0000 7f69027fc640 1 -- 172.21.15.167:0/1336687501 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68f4031600 con 0x7f6904073270 2024-05-11T08:06:58.969 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:58.967+0000 7f69027fc640 1 -- 172.21.15.167:0/1336687501 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68f40319d0 con 0x7f6904073270 2024-05-11T08:06:58.969 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:58.968+0000 7f690a1c9640 1 -- 172.21.15.167:0/1336687501 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6904073270 msgr2=0x7f690415d170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:58.969 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:58.968+0000 7f690a1c9640 1 --2- 172.21.15.167:0/1336687501 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6904073270 0x7f690415d170 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f68f4009e50 tx=0x7f68f402a6c0 comp rx=0 tx=0).stop 2024-05-11T08:06:58.969 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:58.968+0000 7f690a1c9640 1 -- 172.21.15.167:0/1336687501 shutdown_connections 2024-05-11T08:06:58.969 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:58.968+0000 7f690a1c9640 1 --2- 172.21.15.167:0/1336687501 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f69040ce8f0 0x7f69040ced80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:58.969 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:58.968+0000 7f690a1c9640 1 --2- 172.21.15.167:0/1336687501 >> v2:172.21.15.204:3300/0 conn(0x7f6904073650 0x7f690406e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:58.969 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:58.968+0000 7f690a1c9640 1 --2- 172.21.15.167:0/1336687501 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6904073270 0x7f690415d170 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:58.969 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:58.968+0000 7f690a1c9640 1 -- 172.21.15.167:0/1336687501 >> 172.21.15.167:0/1336687501 conn(0x7f690406c450 msgr2=0x7f690406c860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:58.969 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:58.968+0000 7f690a1c9640 1 -- 172.21.15.167:0/1336687501 shutdown_connections 2024-05-11T08:06:58.969 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:58.968+0000 7f690a1c9640 1 -- 172.21.15.167:0/1336687501 wait complete. 2024-05-11T08:06:58.969 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:58.969+0000 7f690a1c9640 1 Processor -- start 2024-05-11T08:06:58.971 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:58.969+0000 7f690a1c9640 1 -- start start 2024-05-11T08:06:58.971 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:58.970+0000 7f690a1c9640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6904073270 0x7f69041378d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:58.971 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:58.970+0000 7f690a1c9640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6904073650 0x7f690413b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:58.972 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:58.970+0000 7f690a1c9640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f69040ce8f0 0x7f690415d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:58.972 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:58.970+0000 7f690a1c9640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6904073b00 con 0x7f6904073650 2024-05-11T08:06:58.972 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:58.970+0000 7f690a1c9640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6904070390 con 0x7f6904073270 2024-05-11T08:06:58.972 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:58.970+0000 7f690a1c9640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f690406fd60 con 0x7f69040ce8f0 2024-05-11T08:06:58.972 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:58.970+0000 7f6902ffd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6904073650 0x7f690413b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:58.972 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:58.970+0000 7f6902ffd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6904073650 0x7f690413b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:43070/0 (socket says 172.21.15.167:43070) 2024-05-11T08:06:58.972 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:58.970+0000 7f6902ffd640 1 -- 172.21.15.167:0/2911919440 learned_addr learned my addr 172.21.15.167:0/2911919440 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:58.972 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:58.970+0000 7f6902ffd640 1 -- 172.21.15.167:0/2911919440 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f69040ce8f0 msgr2=0x7f690415d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:58.972 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:58.970+0000 7f6902ffd640 1 --2- 172.21.15.167:0/2911919440 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f69040ce8f0 0x7f690415d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:58.972 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:58.970+0000 7f6902ffd640 1 -- 172.21.15.167:0/2911919440 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6904073270 msgr2=0x7f69041378d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:58.972 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:58.970+0000 7f6902ffd640 1 --2- 172.21.15.167:0/2911919440 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6904073270 0x7f69041378d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:58.972 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:58.970+0000 7f6902ffd640 1 -- 172.21.15.167:0/2911919440 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f690413a320 con 0x7f6904073650 2024-05-11T08:06:58.972 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:58.970+0000 7f6902ffd640 1 --2- 172.21.15.167:0/2911919440 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6904073650 0x7f690413b3c0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f68f800c820 tx=0x7f68f800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:58.973 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:58.970+0000 7f68ebfff640 1 -- 172.21.15.167:0/2911919440 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68f8015070 con 0x7f6904073650 2024-05-11T08:06:58.973 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:58.970+0000 7f68ebfff640 1 -- 172.21.15.167:0/2911919440 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68f8004220 con 0x7f6904073650 2024-05-11T08:06:58.973 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:58.970+0000 7f690a1c9640 1 -- 172.21.15.167:0/2911919440 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f690413a600 con 0x7f6904073650 2024-05-11T08:06:58.973 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:58.970+0000 7f68ebfff640 1 -- 172.21.15.167:0/2911919440 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68f800ce10 con 0x7f6904073650 2024-05-11T08:06:58.973 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:58.971+0000 7f690a1c9640 1 -- 172.21.15.167:0/2911919440 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f690413abc0 con 0x7f6904073650 2024-05-11T08:06:58.973 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:58.971+0000 7f690a1c9640 1 -- 172.21.15.167:0/2911919440 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6904069520 con 0x7f6904073650 2024-05-11T08:06:58.973 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:58.972+0000 7f68ebfff640 1 -- 172.21.15.167:0/2911919440 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f68f8007830 con 0x7f6904073650 2024-05-11T08:06:58.974 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:58.972+0000 7f68ebfff640 1 --2- 172.21.15.167:0/2911919440 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f68f00762d0 0x7f68f0078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:58.974 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:58.973+0000 7f68ebfff640 1 -- 172.21.15.167:0/2911919440 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6101+0+0 (secure 0 0 0) 0x7f68f80975d0 con 0x7f6904073650 2024-05-11T08:06:58.974 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:58.973+0000 7f69037fe640 1 --2- 172.21.15.167:0/2911919440 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f68f00762d0 0x7f68f0078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:58.974 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:58.973+0000 7f69037fe640 1 --2- 172.21.15.167:0/2911919440 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f68f00762d0 0x7f68f0078790 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f68f4033000 tx=0x7f68f402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:58.977 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:58.977+0000 7f68ebfff640 1 -- 172.21.15.167:0/2911919440 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f68f8060d10 con 0x7f6904073650 2024-05-11T08:06:59.157 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:59.157+0000 7f690a1c9640 1 -- 172.21.15.167:0/2911919440 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command([{prefix=config set, name=mds_symlink_recovery}] v 0) -- 0x7f69041608e0 con 0x7f6904073650 2024-05-11T08:06:59.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:59.160+0000 7f68ebfff640 1 -- 172.21.15.167:0/2911919440 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_symlink_recovery}]=0 v11)=0 v11) ==== 122+0+0 (secure 0 0 0) 0x7f68f8060610 con 0x7f6904073650 2024-05-11T08:06:59.163 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:59.162+0000 7f690a1c9640 1 -- 172.21.15.167:0/2911919440 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f68f00762d0 msgr2=0x7f68f0078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:59.163 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:59.162+0000 7f690a1c9640 1 --2- 172.21.15.167:0/2911919440 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f68f00762d0 0x7f68f0078790 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f68f4033000 tx=0x7f68f402e000 comp rx=0 tx=0).stop 2024-05-11T08:06:59.163 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:59.162+0000 7f690a1c9640 1 -- 172.21.15.167:0/2911919440 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6904073650 msgr2=0x7f690413b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:59.163 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:59.162+0000 7f690a1c9640 1 --2- 172.21.15.167:0/2911919440 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6904073650 0x7f690413b3c0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f68f800c820 tx=0x7f68f800cce0 comp rx=0 tx=0).stop 2024-05-11T08:06:59.163 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:59.162+0000 7f690a1c9640 1 -- 172.21.15.167:0/2911919440 shutdown_connections 2024-05-11T08:06:59.163 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:59.162+0000 7f690a1c9640 1 --2- 172.21.15.167:0/2911919440 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f68f00762d0 0x7f68f0078790 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:59.163 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:59.162+0000 7f690a1c9640 1 --2- 172.21.15.167:0/2911919440 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f69040ce8f0 0x7f690415d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:59.163 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:59.162+0000 7f690a1c9640 1 --2- 172.21.15.167:0/2911919440 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6904073650 0x7f690413b3c0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:59.163 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:59.162+0000 7f690a1c9640 1 --2- 172.21.15.167:0/2911919440 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6904073270 0x7f69041378d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:59.163 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:59.162+0000 7f690a1c9640 1 -- 172.21.15.167:0/2911919440 >> 172.21.15.167:0/2911919440 conn(0x7f690406c450 msgr2=0x7f69040b0ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:59.163 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:59.162+0000 7f690a1c9640 1 -- 172.21.15.167:0/2911919440 shutdown_connections 2024-05-11T08:06:59.164 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:59.162+0000 7f690a1c9640 1 -- 172.21.15.167:0/2911919440 wait complete. 2024-05-11T08:06:59.174 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T08:06:59.174 DEBUG:teuthology.orchestra.run.smithi167:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir sym_dir0) 2024-05-11T08:06:59.195 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T08:06:59.196 DEBUG:teuthology.orchestra.run.smithi167:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch sym_dir0/file1) 2024-05-11T08:06:59.253 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T08:06:59.253 DEBUG:teuthology.orchestra.run.smithi167:> (cd /home/ubuntu/cephtest/mnt.0 && exec ln -s sym_dir0/file1 sym_dir0/symlink_file1) 2024-05-11T08:06:59.310 DEBUG:teuthology.orchestra.run.smithi167:> stdin-killer -- python3 -c ' 2024-05-11T08:06:59.310 DEBUG:teuthology.orchestra.run.smithi167:> import os 2024-05-11T08:06:59.311 DEBUG:teuthology.orchestra.run.smithi167:> import stat 2024-05-11T08:06:59.311 DEBUG:teuthology.orchestra.run.smithi167:> 2024-05-11T08:06:59.311 DEBUG:teuthology.orchestra.run.smithi167:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/sym_dir0/symlink_file1").st_ino) 2024-05-11T08:06:59.311 DEBUG:teuthology.orchestra.run.smithi167:> ' 2024-05-11T08:06:59.417 INFO:teuthology.orchestra.run.smithi167.stdout:1099511628278 2024-05-11T08:06:59.420 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T08:06:59.426 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-11T08:06:59.565 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:59.565+0000 7f0997f58640 1 Processor -- start 2024-05-11T08:06:59.565 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:59.565+0000 7f0997f58640 1 -- start start 2024-05-11T08:06:59.567 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:59.566+0000 7f0997f58640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f099013f290 0x7f09900735c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:59.567 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:59.566+0000 7f0997f58640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f09900b50a0 0x7f0990073b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:59.568 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:59.566+0000 7f0997f58640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f099015fb90 con 0x7f099013eeb0 2024-05-11T08:06:59.568 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:59.566+0000 7f0997f58640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f099015d5b0 con 0x7f099013f290 2024-05-11T08:06:59.568 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:59.566+0000 7f0997f58640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f099015fe70 con 0x7f09900b50a0 2024-05-11T08:06:59.568 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:59.566+0000 7f09954f3640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f099013f290 0x7f09900735c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:59.568 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:59.566+0000 7f0995cf4640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f099013eeb0 0x7f099006f480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:48356/0 (socket says 172.21.15.167:48356) 2024-05-11T08:06:59.568 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:59.566+0000 7f0995cf4640 1 -- 172.21.15.167:0/1982478305 learned_addr learned my addr 172.21.15.167:0/1982478305 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T08:06:59.568 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:59.567+0000 7f09964f5640 1 --2- 172.21.15.167:0/1982478305 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f09900b50a0 0x7f0990073b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:59.568 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:59.567+0000 7f09954f3640 1 -- 172.21.15.167:0/1982478305 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f09900b50a0 msgr2=0x7f0990073b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:06:59.568 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:59.567+0000 7f09954f3640 1 --2- 172.21.15.167:0/1982478305 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f09900b50a0 0x7f0990073b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:59.568 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:59.567+0000 7f09954f3640 1 -- 172.21.15.167:0/1982478305 >> v1:172.21.15.167:6789/0 conn(0x7f099013eeb0 legacy=0x7f099006f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:59.568 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:59.567+0000 7f0994cf2640 1 -- 172.21.15.167:0/1982478305 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2772479620 0 0) 0x7f099015fb90 con 0x7f099013eeb0 2024-05-11T08:06:59.568 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:59.567+0000 7f09954f3640 1 -- 172.21.15.167:0/1982478305 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0990160150 con 0x7f099013f290 2024-05-11T08:06:59.568 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:59.567+0000 7f09964f5640 1 --2- 172.21.15.167:0/1982478305 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f09900b50a0 0x7f0990073b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T08:06:59.569 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:59.567+0000 7f09954f3640 1 --2- 172.21.15.167:0/1982478305 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f099013f290 0x7f09900735c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f09800097a0 tx=0x7f098002a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c2cf0af2427bfc9 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:59.569 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:59.567+0000 7f0994cf2640 1 -- 172.21.15.167:0/1982478305 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0980037070 con 0x7f099013f290 2024-05-11T08:06:59.569 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:59.567+0000 7f0994cf2640 1 -- 172.21.15.167:0/1982478305 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f098002ae50 con 0x7f099013f290 2024-05-11T08:06:59.569 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:59.568+0000 7f0994cf2640 1 -- 172.21.15.167:0/1982478305 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0980032660 con 0x7f099013f290 2024-05-11T08:06:59.572 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:59.568+0000 7f0997f58640 1 -- 172.21.15.167:0/1982478305 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f099013f290 msgr2=0x7f09900735c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:59.572 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:59.568+0000 7f0997f58640 1 --2- 172.21.15.167:0/1982478305 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f099013f290 0x7f09900735c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f09800097a0 tx=0x7f098002a100 comp rx=0 tx=0).stop 2024-05-11T08:06:59.572 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:59.568+0000 7f0997f58640 1 -- 172.21.15.167:0/1982478305 shutdown_connections 2024-05-11T08:06:59.572 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:59.568+0000 7f0997f58640 1 --2- 172.21.15.167:0/1982478305 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f09900b50a0 0x7f0990073b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:59.572 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:59.568+0000 7f0997f58640 1 --2- 172.21.15.167:0/1982478305 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f099013f290 0x7f09900735c0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:59.572 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:59.568+0000 7f0997f58640 1 -- 172.21.15.167:0/1982478305 >> 172.21.15.167:0/1982478305 conn(0x7f09900cd8f0 msgr2=0x7f09900cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:59.572 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:59.568+0000 7f0997f58640 1 -- 172.21.15.167:0/1982478305 shutdown_connections 2024-05-11T08:06:59.572 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:59.568+0000 7f0997f58640 1 -- 172.21.15.167:0/1982478305 wait complete. 2024-05-11T08:06:59.572 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:59.569+0000 7f0997f58640 1 Processor -- start 2024-05-11T08:06:59.573 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:59.569+0000 7f0997f58640 1 -- start start 2024-05-11T08:06:59.573 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:59.570+0000 7f0997f58640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f09900b50a0 0x7f099013c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:59.573 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:59.570+0000 7f0997f58640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f099013eeb0 0x7f099014cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:59.573 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:59.570+0000 7f0997f58640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f099013f290 0x7f099013ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:59.573 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:59.570+0000 7f0997f58640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0990074040 con 0x7f09900b50a0 2024-05-11T08:06:59.573 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:59.570+0000 7f0997f58640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0990160710 con 0x7f099013f290 2024-05-11T08:06:59.573 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:59.570+0000 7f0997f58640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f099006fb90 con 0x7f099013eeb0 2024-05-11T08:06:59.573 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:59.570+0000 7f0995cf4640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f09900b50a0 0x7f099013c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:59.573 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:59.570+0000 7f0995cf4640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f09900b50a0 0x7f099013c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:43096/0 (socket says 172.21.15.167:43096) 2024-05-11T08:06:59.573 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:59.570+0000 7f0995cf4640 1 -- 172.21.15.167:0/1848030839 learned_addr learned my addr 172.21.15.167:0/1848030839 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:06:59.573 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:59.570+0000 7f09954f3640 1 --2- 172.21.15.167:0/1848030839 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f099013eeb0 0x7f099014cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:59.573 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:59.570+0000 7f09964f5640 1 --2- 172.21.15.167:0/1848030839 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f099013f290 0x7f099013ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:59.574 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:59.570+0000 7f0995cf4640 1 -- 172.21.15.167:0/1848030839 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f099013eeb0 msgr2=0x7f099014cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:59.574 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:59.570+0000 7f0995cf4640 1 --2- 172.21.15.167:0/1848030839 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f099013eeb0 0x7f099014cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:59.574 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:59.570+0000 7f0995cf4640 1 -- 172.21.15.167:0/1848030839 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f099013f290 msgr2=0x7f099013ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:59.574 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:59.570+0000 7f0995cf4640 1 --2- 172.21.15.167:0/1848030839 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f099013f290 0x7f099013ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:59.574 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:59.570+0000 7f0995cf4640 1 -- 172.21.15.167:0/1848030839 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0990139b80 con 0x7f09900b50a0 2024-05-11T08:06:59.574 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:59.571+0000 7f0995cf4640 1 --2- 172.21.15.167:0/1848030839 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f09900b50a0 0x7f099013c4a0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f098c00c870 tx=0x7f098c00cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:59.574 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:59.571+0000 7f09867fc640 1 -- 172.21.15.167:0/1848030839 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f098c018070 con 0x7f09900b50a0 2024-05-11T08:06:59.574 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:59.571+0000 7f09867fc640 1 -- 172.21.15.167:0/1848030839 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f098c00a5e0 con 0x7f09900b50a0 2024-05-11T08:06:59.574 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:59.571+0000 7f0997f58640 1 -- 172.21.15.167:0/1848030839 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0990139e60 con 0x7f09900b50a0 2024-05-11T08:06:59.574 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:59.571+0000 7f09867fc640 1 -- 172.21.15.167:0/1848030839 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f098c00a9d0 con 0x7f09900b50a0 2024-05-11T08:06:59.574 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:59.571+0000 7f0997f58640 1 -- 172.21.15.167:0/1848030839 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f099013a420 con 0x7f09900b50a0 2024-05-11T08:06:59.574 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:59.572+0000 7f0997f58640 1 -- 172.21.15.167:0/1848030839 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0990069590 con 0x7f09900b50a0 2024-05-11T08:06:59.575 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:59.572+0000 7f09867fc640 1 -- 172.21.15.167:0/1848030839 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f098c01b070 con 0x7f09900b50a0 2024-05-11T08:06:59.575 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:59.573+0000 7f09867fc640 1 --2- 172.21.15.167:0/1848030839 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f09740762d0 0x7f0974078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:06:59.575 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:59.573+0000 7f09867fc640 1 -- 172.21.15.167:0/1848030839 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6101+0+0 (secure 0 0 0) 0x7f098c096db0 con 0x7f09900b50a0 2024-05-11T08:06:59.575 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:59.573+0000 7f09954f3640 1 --2- 172.21.15.167:0/1848030839 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f09740762d0 0x7f0974078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:06:59.575 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:59.573+0000 7f09954f3640 1 --2- 172.21.15.167:0/1848030839 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f09740762d0 0x7f0974078790 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f0980034000 tx=0x7f098002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:06:59.577 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:59.577+0000 7f09867fc640 1 -- 172.21.15.167:0/1848030839 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f098c060610 con 0x7f09900b50a0 2024-05-11T08:06:59.797 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:59.796+0000 7f0997f58640 1 -- 172.21.15.167:0/1848030839 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0990165f40 con 0x7f09900b50a0 2024-05-11T08:06:59.797 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:59.797+0000 7f09867fc640 1 -- 172.21.15.167:0/1848030839 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5190 (secure 0 0 0) 0x7f098c05ff10 con 0x7f09900b50a0 2024-05-11T08:06:59.797 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T08:06:59.797 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4604,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2674983673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":2674983673},{"type":"v1","addr":"172.21.15.167:6837","nonce":2674983673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4607,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6834/1018548066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1018548066},{"type":"v1","addr":"172.21.15.204:6834","nonce":1018548066}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4729,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3369722249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3369722249},{"type":"v1","addr":"172.21.15.167:6836","nonce":3369722249}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-11T08:06:47.943939+0000","modified":"2024-05-11T08:06:49.955430+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4750},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4750":{"gid":4750,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.204:6835/469864006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6833","nonce":469864006},{"type":"v1","addr":"172.21.15.204:6835","nonce":469864006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4750,"qdb_cluster":[4750]},"id":2}]} 2024-05-11T08:06:59.798 INFO:teuthology.orchestra.run.smithi167.stderr:dumped fsmap epoch 16 2024-05-11T08:06:59.799 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:59.799+0000 7f0997f58640 1 -- 172.21.15.167:0/1848030839 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f09740762d0 msgr2=0x7f0974078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:59.799 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:59.799+0000 7f0997f58640 1 --2- 172.21.15.167:0/1848030839 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f09740762d0 0x7f0974078790 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f0980034000 tx=0x7f098002e000 comp rx=0 tx=0).stop 2024-05-11T08:06:59.799 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:59.799+0000 7f0997f58640 1 -- 172.21.15.167:0/1848030839 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f09900b50a0 msgr2=0x7f099013c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:06:59.799 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:59.799+0000 7f0997f58640 1 --2- 172.21.15.167:0/1848030839 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f09900b50a0 0x7f099013c4a0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f098c00c870 tx=0x7f098c00cd30 comp rx=0 tx=0).stop 2024-05-11T08:06:59.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:59.799+0000 7f0997f58640 1 -- 172.21.15.167:0/1848030839 shutdown_connections 2024-05-11T08:06:59.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:59.799+0000 7f0997f58640 1 --2- 172.21.15.167:0/1848030839 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f09740762d0 0x7f0974078790 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:59.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:59.799+0000 7f0997f58640 1 --2- 172.21.15.167:0/1848030839 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f099013f290 0x7f099013ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:59.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:59.799+0000 7f0997f58640 1 --2- 172.21.15.167:0/1848030839 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f099013eeb0 0x7f099014cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:59.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:59.799+0000 7f0997f58640 1 --2- 172.21.15.167:0/1848030839 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f09900b50a0 0x7f099013c4a0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:06:59.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:59.799+0000 7f0997f58640 1 -- 172.21.15.167:0/1848030839 >> 172.21.15.167:0/1848030839 conn(0x7f09900cd8f0 msgr2=0x7f099006e400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:06:59.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:59.799+0000 7f0997f58640 1 -- 172.21.15.167:0/1848030839 shutdown_connections 2024-05-11T08:06:59.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:59.799+0000 7f0997f58640 1 -- 172.21.15.167:0/1848030839 wait complete. 2024-05-11T08:06:59.810 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json flush journal 2024-05-11T08:06:59.951 INFO:teuthology.orchestra.run.smithi204.stdout:{"message":"","return_code":0} 2024-05-11T08:06:59.951 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-05-11T08:06:59.952 DEBUG:teuthology.orchestra.run.smithi167:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f6.00000000 symlink 2024-05-11T08:06:59.996 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:59.996+0000 7f6669e23f40 1 Processor -- start 2024-05-11T08:06:59.996 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:59.996+0000 7f6669e23f40 1 -- start start 2024-05-11T08:07:00.001 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:59.997+0000 7f6669e23f40 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5555f7a5a1d0 0x5555f7ad6460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:00.001 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:59.997+0000 7f6669e23f40 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5555f7ad69a0 0x5555f7a402d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:00.001 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:59.997+0000 7f6669e23f40 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x5555f7a5a5b0 0x5555f7a40810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:00.001 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:59.997+0000 7f6669e23f40 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5555f79bcf50 con 0x5555f7ad69a0 2024-05-11T08:07:00.001 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:59.997+0000 7f6669e23f40 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5555f79c0f90 con 0x5555f7a5a1d0 2024-05-11T08:07:00.001 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:59.997+0000 7f6669e23f40 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x5555f7a14410 con 0x5555f7a5a5b0 2024-05-11T08:07:00.001 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:59.997+0000 7f666711c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5555f7ad69a0 0x5555f7a402d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:00.001 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:59.997+0000 7f666711c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5555f7ad69a0 0x5555f7a402d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:43112/0 (socket says 172.21.15.167:43112) 2024-05-11T08:07:00.001 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:59.997+0000 7f666711c640 1 -- 172.21.15.167:0/3555421163 learned_addr learned my addr 172.21.15.167:0/3555421163 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:07:00.001 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:59.997+0000 7f666711c640 1 -- 172.21.15.167:0/3555421163 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x5555f7a5a5b0 msgr2=0x5555f7a40810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:07:00.001 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:59.997+0000 7f666711c640 1 --2- 172.21.15.167:0/3555421163 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x5555f7a5a5b0 0x5555f7a40810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:00.002 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:59.997+0000 7f666711c640 1 -- 172.21.15.167:0/3555421163 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5555f7a5a1d0 msgr2=0x5555f7ad6460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:07:00.002 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:59.997+0000 7f666791d640 1 --2- 172.21.15.167:0/3555421163 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5555f7a5a1d0 0x5555f7ad6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:00.002 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:59.997+0000 7f666711c640 1 --2- 172.21.15.167:0/3555421163 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5555f7a5a1d0 0x5555f7ad6460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:00.002 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:59.997+0000 7f666711c640 1 -- 172.21.15.167:0/3555421163 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555f7aebd30 con 0x5555f7ad69a0 2024-05-11T08:07:00.002 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:59.997+0000 7f666711c640 1 --2- 172.21.15.167:0/3555421163 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5555f7ad69a0 0x5555f7a402d0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f6650009730 tx=0x7f6650005a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27366aeeae739ff6 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:00.002 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:59.997+0000 7f666611a640 1 -- 172.21.15.167:0/3555421163 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66500078f0 con 0x5555f7ad69a0 2024-05-11T08:07:00.002 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:59.997+0000 7f666611a640 1 -- 172.21.15.167:0/3555421163 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6650007a90 con 0x5555f7ad69a0 2024-05-11T08:07:00.002 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:59.997+0000 7f666611a640 1 -- 172.21.15.167:0/3555421163 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6650007e60 con 0x5555f7ad69a0 2024-05-11T08:07:00.002 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:59.998+0000 7f6669e23f40 1 -- 172.21.15.167:0/3555421163 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5555f7ad69a0 msgr2=0x5555f7a402d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:00.002 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:59.998+0000 7f6669e23f40 1 --2- 172.21.15.167:0/3555421163 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5555f7ad69a0 0x5555f7a402d0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f6650009730 tx=0x7f6650005a00 comp rx=0 tx=0).stop 2024-05-11T08:07:00.002 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:59.998+0000 7f6669e23f40 1 -- 172.21.15.167:0/3555421163 shutdown_connections 2024-05-11T08:07:00.002 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:59.998+0000 7f6669e23f40 1 --2- 172.21.15.167:0/3555421163 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x5555f7a5a5b0 0x5555f7a40810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:00.003 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:59.998+0000 7f6669e23f40 1 --2- 172.21.15.167:0/3555421163 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5555f7ad69a0 0x5555f7a402d0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:00.003 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:59.998+0000 7f6669e23f40 1 --2- 172.21.15.167:0/3555421163 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5555f7a5a1d0 0x5555f7ad6460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:00.003 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:59.998+0000 7f6669e23f40 1 -- 172.21.15.167:0/3555421163 >> 172.21.15.167:0/3555421163 conn(0x5555f7a41950 msgr2=0x5555f7a58fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:07:00.003 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:59.998+0000 7f6669e23f40 1 -- 172.21.15.167:0/3555421163 shutdown_connections 2024-05-11T08:07:00.003 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:59.998+0000 7f6669e23f40 1 -- 172.21.15.167:0/3555421163 wait complete. 2024-05-11T08:07:00.003 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:59.998+0000 7f6669e23f40 1 Processor -- start 2024-05-11T08:07:00.003 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:59.998+0000 7f6669e23f40 1 -- start start 2024-05-11T08:07:00.003 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:59.999+0000 7f6669e23f40 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5555f7a5a1d0 0x5555f7aef500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:00.003 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:59.999+0000 7f6669e23f40 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x5555f7a5a5b0 0x5555f7b83b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:00.003 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:59.999+0000 7f6669e23f40 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5555f7ad4d40 0x5555f7a5ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:00.003 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:59.999+0000 7f6669e23f40 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5555f7aeb470 con 0x5555f7a5a1d0 2024-05-11T08:07:00.003 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:59.999+0000 7f6669e23f40 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5555f7a40eb0 con 0x5555f7ad4d40 2024-05-11T08:07:00.003 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:59.999+0000 7f6669e23f40 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x5555f7a420a0 con 0x5555f7a5a5b0 2024-05-11T08:07:00.003 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:59.999+0000 7f666791d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5555f7a5a1d0 0x5555f7aef500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:00.004 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:59.999+0000 7f666791d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5555f7a5a1d0 0x5555f7aef500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:43114/0 (socket says 172.21.15.167:43114) 2024-05-11T08:07:00.004 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:59.999+0000 7f666791d640 1 -- 172.21.15.167:0/2974766215 learned_addr learned my addr 172.21.15.167:0/2974766215 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:07:00.004 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:59.999+0000 7f666791d640 1 -- 172.21.15.167:0/2974766215 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x5555f7a5a5b0 msgr2=0x5555f7b83b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:00.004 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:59.999+0000 7f666791d640 1 --2- 172.21.15.167:0/2974766215 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x5555f7a5a5b0 0x5555f7b83b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:00.004 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:59.999+0000 7f666791d640 1 -- 172.21.15.167:0/2974766215 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5555f7ad4d40 msgr2=0x5555f7a5ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:00.004 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:06:59.999+0000 7f666791d640 1 --2- 172.21.15.167:0/2974766215 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5555f7ad4d40 0x5555f7a5ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:00.004 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.000+0000 7f666791d640 1 -- 172.21.15.167:0/2974766215 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555f7a5bae0 con 0x5555f7a5a1d0 2024-05-11T08:07:00.004 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.000+0000 7f666791d640 1 --2- 172.21.15.167:0/2974766215 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5555f7a5a1d0 0x5555f7aef500 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f666000e5f0 tx=0x7f666000eab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:00.004 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.000+0000 7f6665118640 1 -- 172.21.15.167:0/2974766215 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6660016020 con 0x5555f7a5a1d0 2024-05-11T08:07:00.004 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.000+0000 7f6665118640 1 -- 172.21.15.167:0/2974766215 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6660006260 con 0x5555f7a5a1d0 2024-05-11T08:07:00.004 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.000+0000 7f6669e23f40 1 -- 172.21.15.167:0/2974766215 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5555f7a5bdc0 con 0x5555f7a5a1d0 2024-05-11T08:07:00.004 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.000+0000 7f6665118640 1 -- 172.21.15.167:0/2974766215 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6660005660 con 0x5555f7a5a1d0 2024-05-11T08:07:00.004 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.000+0000 7f6669e23f40 1 -- 172.21.15.167:0/2974766215 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555f7a5b350 con 0x5555f7a5a1d0 2024-05-11T08:07:00.007 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-11T08:07:00.007 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.001+0000 7f6665118640 1 -- 172.21.15.167:0/2974766215 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f6660019020 con 0x5555f7a5a1d0 2024-05-11T08:07:00.007 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.002+0000 7f6665118640 1 --2- 172.21.15.167:0/2974766215 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f6648074ce0 0x7f66480771a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:00.007 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.002+0000 7f6665118640 1 -- 172.21.15.167:0/2974766215 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6101+0+0 (secure 0 0 0) 0x7f66600964e0 con 0x5555f7a5a1d0 2024-05-11T08:07:00.007 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.002+0000 7f666711c640 1 --2- 172.21.15.167:0/2974766215 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f6648074ce0 0x7f66480771a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:00.007 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.002+0000 7f6669e23f40 1 --2- 172.21.15.167:0/2974766215 >> [v2:172.21.15.204:6800/1311831500,v1:172.21.15.204:6801/1311831500] conn(0x5555f7a5b640 0x5555f7acfb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:00.007 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.002+0000 7f6669e23f40 1 -- 172.21.15.167:0/2974766215 --> [v2:172.21.15.204:6800/1311831500,v1:172.21.15.204:6801/1311831500] -- osd_op(unknown.0.0:1 6.2a 6:57fe7535:::100000001f6.00000000:head [getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41) -- 0x5555f7b90150 con 0x5555f7a5b640 2024-05-11T08:07:00.007 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.002+0000 7f666711c640 1 --2- 172.21.15.167:0/2974766215 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f6648074ce0 0x7f66480771a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x5555f7a18460 tx=0x7f6650014000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:00.008 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.002+0000 7f666811e640 1 --2- 172.21.15.167:0/2974766215 >> [v2:172.21.15.204:6800/1311831500,v1:172.21.15.204:6801/1311831500] conn(0x5555f7a5b640 0x5555f7acfb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:00.008 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.003+0000 7f666811e640 1 --2- 172.21.15.167:0/2974766215 >> [v2:172.21.15.204:6800/1311831500,v1:172.21.15.204:6801/1311831500] conn(0x5555f7a5b640 0x5555f7acfb00 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:00.008 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.003+0000 7f666811e640 1 -- 172.21.15.167:0/2974766215 <== osd.4 v2:172.21.15.204:6800/1311831500 1 ==== osd_op_reply(1 100000001f6.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) ==== 164+0+0 (crc 0 0 0) 0x7f66580014f0 con 0x5555f7a5b640 2024-05-11T08:07:00.008 INFO:teuthology.orchestra.run.smithi167.stderr:error getting xattr cephfs_data/100000001f6.00000000/symlink: (61) No data available 2024-05-11T08:07:00.008 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.003+0000 7f6669e23f40 1 -- 172.21.15.167:0/2974766215 >> [v2:172.21.15.204:6800/1311831500,v1:172.21.15.204:6801/1311831500] conn(0x5555f7a5b640 msgr2=0x5555f7acfb00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:00.008 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.003+0000 7f6669e23f40 1 --2- 172.21.15.167:0/2974766215 >> [v2:172.21.15.204:6800/1311831500,v1:172.21.15.204:6801/1311831500] conn(0x5555f7a5b640 0x5555f7acfb00 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:00.008 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.003+0000 7f6669e23f40 1 -- 172.21.15.167:0/2974766215 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f6648074ce0 msgr2=0x7f66480771a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:00.008 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.003+0000 7f6669e23f40 1 --2- 172.21.15.167:0/2974766215 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f6648074ce0 0x7f66480771a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x5555f7a18460 tx=0x7f6650014000 comp rx=0 tx=0).stop 2024-05-11T08:07:00.008 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.003+0000 7f6669e23f40 1 -- 172.21.15.167:0/2974766215 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5555f7a5a1d0 msgr2=0x5555f7aef500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:00.008 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.003+0000 7f6669e23f40 1 --2- 172.21.15.167:0/2974766215 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5555f7a5a1d0 0x5555f7aef500 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f666000e5f0 tx=0x7f666000eab0 comp rx=0 tx=0).stop 2024-05-11T08:07:00.008 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.003+0000 7f666811e640 1 -- 172.21.15.167:0/2974766215 reap_dead start 2024-05-11T08:07:00.008 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.003+0000 7f6669e23f40 1 -- 172.21.15.167:0/2974766215 shutdown_connections 2024-05-11T08:07:00.008 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.003+0000 7f6669e23f40 1 -- 172.21.15.167:0/2974766215 >> 172.21.15.167:0/2974766215 conn(0x5555f7a41950 msgr2=0x5555f7a58fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:07:00.009 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.003+0000 7f6669e23f40 1 -- 172.21.15.167:0/2974766215 shutdown_connections 2024-05-11T08:07:00.009 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.003+0000 7f6669e23f40 1 -- 172.21.15.167:0/2974766215 wait complete. 2024-05-11T08:07:00.009 ERROR:tasks.cephfs.filesystem:Command failed on smithi167 with status 1: 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f6.00000000 symlink' 2024-05-11T08:07:00.009 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_symlink_recovery true 2024-05-11T08:07:00.151 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.151+0000 7f649fd46640 1 Processor -- start 2024-05-11T08:07:00.151 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.151+0000 7f649fd46640 1 -- start start 2024-05-11T08:07:00.153 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.152+0000 7f649fd46640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f649815f910 0x7f649806c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:00.153 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.152+0000 7f649fd46640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f649806cc30 0x7f649815fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:00.153 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.152+0000 7f649fd46640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f6498160230 0x7f64980d3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:00.153 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.152+0000 7f649fd46640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f6498156820 con 0x7f649815f910 2024-05-11T08:07:00.153 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.152+0000 7f649fd46640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f649806dcc0 con 0x7f6498160230 2024-05-11T08:07:00.153 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.152+0000 7f649fd46640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f649806d140 con 0x7f649806cc30 2024-05-11T08:07:00.153 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.152+0000 7f649dae2640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f649815f910 0x7f649806c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:00.153 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.152+0000 7f649dae2640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f649815f910 0x7f649806c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:43128/0 (socket says 172.21.15.167:43128) 2024-05-11T08:07:00.153 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.152+0000 7f649dae2640 1 -- 172.21.15.167:0/986682573 learned_addr learned my addr 172.21.15.167:0/986682573 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:07:00.153 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.152+0000 7f649dae2640 1 -- 172.21.15.167:0/986682573 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f649806cc30 msgr2=0x7f649815fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:07:00.153 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.152+0000 7f649e2e3640 1 --2- 172.21.15.167:0/986682573 >> v2:172.21.15.204:3300/0 conn(0x7f6498160230 0x7f64980d3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:00.153 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.152+0000 7f649d2e1640 1 --2- 172.21.15.167:0/986682573 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f649806cc30 0x7f649815fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:00.153 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.152+0000 7f649dae2640 1 --2- 172.21.15.167:0/986682573 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f649806cc30 0x7f649815fcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:00.154 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.152+0000 7f649dae2640 1 -- 172.21.15.167:0/986682573 >> v2:172.21.15.204:3300/0 conn(0x7f6498160230 msgr2=0x7f64980d3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:07:00.154 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.152+0000 7f649dae2640 1 --2- 172.21.15.167:0/986682573 >> v2:172.21.15.204:3300/0 conn(0x7f6498160230 0x7f64980d3090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:00.154 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.152+0000 7f649dae2640 1 -- 172.21.15.167:0/986682573 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f649806d420 con 0x7f649815f910 2024-05-11T08:07:00.155 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.153+0000 7f649dae2640 1 --2- 172.21.15.167:0/986682573 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f649815f910 0x7f649806c6f0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f64980d5010 tx=0x7f649402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73dba7c8e45c1a30 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:00.155 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.153+0000 7f649cae0640 1 -- 172.21.15.167:0/986682573 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6494037070 con 0x7f649815f910 2024-05-11T08:07:00.155 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.153+0000 7f649cae0640 1 -- 172.21.15.167:0/986682573 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6494032440 con 0x7f649815f910 2024-05-11T08:07:00.155 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.153+0000 7f649cae0640 1 -- 172.21.15.167:0/986682573 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6494032810 con 0x7f649815f910 2024-05-11T08:07:00.156 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.154+0000 7f649fd46640 1 -- 172.21.15.167:0/986682573 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f649815f910 msgr2=0x7f649806c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:00.156 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.154+0000 7f649fd46640 1 --2- 172.21.15.167:0/986682573 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f649815f910 0x7f649806c6f0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f64980d5010 tx=0x7f649402a1e0 comp rx=0 tx=0).stop 2024-05-11T08:07:00.156 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.154+0000 7f649fd46640 1 -- 172.21.15.167:0/986682573 shutdown_connections 2024-05-11T08:07:00.156 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.154+0000 7f649fd46640 1 --2- 172.21.15.167:0/986682573 >> v2:172.21.15.204:3300/0 conn(0x7f6498160230 0x7f64980d3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:00.156 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.154+0000 7f649fd46640 1 --2- 172.21.15.167:0/986682573 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f649806cc30 0x7f649815fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:00.156 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.154+0000 7f649fd46640 1 --2- 172.21.15.167:0/986682573 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f649815f910 0x7f649806c6f0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:00.156 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.154+0000 7f649fd46640 1 -- 172.21.15.167:0/986682573 >> 172.21.15.167:0/986682573 conn(0x7f64980d51e0 msgr2=0x7f64980d55f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:07:00.156 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.154+0000 7f649fd46640 1 -- 172.21.15.167:0/986682573 shutdown_connections 2024-05-11T08:07:00.156 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.154+0000 7f649fd46640 1 -- 172.21.15.167:0/986682573 wait complete. 2024-05-11T08:07:00.156 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.155+0000 7f649fd46640 1 Processor -- start 2024-05-11T08:07:00.156 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.155+0000 7f649fd46640 1 -- start start 2024-05-11T08:07:00.159 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.156+0000 7f649fd46640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f649806cc30 0x7f6498157820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:00.159 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.156+0000 7f649fd46640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f649815f910 0x7f64981f4b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:00.159 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.156+0000 7f649fd46640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6498160230 0x7f64981f50b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:00.159 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.156+0000 7f649fd46640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f64981606b0 con 0x7f649815f910 2024-05-11T08:07:00.159 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.156+0000 7f649fd46640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f649806d9e0 con 0x7f649806cc30 2024-05-11T08:07:00.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.156+0000 7f649fd46640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f64980d0060 con 0x7f6498160230 2024-05-11T08:07:00.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.156+0000 7f649d2e1640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f649815f910 0x7f64981f4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:00.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.156+0000 7f649d2e1640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f649815f910 0x7f64981f4b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:43132/0 (socket says 172.21.15.167:43132) 2024-05-11T08:07:00.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.156+0000 7f649d2e1640 1 -- 172.21.15.167:0/1546798276 learned_addr learned my addr 172.21.15.167:0/1546798276 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:07:00.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.156+0000 7f649d2e1640 1 -- 172.21.15.167:0/1546798276 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6498160230 msgr2=0x7f64981f50b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:00.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.156+0000 7f649e2e3640 1 --2- 172.21.15.167:0/1546798276 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6498160230 0x7f64981f50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:00.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.156+0000 7f649dae2640 1 --2- 172.21.15.167:0/1546798276 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f649806cc30 0x7f6498157820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:00.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.156+0000 7f649d2e1640 1 --2- 172.21.15.167:0/1546798276 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6498160230 0x7f64981f50b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:00.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.156+0000 7f649d2e1640 1 -- 172.21.15.167:0/1546798276 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f649806cc30 msgr2=0x7f6498157820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:00.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.156+0000 7f649d2e1640 1 --2- 172.21.15.167:0/1546798276 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f649806cc30 0x7f6498157820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:00.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.156+0000 7f649d2e1640 1 -- 172.21.15.167:0/1546798276 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64980cf290 con 0x7f649815f910 2024-05-11T08:07:00.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.156+0000 7f649e2e3640 1 --2- 172.21.15.167:0/1546798276 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6498160230 0x7f64981f50b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T08:07:00.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.156+0000 7f649dae2640 1 --2- 172.21.15.167:0/1546798276 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f649806cc30 0x7f6498157820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T08:07:00.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.156+0000 7f649d2e1640 1 --2- 172.21.15.167:0/1546798276 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f649815f910 0x7f64981f4b70 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f648800c540 tx=0x7f648800ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:00.161 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.156+0000 7f648e7fc640 1 -- 172.21.15.167:0/1546798276 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6488015070 con 0x7f649815f910 2024-05-11T08:07:00.161 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.157+0000 7f648e7fc640 1 -- 172.21.15.167:0/1546798276 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6488011800 con 0x7f649815f910 2024-05-11T08:07:00.161 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.157+0000 7f649fd46640 1 -- 172.21.15.167:0/1546798276 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6498156a60 con 0x7f649815f910 2024-05-11T08:07:00.161 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.157+0000 7f648e7fc640 1 -- 172.21.15.167:0/1546798276 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6488011bb0 con 0x7f649815f910 2024-05-11T08:07:00.161 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.157+0000 7f649fd46640 1 -- 172.21.15.167:0/1546798276 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6498157020 con 0x7f649815f910 2024-05-11T08:07:00.161 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.157+0000 7f649fd46640 1 -- 172.21.15.167:0/1546798276 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6498069520 con 0x7f649815f910 2024-05-11T08:07:00.161 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.158+0000 7f648e7fc640 1 -- 172.21.15.167:0/1546798276 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f6488018070 con 0x7f649815f910 2024-05-11T08:07:00.161 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.159+0000 7f648e7fc640 1 --2- 172.21.15.167:0/1546798276 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f647c076300 0x7f647c0787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:00.161 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.159+0000 7f648e7fc640 1 -- 172.21.15.167:0/1546798276 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6101+0+0 (secure 0 0 0) 0x7f6488097490 con 0x7f649815f910 2024-05-11T08:07:00.161 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.159+0000 7f649dae2640 1 --2- 172.21.15.167:0/1546798276 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f647c076300 0x7f647c0787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:00.161 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.159+0000 7f649dae2640 1 --2- 172.21.15.167:0/1546798276 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f647c076300 0x7f647c0787c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f6494034000 tx=0x7f649402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:00.163 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.163+0000 7f648e7fc640 1 -- 172.21.15.167:0/1546798276 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f6488060cf0 con 0x7f649815f910 2024-05-11T08:07:00.342 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.342+0000 7f649fd46640 1 -- 172.21.15.167:0/1546798276 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command([{prefix=config set, name=mds_symlink_recovery}] v 0) -- 0x7f64980ba800 con 0x7f649815f910 2024-05-11T08:07:00.345 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.345+0000 7f648e7fc640 1 -- 172.21.15.167:0/1546798276 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_symlink_recovery}]=0 v12)=0 v12) ==== 121+0+0 (secure 0 0 0) 0x7f64880605f0 con 0x7f649815f910 2024-05-11T08:07:00.348 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.348+0000 7f649fd46640 1 -- 172.21.15.167:0/1546798276 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f647c076300 msgr2=0x7f647c0787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:00.348 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.348+0000 7f649fd46640 1 --2- 172.21.15.167:0/1546798276 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f647c076300 0x7f647c0787c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f6494034000 tx=0x7f649402e000 comp rx=0 tx=0).stop 2024-05-11T08:07:00.349 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.348+0000 7f649fd46640 1 -- 172.21.15.167:0/1546798276 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f649815f910 msgr2=0x7f64981f4b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:00.349 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.348+0000 7f649fd46640 1 --2- 172.21.15.167:0/1546798276 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f649815f910 0x7f64981f4b70 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f648800c540 tx=0x7f648800ca00 comp rx=0 tx=0).stop 2024-05-11T08:07:00.349 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.348+0000 7f649fd46640 1 -- 172.21.15.167:0/1546798276 shutdown_connections 2024-05-11T08:07:00.349 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.348+0000 7f649fd46640 1 --2- 172.21.15.167:0/1546798276 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f647c076300 0x7f647c0787c0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:00.349 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.348+0000 7f649fd46640 1 --2- 172.21.15.167:0/1546798276 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6498160230 0x7f64981f50b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:00.349 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.348+0000 7f649fd46640 1 --2- 172.21.15.167:0/1546798276 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f649815f910 0x7f64981f4b70 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:00.349 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.348+0000 7f649fd46640 1 --2- 172.21.15.167:0/1546798276 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f649806cc30 0x7f6498157820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:00.349 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.348+0000 7f649fd46640 1 -- 172.21.15.167:0/1546798276 >> 172.21.15.167:0/1546798276 conn(0x7f64980d51e0 msgr2=0x7f64980d0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:07:00.349 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.348+0000 7f649fd46640 1 -- 172.21.15.167:0/1546798276 shutdown_connections 2024-05-11T08:07:00.349 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.348+0000 7f649fd46640 1 -- 172.21.15.167:0/1546798276 wait complete. 2024-05-11T08:07:00.359 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T08:07:00.359 DEBUG:teuthology.orchestra.run.smithi167:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir sym_dir) 2024-05-11T08:07:00.377 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T08:07:00.377 DEBUG:teuthology.orchestra.run.smithi167:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch sym_dir/file1) 2024-05-11T08:07:00.438 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T08:07:00.438 DEBUG:teuthology.orchestra.run.smithi167:> (cd /home/ubuntu/cephtest/mnt.0 && exec ln -s ./file1 sym_dir/symlink_file1) 2024-05-11T08:07:00.494 DEBUG:teuthology.orchestra.run.smithi167:> stdin-killer -- python3 -c ' 2024-05-11T08:07:00.495 DEBUG:teuthology.orchestra.run.smithi167:> import os 2024-05-11T08:07:00.495 DEBUG:teuthology.orchestra.run.smithi167:> import stat 2024-05-11T08:07:00.495 DEBUG:teuthology.orchestra.run.smithi167:> 2024-05-11T08:07:00.495 DEBUG:teuthology.orchestra.run.smithi167:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/sym_dir/symlink_file1").st_ino) 2024-05-11T08:07:00.495 DEBUG:teuthology.orchestra.run.smithi167:> ' 2024-05-11T08:07:00.600 INFO:teuthology.orchestra.run.smithi167.stdout:1099511628281 2024-05-11T08:07:00.603 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T08:07:00.609 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-11T08:07:00.748 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.748+0000 7f748482a640 1 Processor -- start 2024-05-11T08:07:00.749 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.748+0000 7f748482a640 1 -- start start 2024-05-11T08:07:00.753 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.749+0000 7f748482a640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f74801598d0 0x7f74800d2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:00.753 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.749+0000 7f748482a640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f74800d0640 0x7f74800d0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:00.753 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.749+0000 7f748482a640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74800d1190 con 0x7f74800cafb0 2024-05-11T08:07:00.753 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.749+0000 7f748482a640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f748006d500 con 0x7f74800d0640 2024-05-11T08:07:00.753 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.749+0000 7f748482a640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f748006d7e0 con 0x7f74801598d0 2024-05-11T08:07:00.753 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.749+0000 7f747ed9d640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f74800cafb0 0x7f748006dfe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:48364/0 (socket says 172.21.15.167:48364) 2024-05-11T08:07:00.753 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.749+0000 7f747ed9d640 1 -- 172.21.15.167:0/3810183216 learned_addr learned my addr 172.21.15.167:0/3810183216 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T08:07:00.754 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.750+0000 7f747dd9b640 1 -- 172.21.15.167:0/3810183216 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4001236657 0 0) 0x7f74800d1190 con 0x7f74800cafb0 2024-05-11T08:07:00.754 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.750+0000 7f747dd9b640 1 -- 172.21.15.167:0/3810183216 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f746c003520 con 0x7f74800cafb0 2024-05-11T08:07:00.754 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.750+0000 7f747e59c640 1 --2- 172.21.15.167:0/3810183216 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f74801598d0 0x7f74800d2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:00.754 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.750+0000 7f7475d9b640 1 --2- 172.21.15.167:0/3810183216 >> v2:172.21.15.204:3300/0 conn(0x7f74800d0640 0x7f74800d0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:00.754 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.750+0000 7f747dd9b640 1 -- 172.21.15.167:0/3810183216 <== mon.0 v1:172.21.15.167:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 903314585 0 0) 0x7f746c003520 con 0x7f74800cafb0 2024-05-11T08:07:00.754 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.750+0000 7f747dd9b640 1 -- 172.21.15.167:0/3810183216 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f74801598d0 msgr2=0x7f74800d2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:07:00.754 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.750+0000 7f747dd9b640 1 --2- 172.21.15.167:0/3810183216 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f74801598d0 0x7f74800d2cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:00.754 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.750+0000 7f747dd9b640 1 -- 172.21.15.167:0/3810183216 >> v2:172.21.15.204:3300/0 conn(0x7f74800d0640 msgr2=0x7f74800d0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:07:00.754 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.750+0000 7f747dd9b640 1 --2- 172.21.15.167:0/3810183216 >> v2:172.21.15.204:3300/0 conn(0x7f74800d0640 0x7f74800d0aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:00.754 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.750+0000 7f747dd9b640 1 -- 172.21.15.167:0/3810183216 --> v1:172.21.15.167:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f748006dac0 con 0x7f74800cafb0 2024-05-11T08:07:00.754 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.750+0000 7f747dd9b640 1 -- 172.21.15.167:0/3810183216 <== mon.0 v1:172.21.15.167:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3602277846 0 0) 0x7f7478001a70 con 0x7f74800cafb0 2024-05-11T08:07:00.755 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.750+0000 7f747dd9b640 1 -- 172.21.15.167:0/3810183216 >> v1:172.21.15.167:6789/0 conn(0x7f74800cafb0 legacy=0x7f748006dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:00.755 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.750+0000 7f747dd9b640 1 --2- 172.21.15.167:0/3810183216 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f746c003a00 0x7f746c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:00.755 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.750+0000 7f747dd9b640 1 --2- 172.21.15.167:0/3810183216 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f74801598d0 0x7f746c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:00.755 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.750+0000 7f747dd9b640 1 --2- 172.21.15.167:0/3810183216 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f746c004d40 0x7f746c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:00.755 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.750+0000 7f747dd9b640 1 -- 172.21.15.167:0/3810183216 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f748006d500 con 0x7f746c003a00 2024-05-11T08:07:00.755 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.750+0000 7f747dd9b640 1 -- 172.21.15.167:0/3810183216 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f748006d7e0 con 0x7f746c004d40 2024-05-11T08:07:00.755 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.750+0000 7f747dd9b640 1 -- 172.21.15.167:0/3810183216 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f746c00bb20 con 0x7f74801598d0 2024-05-11T08:07:00.755 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.750+0000 7f747e59c640 1 --2- 172.21.15.167:0/3810183216 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f746c003a00 0x7f746c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:00.755 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.750+0000 7f7475d9b640 1 --2- 172.21.15.167:0/3810183216 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f74801598d0 0x7f746c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:00.755 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.750+0000 7f747ed9d640 1 --2- 172.21.15.167:0/3810183216 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f746c004d40 0x7f746c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:00.755 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.750+0000 7f747e59c640 1 -- 172.21.15.167:0/3810183216 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f74801598d0 msgr2=0x7f746c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:07:00.755 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.750+0000 7f747e59c640 1 --2- 172.21.15.167:0/3810183216 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f74801598d0 0x7f746c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:00.755 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.750+0000 7f747e59c640 1 -- 172.21.15.167:0/3810183216 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f746c004d40 msgr2=0x7f746c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:07:00.755 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.750+0000 7f747e59c640 1 --2- 172.21.15.167:0/3810183216 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f746c004d40 0x7f746c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:00.756 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.750+0000 7f747e59c640 1 -- 172.21.15.167:0/3810183216 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f746c003520 con 0x7f746c003a00 2024-05-11T08:07:00.756 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.750+0000 7f7475d9b640 1 --2- 172.21.15.167:0/3810183216 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f74801598d0 0x7f746c004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T08:07:00.756 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.751+0000 7f747e59c640 1 --2- 172.21.15.167:0/3810183216 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f746c003a00 0x7f746c003e20 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f747000ba00 tx=0x7f746c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb70332e38a2b676 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:00.756 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.751+0000 7f747dd9b640 1 -- 172.21.15.167:0/3810183216 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7470030070 con 0x7f746c003a00 2024-05-11T08:07:00.756 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.751+0000 7f747dd9b640 1 -- 172.21.15.167:0/3810183216 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f747002adb0 con 0x7f746c003a00 2024-05-11T08:07:00.756 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.751+0000 7f747dd9b640 1 -- 172.21.15.167:0/3810183216 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f747000b600 con 0x7f746c003a00 2024-05-11T08:07:00.756 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.752+0000 7f748482a640 1 -- 172.21.15.167:0/3810183216 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f746c003a00 msgr2=0x7f746c003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:00.756 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.752+0000 7f748482a640 1 --2- 172.21.15.167:0/3810183216 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f746c003a00 0x7f746c003e20 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f747000ba00 tx=0x7f746c005830 comp rx=0 tx=0).stop 2024-05-11T08:07:00.756 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.752+0000 7f747ed9d640 1 -- 172.21.15.167:0/3810183216 reap_dead start 2024-05-11T08:07:00.756 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.752+0000 7f748482a640 1 -- 172.21.15.167:0/3810183216 shutdown_connections 2024-05-11T08:07:00.756 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.752+0000 7f748482a640 1 -- 172.21.15.167:0/3810183216 >> 172.21.15.167:0/3810183216 conn(0x7f74800d1ba0 msgr2=0x7f74800d1fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:07:00.757 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.752+0000 7f748482a640 1 -- 172.21.15.167:0/3810183216 shutdown_connections 2024-05-11T08:07:00.757 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.752+0000 7f748482a640 1 -- 172.21.15.167:0/3810183216 wait complete. 2024-05-11T08:07:00.757 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.753+0000 7f748482a640 1 Processor -- start 2024-05-11T08:07:00.757 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.753+0000 7f748482a640 1 -- start start 2024-05-11T08:07:00.759 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.754+0000 7f748482a640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f746c003a00 0x7f74801f5f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:00.759 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.754+0000 7f748482a640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7480157140 0x7f74800babd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:00.759 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.754+0000 7f748482a640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f74800d3130 0x7f74800d35b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:00.760 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.754+0000 7f748482a640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f748006d1b0 con 0x7f74800d3130 2024-05-11T08:07:00.760 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.754+0000 7f748482a640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7480159cb0 con 0x7f7480157140 2024-05-11T08:07:00.760 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.754+0000 7f748482a640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f74800d2cf0 con 0x7f746c003a00 2024-05-11T08:07:00.760 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.754+0000 7f747ed9d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f74800d3130 0x7f74800d35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:00.760 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.754+0000 7f747ed9d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f74800d3130 0x7f74800d35b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:43152/0 (socket says 172.21.15.167:43152) 2024-05-11T08:07:00.760 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.754+0000 7f747ed9d640 1 -- 172.21.15.167:0/2777048073 learned_addr learned my addr 172.21.15.167:0/2777048073 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:07:00.760 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.754+0000 7f7475d9b640 1 --2- 172.21.15.167:0/2777048073 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7480157140 0x7f74800babd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:00.760 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.754+0000 7f747ed9d640 1 -- 172.21.15.167:0/2777048073 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f746c003a00 msgr2=0x7f74801f5f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:00.760 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.754+0000 7f747e59c640 1 --2- 172.21.15.167:0/2777048073 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f746c003a00 0x7f74801f5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:00.760 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.754+0000 7f747ed9d640 1 --2- 172.21.15.167:0/2777048073 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f746c003a00 0x7f74801f5f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:00.760 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.754+0000 7f747ed9d640 1 -- 172.21.15.167:0/2777048073 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7480157140 msgr2=0x7f74800babd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:00.760 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.754+0000 7f747ed9d640 1 --2- 172.21.15.167:0/2777048073 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7480157140 0x7f74800babd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:00.760 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.754+0000 7f747ed9d640 1 -- 172.21.15.167:0/2777048073 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74800b7f30 con 0x7f74800d3130 2024-05-11T08:07:00.760 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.754+0000 7f7475d9b640 1 --2- 172.21.15.167:0/2777048073 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7480157140 0x7f74800babd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T08:07:00.760 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.754+0000 7f747e59c640 1 --2- 172.21.15.167:0/2777048073 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f746c003a00 0x7f74801f5f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T08:07:00.761 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.755+0000 7f747ed9d640 1 --2- 172.21.15.167:0/2777048073 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f74800d3130 0x7f74800d35b0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f748008d610 tx=0x7f747800e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:00.761 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.755+0000 7f74777fe640 1 -- 172.21.15.167:0/2777048073 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f747800eb90 con 0x7f74800d3130 2024-05-11T08:07:00.761 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.755+0000 7f74777fe640 1 -- 172.21.15.167:0/2777048073 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f747800ed50 con 0x7f74800d3130 2024-05-11T08:07:00.761 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.755+0000 7f748482a640 1 -- 172.21.15.167:0/2777048073 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74800b8210 con 0x7f74800d3130 2024-05-11T08:07:00.761 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.755+0000 7f74777fe640 1 -- 172.21.15.167:0/2777048073 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7478014670 con 0x7f74800d3130 2024-05-11T08:07:00.761 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.755+0000 7f748482a640 1 -- 172.21.15.167:0/2777048073 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74800b87d0 con 0x7f74800d3130 2024-05-11T08:07:00.761 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.756+0000 7f748482a640 1 -- 172.21.15.167:0/2777048073 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7480069590 con 0x7f74800d3130 2024-05-11T08:07:00.761 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.756+0000 7f74777fe640 1 -- 172.21.15.167:0/2777048073 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f7478014810 con 0x7f74800d3130 2024-05-11T08:07:00.761 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.757+0000 7f74777fe640 1 --2- 172.21.15.167:0/2777048073 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f7460076300 0x7f74600787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:00.761 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.757+0000 7f74777fe640 1 -- 172.21.15.167:0/2777048073 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6101+0+0 (secure 0 0 0) 0x7f74780974c0 con 0x7f74800d3130 2024-05-11T08:07:00.761 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.757+0000 7f747e59c640 1 --2- 172.21.15.167:0/2777048073 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f7460076300 0x7f74600787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:00.761 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.757+0000 7f747e59c640 1 --2- 172.21.15.167:0/2777048073 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f7460076300 0x7f74600787c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f746c0057e0 tx=0x7f7470030000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:00.762 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.761+0000 7f74777fe640 1 -- 172.21.15.167:0/2777048073 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f7478060d20 con 0x7f74800d3130 2024-05-11T08:07:00.979 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.979+0000 7f748482a640 1 -- 172.21.15.167:0/2777048073 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f74800cf220 con 0x7f74800d3130 2024-05-11T08:07:00.979 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.979+0000 7f74777fe640 1 -- 172.21.15.167:0/2777048073 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5190 (secure 0 0 0) 0x7f7478060620 con 0x7f74800d3130 2024-05-11T08:07:00.979 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T08:07:00.979 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4604,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2674983673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":2674983673},{"type":"v1","addr":"172.21.15.167:6837","nonce":2674983673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4607,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6834/1018548066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1018548066},{"type":"v1","addr":"172.21.15.204:6834","nonce":1018548066}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4729,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3369722249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3369722249},{"type":"v1","addr":"172.21.15.167:6836","nonce":3369722249}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-11T08:06:47.943939+0000","modified":"2024-05-11T08:06:49.955430+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4750},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4750":{"gid":4750,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.204:6835/469864006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6833","nonce":469864006},{"type":"v1","addr":"172.21.15.204:6835","nonce":469864006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4750,"qdb_cluster":[4750]},"id":2}]} 2024-05-11T08:07:00.980 INFO:teuthology.orchestra.run.smithi167.stderr:dumped fsmap epoch 16 2024-05-11T08:07:00.982 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.981+0000 7f748482a640 1 -- 172.21.15.167:0/2777048073 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f7460076300 msgr2=0x7f74600787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:00.982 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.981+0000 7f748482a640 1 --2- 172.21.15.167:0/2777048073 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f7460076300 0x7f74600787c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f746c0057e0 tx=0x7f7470030000 comp rx=0 tx=0).stop 2024-05-11T08:07:00.982 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.981+0000 7f748482a640 1 -- 172.21.15.167:0/2777048073 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f74800d3130 msgr2=0x7f74800d35b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:00.982 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.981+0000 7f748482a640 1 --2- 172.21.15.167:0/2777048073 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f74800d3130 0x7f74800d35b0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f748008d610 tx=0x7f747800e530 comp rx=0 tx=0).stop 2024-05-11T08:07:00.982 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.981+0000 7f748482a640 1 -- 172.21.15.167:0/2777048073 shutdown_connections 2024-05-11T08:07:00.982 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.981+0000 7f748482a640 1 --2- 172.21.15.167:0/2777048073 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f7460076300 0x7f74600787c0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:00.982 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.981+0000 7f748482a640 1 --2- 172.21.15.167:0/2777048073 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f74800d3130 0x7f74800d35b0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:00.982 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.981+0000 7f748482a640 1 --2- 172.21.15.167:0/2777048073 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7480157140 0x7f74800babd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:00.982 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.981+0000 7f748482a640 1 --2- 172.21.15.167:0/2777048073 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f746c003a00 0x7f74801f5f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:00.982 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.981+0000 7f748482a640 1 -- 172.21.15.167:0/2777048073 >> 172.21.15.167:0/2777048073 conn(0x7f74800d1ba0 msgr2=0x7f7480160b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:07:00.982 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.981+0000 7f748482a640 1 -- 172.21.15.167:0/2777048073 shutdown_connections 2024-05-11T08:07:00.982 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:00.981+0000 7f748482a640 1 -- 172.21.15.167:0/2777048073 wait complete. 2024-05-11T08:07:00.993 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json flush journal 2024-05-11T08:07:01.130 INFO:teuthology.orchestra.run.smithi204.stdout:{"message":"","return_code":0} 2024-05-11T08:07:01.130 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-05-11T08:07:01.130 DEBUG:teuthology.orchestra.run.smithi167:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f9.00000000 symlink 2024-05-11T08:07:01.177 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.174+0000 7f15cc51ff40 1 Processor -- start 2024-05-11T08:07:01.177 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.174+0000 7f15cc51ff40 1 -- start start 2024-05-11T08:07:01.177 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.175+0000 7f15cc51ff40 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x558ac9ddffc0 0x558ac9ea1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:01.177 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.175+0000 7f15cc51ff40 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558ac9e2a260 0x558ac9e26080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:01.177 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.175+0000 7f15cc51ff40 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ac9eb8760 0x558ac9e265c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:01.177 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.175+0000 7f15cc51ff40 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x558ac9d88f50 con 0x558ac9e2a260 2024-05-11T08:07:01.177 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.175+0000 7f15cc51ff40 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x558ac9d8cf90 con 0x558ac9eb8760 2024-05-11T08:07:01.177 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.175+0000 7f15cc51ff40 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x558ac9de03f0 con 0x558ac9ddffc0 2024-05-11T08:07:01.177 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.175+0000 7f15c3fff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558ac9e2a260 0x558ac9e26080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:01.177 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.175+0000 7f15c3fff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558ac9e2a260 0x558ac9e26080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:43174/0 (socket says 172.21.15.167:43174) 2024-05-11T08:07:01.177 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.175+0000 7f15c3fff640 1 -- 172.21.15.167:0/1787630034 learned_addr learned my addr 172.21.15.167:0/1787630034 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:07:01.177 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.175+0000 7f15c3fff640 1 -- 172.21.15.167:0/1787630034 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x558ac9ddffc0 msgr2=0x558ac9ea1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:07:01.177 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.175+0000 7f15c3fff640 1 --2- 172.21.15.167:0/1787630034 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x558ac9ddffc0 0x558ac9ea1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:01.177 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.175+0000 7f15c3fff640 1 -- 172.21.15.167:0/1787630034 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ac9eb8760 msgr2=0x558ac9e265c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:07:01.178 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.175+0000 7f15c9eef640 1 --2- 172.21.15.167:0/1787630034 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ac9eb8760 0x558ac9e265c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:01.178 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.175+0000 7f15c3fff640 1 --2- 172.21.15.167:0/1787630034 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ac9eb8760 0x558ac9e265c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:01.178 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.175+0000 7f15c3fff640 1 -- 172.21.15.167:0/1787630034 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ac9ea2780 con 0x558ac9e2a260 2024-05-11T08:07:01.178 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.175+0000 7f15c9eef640 1 --2- 172.21.15.167:0/1787630034 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ac9eb8760 0x558ac9e265c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T08:07:01.178 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.175+0000 7f15c3fff640 1 --2- 172.21.15.167:0/1787630034 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558ac9e2a260 0x558ac9e26080 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f15b4025e20 tx=0x7f15b4051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86cf12826e56770c server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:01.178 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.176+0000 7f15c37fe640 1 -- 172.21.15.167:0/1787630034 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15b4068070 con 0x558ac9e2a260 2024-05-11T08:07:01.178 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.176+0000 7f15c37fe640 1 -- 172.21.15.167:0/1787630034 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15b40517f0 con 0x558ac9e2a260 2024-05-11T08:07:01.178 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.176+0000 7f15c37fe640 1 -- 172.21.15.167:0/1787630034 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15b4055070 con 0x558ac9e2a260 2024-05-11T08:07:01.178 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.176+0000 7f15cc51ff40 1 -- 172.21.15.167:0/1787630034 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558ac9e2a260 msgr2=0x558ac9e26080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:01.178 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.176+0000 7f15cc51ff40 1 --2- 172.21.15.167:0/1787630034 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558ac9e2a260 0x558ac9e26080 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f15b4025e20 tx=0x7f15b4051000 comp rx=0 tx=0).stop 2024-05-11T08:07:01.178 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.176+0000 7f15cc51ff40 1 -- 172.21.15.167:0/1787630034 shutdown_connections 2024-05-11T08:07:01.178 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.176+0000 7f15cc51ff40 1 --2- 172.21.15.167:0/1787630034 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ac9eb8760 0x558ac9e265c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:01.178 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.176+0000 7f15cc51ff40 1 --2- 172.21.15.167:0/1787630034 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558ac9e2a260 0x558ac9e26080 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:01.179 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.176+0000 7f15cc51ff40 1 --2- 172.21.15.167:0/1787630034 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x558ac9ddffc0 0x558ac9ea1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:01.179 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.176+0000 7f15cc51ff40 1 -- 172.21.15.167:0/1787630034 >> 172.21.15.167:0/1787630034 conn(0x558ac9e0d950 msgr2=0x558ac9e24fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:07:01.179 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.176+0000 7f15cc51ff40 1 -- 172.21.15.167:0/1787630034 shutdown_connections 2024-05-11T08:07:01.179 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.176+0000 7f15cc51ff40 1 -- 172.21.15.167:0/1787630034 wait complete. 2024-05-11T08:07:01.179 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.177+0000 7f15cc51ff40 1 Processor -- start 2024-05-11T08:07:01.179 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.177+0000 7f15cc51ff40 1 -- start start 2024-05-11T08:07:01.182 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.177+0000 7f15cc51ff40 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ac9ddffc0 0x558ac9eb8b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:01.182 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.177+0000 7f15cc51ff40 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558ac9e2a260 0x558ac9f4e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:01.183 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.177+0000 7f15cc51ff40 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x558ac9eb8760 0x558ac9e25e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:01.183 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.177+0000 7f15cc51ff40 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x558ac9e0d260 con 0x558ac9e2a260 2024-05-11T08:07:01.183 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.177+0000 7f15cc51ff40 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x558ac9e0cf80 con 0x558ac9ddffc0 2024-05-11T08:07:01.183 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.177+0000 7f15cc51ff40 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x558ac9eb6c10 con 0x558ac9eb8760 2024-05-11T08:07:01.183 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.178+0000 7f15c3fff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558ac9e2a260 0x558ac9f4e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:01.183 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.178+0000 7f15c3fff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558ac9e2a260 0x558ac9f4e900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:43176/0 (socket says 172.21.15.167:43176) 2024-05-11T08:07:01.183 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.178+0000 7f15c3fff640 1 -- 172.21.15.167:0/1013630268 learned_addr learned my addr 172.21.15.167:0/1013630268 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:07:01.183 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.178+0000 7f15c96ee640 1 --2- 172.21.15.167:0/1013630268 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ac9ddffc0 0x558ac9eb8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:01.183 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.178+0000 7f15c9eef640 1 --2- 172.21.15.167:0/1013630268 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x558ac9eb8760 0x558ac9e25e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:01.183 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.178+0000 7f15c3fff640 1 -- 172.21.15.167:0/1013630268 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x558ac9eb8760 msgr2=0x558ac9e25e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:01.183 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.178+0000 7f15c3fff640 1 --2- 172.21.15.167:0/1013630268 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x558ac9eb8760 0x558ac9e25e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:01.183 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.178+0000 7f15c3fff640 1 -- 172.21.15.167:0/1013630268 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ac9ddffc0 msgr2=0x558ac9eb8b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:01.183 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.178+0000 7f15c3fff640 1 --2- 172.21.15.167:0/1013630268 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ac9ddffc0 0x558ac9eb8b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:01.183 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.178+0000 7f15c3fff640 1 -- 172.21.15.167:0/1013630268 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ac9eb9080 con 0x558ac9e2a260 2024-05-11T08:07:01.183 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.178+0000 7f15c3fff640 1 --2- 172.21.15.167:0/1013630268 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558ac9e2a260 0x558ac9f4e900 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f15b4025e20 tx=0x7f15b4002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:01.184 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.178+0000 7f15c27fc640 1 -- 172.21.15.167:0/1013630268 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15b4055050 con 0x558ac9e2a260 2024-05-11T08:07:01.184 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.178+0000 7f15c27fc640 1 -- 172.21.15.167:0/1013630268 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15b4051dc0 con 0x558ac9e2a260 2024-05-11T08:07:01.184 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.178+0000 7f15cc51ff40 1 -- 172.21.15.167:0/1013630268 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x558ac9f4ee40 con 0x558ac9e2a260 2024-05-11T08:07:01.184 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.178+0000 7f15c27fc640 1 -- 172.21.15.167:0/1013630268 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15b4068040 con 0x558ac9e2a260 2024-05-11T08:07:01.184 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.178+0000 7f15cc51ff40 1 -- 172.21.15.167:0/1013630268 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ac9e0ba20 con 0x558ac9e2a260 2024-05-11T08:07:01.184 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.180+0000 7f15c27fc640 1 -- 172.21.15.167:0/1013630268 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f15b4058070 con 0x558ac9e2a260 2024-05-11T08:07:01.184 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.180+0000 7f15c27fc640 1 --2- 172.21.15.167:0/1013630268 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f15ac074cb0 0x7f15ac077170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:01.184 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.180+0000 7f15c96ee640 1 --2- 172.21.15.167:0/1013630268 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f15ac074cb0 0x7f15ac077170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:01.184 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.180+0000 7f15c27fc640 1 -- 172.21.15.167:0/1013630268 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6101+0+0 (secure 0 0 0) 0x7f15b40db3c0 con 0x558ac9e2a260 2024-05-11T08:07:01.184 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.180+0000 7f15cc51ff40 1 --2- 172.21.15.167:0/1013630268 >> [v2:172.21.15.204:6824/2921847521,v1:172.21.15.204:6825/2921847521] conn(0x558ac9e0fbd0 0x558ac9e0ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:01.184 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.180+0000 7f15cc51ff40 1 -- 172.21.15.167:0/1013630268 --> [v2:172.21.15.204:6824/2921847521,v1:172.21.15.204:6825/2921847521] -- osd_op(unknown.0.0:1 6.36 6:6ee5e2c4:::100000001f9.00000000:head [getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41) -- 0x558ac9e104f0 con 0x558ac9e0fbd0 2024-05-11T08:07:01.184 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.181+0000 7f15c96ee640 1 --2- 172.21.15.167:0/1013630268 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f15ac074cb0 0x7f15ac077170 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x558ac9e28950 tx=0x7f15bc005840 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:01.184 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.181+0000 7f15c9eef640 1 --2- 172.21.15.167:0/1013630268 >> [v2:172.21.15.204:6824/2921847521,v1:172.21.15.204:6825/2921847521] conn(0x558ac9e0fbd0 0x558ac9e0ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:01.184 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.181+0000 7f15c9eef640 1 --2- 172.21.15.167:0/1013630268 >> [v2:172.21.15.204:6824/2921847521,v1:172.21.15.204:6825/2921847521] conn(0x558ac9e0fbd0 0x558ac9e0ffb0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:01.184 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.181+0000 7f15c9eef640 1 -- 172.21.15.167:0/1013630268 <== osd.5 v2:172.21.15.204:6824/2921847521 1 ==== osd_op_reply(1 100000001f9.00000000 [getxattr (11) out=11b] v0'0 uv1 ondisk = 0) ==== 164+0+11 (crc 0 0 0) 0x7f15c4003450 con 0x558ac9e0fbd0 2024-05-11T08:07:01.184 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.181+0000 7f15cc51ff40 1 -- 172.21.15.167:0/1013630268 >> [v2:172.21.15.204:6824/2921847521,v1:172.21.15.204:6825/2921847521] conn(0x558ac9e0fbd0 msgr2=0x558ac9e0ffb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:01.185 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.181+0000 7f15cc51ff40 1 --2- 172.21.15.167:0/1013630268 >> [v2:172.21.15.204:6824/2921847521,v1:172.21.15.204:6825/2921847521] conn(0x558ac9e0fbd0 0x558ac9e0ffb0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:01.185 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.181+0000 7f15cc51ff40 1 -- 172.21.15.167:0/1013630268 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f15ac074cb0 msgr2=0x7f15ac077170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:01.185 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.181+0000 7f15cc51ff40 1 --2- 172.21.15.167:0/1013630268 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f15ac074cb0 0x7f15ac077170 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x558ac9e28950 tx=0x7f15bc005840 comp rx=0 tx=0).stop 2024-05-11T08:07:01.185 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.181+0000 7f15cc51ff40 1 -- 172.21.15.167:0/1013630268 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558ac9e2a260 msgr2=0x558ac9f4e900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:01.185 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.181+0000 7f15cc51ff40 1 --2- 172.21.15.167:0/1013630268 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558ac9e2a260 0x558ac9f4e900 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f15b4025e20 tx=0x7f15b4002fb0 comp rx=0 tx=0).stop 2024-05-11T08:07:01.185 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.181+0000 7f15c9eef640 1 -- 172.21.15.167:0/1013630268 reap_dead start 2024-05-11T08:07:01.185 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.181+0000 7f15cc51ff40 1 -- 172.21.15.167:0/1013630268 shutdown_connections 2024-05-11T08:07:01.185 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.181+0000 7f15cc51ff40 1 -- 172.21.15.167:0/1013630268 >> 172.21.15.167:0/1013630268 conn(0x558ac9e0d950 msgr2=0x558ac9e24fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:07:01.185 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.181+0000 7f15cc51ff40 1 -- 172.21.15.167:0/1013630268 shutdown_connections 2024-05-11T08:07:01.185 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.181+0000 7f15cc51ff40 1 -- 172.21.15.167:0/1013630268 wait complete. 2024-05-11T08:07:01.186 INFO:teuthology.orchestra.run.smithi167.stdout:./file1 2024-05-11T08:07:01.186 DEBUG:teuthology.orchestra.run.smithi167:> ceph-dencoder type string_wrapper import - decode dump_json 2024-05-11T08:07:01.339 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-05-11T08:07:01.339 INFO:teuthology.orchestra.run.smithi167.stdout: "s": "./file1" 2024-05-11T08:07:01.339 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-05-11T08:07:01.339 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T08:07:01.367 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T08:07:01.367 DEBUG:teuthology.orchestra.run.smithi167:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parent_a) 2024-05-11T08:07:01.384 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T08:07:01.384 DEBUG:teuthology.orchestra.run.smithi167:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch parent_a/alpha) 2024-05-11T08:07:01.444 DEBUG:teuthology.orchestra.run.smithi167:> stdin-killer -- python3 -c ' 2024-05-11T08:07:01.444 DEBUG:teuthology.orchestra.run.smithi167:> import os 2024-05-11T08:07:01.444 DEBUG:teuthology.orchestra.run.smithi167:> import stat 2024-05-11T08:07:01.444 DEBUG:teuthology.orchestra.run.smithi167:> 2024-05-11T08:07:01.444 DEBUG:teuthology.orchestra.run.smithi167:> print(os.stat("/home/ubuntu/cephtest/mnt.0/parent_a/alpha").st_ino) 2024-05-11T08:07:01.445 DEBUG:teuthology.orchestra.run.smithi167:> ' 2024-05-11T08:07:01.550 INFO:teuthology.orchestra.run.smithi167.stdout:1099511628283 2024-05-11T08:07:01.553 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T08:07:01.559 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-11T08:07:01.701 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.700+0000 7f78fe1c8640 1 Processor -- start 2024-05-11T08:07:01.701 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.700+0000 7f78fe1c8640 1 -- start start 2024-05-11T08:07:01.702 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.701+0000 7f78fe1c8640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f78f8146b40 0x7f78f806e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:01.702 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.701+0000 7f78fe1c8640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f78f806eb50 0x7f78f806cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:01.702 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.701+0000 7f78fe1c8640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f78f815e520 con 0x7f78f8146760 2024-05-11T08:07:01.702 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.701+0000 7f78fe1c8640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f78f80707a0 con 0x7f78f8146b40 2024-05-11T08:07:01.702 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.701+0000 7f78fe1c8640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f78f806ef80 con 0x7f78f806eb50 2024-05-11T08:07:01.703 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.702+0000 7f78f6ffd640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f78f8146b40 0x7f78f806e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:01.703 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.702+0000 7f78f6ffd640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f78f8146b40 0x7f78f806e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:43188/0 (socket says 172.21.15.167:43188) 2024-05-11T08:07:01.703 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.702+0000 7f78f6ffd640 1 -- 172.21.15.167:0/3481049928 learned_addr learned my addr 172.21.15.167:0/3481049928 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:07:01.703 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.702+0000 7f78f77fe640 1 --1- 172.21.15.167:0/3481049928 >> v1:172.21.15.167:6789/0 conn(0x7f78f8146760 0x7f78f80d13a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:48380/0 (socket says 172.21.15.167:48380) 2024-05-11T08:07:01.703 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.702+0000 7f78f7fff640 1 --2- 172.21.15.167:0/3481049928 >> v2:172.21.15.204:3300/0 conn(0x7f78f806eb50 0x7f78f806cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:01.703 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.702+0000 7f78f6ffd640 1 -- 172.21.15.167:0/3481049928 >> v2:172.21.15.204:3300/0 conn(0x7f78f806eb50 msgr2=0x7f78f806cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:07:01.703 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.702+0000 7f78f6ffd640 1 --2- 172.21.15.167:0/3481049928 >> v2:172.21.15.204:3300/0 conn(0x7f78f806eb50 0x7f78f806cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:01.703 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.702+0000 7f78f67fc640 1 -- 172.21.15.167:0/3481049928 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2347860969 0 0) 0x7f78f815e520 con 0x7f78f8146760 2024-05-11T08:07:01.703 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.702+0000 7f78f6ffd640 1 -- 172.21.15.167:0/3481049928 >> v1:172.21.15.167:6789/0 conn(0x7f78f8146760 legacy=0x7f78f80d13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:01.703 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.702+0000 7f78f6ffd640 1 -- 172.21.15.167:0/3481049928 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78f80709d0 con 0x7f78f8146b40 2024-05-11T08:07:01.705 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.703+0000 7f78f6ffd640 1 --2- 172.21.15.167:0/3481049928 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f78f8146b40 0x7f78f806e610 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f78f806e450 tx=0x7f78ec02a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78ecdf3b2ab5bdfc server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:01.705 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.703+0000 7f78f67fc640 1 -- 172.21.15.167:0/3481049928 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78ec037070 con 0x7f78f8146b40 2024-05-11T08:07:01.705 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.703+0000 7f78f67fc640 1 -- 172.21.15.167:0/3481049928 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78ec032440 con 0x7f78f8146b40 2024-05-11T08:07:01.705 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.703+0000 7f78f67fc640 1 -- 172.21.15.167:0/3481049928 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78ec032810 con 0x7f78f8146b40 2024-05-11T08:07:01.705 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.703+0000 7f78fe1c8640 1 -- 172.21.15.167:0/3481049928 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f78f8146b40 msgr2=0x7f78f806e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:01.705 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.703+0000 7f78fe1c8640 1 --2- 172.21.15.167:0/3481049928 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f78f8146b40 0x7f78f806e610 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f78f806e450 tx=0x7f78ec02a2a0 comp rx=0 tx=0).stop 2024-05-11T08:07:01.705 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.703+0000 7f78fe1c8640 1 -- 172.21.15.167:0/3481049928 shutdown_connections 2024-05-11T08:07:01.705 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.703+0000 7f78fe1c8640 1 --2- 172.21.15.167:0/3481049928 >> v2:172.21.15.204:3300/0 conn(0x7f78f806eb50 0x7f78f806cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:01.705 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.703+0000 7f78fe1c8640 1 --2- 172.21.15.167:0/3481049928 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f78f8146b40 0x7f78f806e610 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:01.705 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.703+0000 7f78fe1c8640 1 -- 172.21.15.167:0/3481049928 >> 172.21.15.167:0/3481049928 conn(0x7f78f80d1e40 msgr2=0x7f78f80d2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:07:01.706 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.703+0000 7f78fe1c8640 1 -- 172.21.15.167:0/3481049928 shutdown_connections 2024-05-11T08:07:01.706 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.703+0000 7f78fe1c8640 1 -- 172.21.15.167:0/3481049928 wait complete. 2024-05-11T08:07:01.706 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.704+0000 7f78fe1c8640 1 Processor -- start 2024-05-11T08:07:01.706 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.705+0000 7f78fe1c8640 1 -- start start 2024-05-11T08:07:01.708 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.705+0000 7f78fe1c8640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f78f806eb50 0x7f78f80cd9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:01.709 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.705+0000 7f78fe1c8640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f78f8146760 0x7f78f8137570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:01.709 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.705+0000 7f78fe1c8640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f78f8146b40 0x7f78f80b6440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:01.709 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.705+0000 7f78fe1c8640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f78f8071270 con 0x7f78f806eb50 2024-05-11T08:07:01.709 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.705+0000 7f78fe1c8640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f78f8070d10 con 0x7f78f8146b40 2024-05-11T08:07:01.709 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.705+0000 7f78fe1c8640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f78f8071550 con 0x7f78f8146760 2024-05-11T08:07:01.709 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.706+0000 7f78f77fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f78f806eb50 0x7f78f80cd9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:01.709 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.706+0000 7f78f77fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f78f806eb50 0x7f78f80cd9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:43196/0 (socket says 172.21.15.167:43196) 2024-05-11T08:07:01.709 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.706+0000 7f78f77fe640 1 -- 172.21.15.167:0/2838305879 learned_addr learned my addr 172.21.15.167:0/2838305879 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:07:01.709 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.706+0000 7f78f7fff640 1 --2- 172.21.15.167:0/2838305879 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f78f8146b40 0x7f78f80b6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:01.709 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.706+0000 7f78f6ffd640 1 --2- 172.21.15.167:0/2838305879 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f78f8146760 0x7f78f8137570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:01.709 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.706+0000 7f78f77fe640 1 -- 172.21.15.167:0/2838305879 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f78f8146760 msgr2=0x7f78f8137570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:01.709 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.706+0000 7f78f77fe640 1 --2- 172.21.15.167:0/2838305879 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f78f8146760 0x7f78f8137570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:01.709 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.706+0000 7f78f77fe640 1 -- 172.21.15.167:0/2838305879 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f78f8146b40 msgr2=0x7f78f80b6440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:01.709 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.706+0000 7f78f77fe640 1 --2- 172.21.15.167:0/2838305879 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f78f8146b40 0x7f78f80b6440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:01.709 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.706+0000 7f78f77fe640 1 -- 172.21.15.167:0/2838305879 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78f80ccbf0 con 0x7f78f806eb50 2024-05-11T08:07:01.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.706+0000 7f78f6ffd640 1 --2- 172.21.15.167:0/2838305879 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f78f8146760 0x7f78f8137570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T08:07:01.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.706+0000 7f78f77fe640 1 --2- 172.21.15.167:0/2838305879 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f78f806eb50 0x7f78f80cd9b0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f78e800c4c0 tx=0x7f78e800c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:01.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.706+0000 7f78dffff640 1 -- 172.21.15.167:0/2838305879 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78e8018070 con 0x7f78f806eb50 2024-05-11T08:07:01.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.706+0000 7f78dffff640 1 -- 172.21.15.167:0/2838305879 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78e8007500 con 0x7f78f806eb50 2024-05-11T08:07:01.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.706+0000 7f78dffff640 1 -- 172.21.15.167:0/2838305879 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78e8004a20 con 0x7f78f806eb50 2024-05-11T08:07:01.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.706+0000 7f78fe1c8640 1 -- 172.21.15.167:0/2838305879 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78f80cced0 con 0x7f78f806eb50 2024-05-11T08:07:01.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.706+0000 7f78fe1c8640 1 -- 172.21.15.167:0/2838305879 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78f80b6980 con 0x7f78f806eb50 2024-05-11T08:07:01.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.707+0000 7f78fe1c8640 1 -- 172.21.15.167:0/2838305879 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78f8069bc0 con 0x7f78f806eb50 2024-05-11T08:07:01.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.708+0000 7f78dffff640 1 -- 172.21.15.167:0/2838305879 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f78e801b070 con 0x7f78f806eb50 2024-05-11T08:07:01.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.708+0000 7f78dffff640 1 --2- 172.21.15.167:0/2838305879 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f78e40762d0 0x7f78e4078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:01.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.708+0000 7f78dffff640 1 -- 172.21.15.167:0/2838305879 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6101+0+0 (secure 0 0 0) 0x7f78e8096940 con 0x7f78f806eb50 2024-05-11T08:07:01.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.709+0000 7f78f6ffd640 1 --2- 172.21.15.167:0/2838305879 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f78e40762d0 0x7f78e4078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:01.711 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.709+0000 7f78f6ffd640 1 --2- 172.21.15.167:0/2838305879 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f78e40762d0 0x7f78e4078790 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f78ec034000 tx=0x7f78ec02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:01.713 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.713+0000 7f78dffff640 1 -- 172.21.15.167:0/2838305879 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f78e805fb50 con 0x7f78f806eb50 2024-05-11T08:07:01.931 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.931+0000 7f78fe1c8640 1 -- 172.21.15.167:0/2838305879 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f78f806f260 con 0x7f78f806eb50 2024-05-11T08:07:01.931 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.931+0000 7f78dffff640 1 -- 172.21.15.167:0/2838305879 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5190 (secure 0 0 0) 0x7f78e8063e50 con 0x7f78f806eb50 2024-05-11T08:07:01.932 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T08:07:01.932 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4604,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2674983673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":2674983673},{"type":"v1","addr":"172.21.15.167:6837","nonce":2674983673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4607,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6834/1018548066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1018548066},{"type":"v1","addr":"172.21.15.204:6834","nonce":1018548066}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4729,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3369722249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3369722249},{"type":"v1","addr":"172.21.15.167:6836","nonce":3369722249}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-11T08:06:47.943939+0000","modified":"2024-05-11T08:06:49.955430+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4750},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4750":{"gid":4750,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.204:6835/469864006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6833","nonce":469864006},{"type":"v1","addr":"172.21.15.204:6835","nonce":469864006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4750,"qdb_cluster":[4750]},"id":2}]} 2024-05-11T08:07:01.932 INFO:teuthology.orchestra.run.smithi167.stderr:dumped fsmap epoch 16 2024-05-11T08:07:01.934 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.934+0000 7f78fe1c8640 1 -- 172.21.15.167:0/2838305879 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f78e40762d0 msgr2=0x7f78e4078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:01.934 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.934+0000 7f78fe1c8640 1 --2- 172.21.15.167:0/2838305879 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f78e40762d0 0x7f78e4078790 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f78ec034000 tx=0x7f78ec02e000 comp rx=0 tx=0).stop 2024-05-11T08:07:01.934 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.934+0000 7f78fe1c8640 1 -- 172.21.15.167:0/2838305879 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f78f806eb50 msgr2=0x7f78f80cd9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:01.934 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.934+0000 7f78fe1c8640 1 --2- 172.21.15.167:0/2838305879 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f78f806eb50 0x7f78f80cd9b0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f78e800c4c0 tx=0x7f78e800c980 comp rx=0 tx=0).stop 2024-05-11T08:07:01.934 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.934+0000 7f78fe1c8640 1 -- 172.21.15.167:0/2838305879 shutdown_connections 2024-05-11T08:07:01.934 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.934+0000 7f78fe1c8640 1 --2- 172.21.15.167:0/2838305879 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f78e40762d0 0x7f78e4078790 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:01.934 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.934+0000 7f78fe1c8640 1 --2- 172.21.15.167:0/2838305879 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f78f8146b40 0x7f78f80b6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:01.934 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.934+0000 7f78fe1c8640 1 --2- 172.21.15.167:0/2838305879 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f78f8146760 0x7f78f8137570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:01.934 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.934+0000 7f78fe1c8640 1 --2- 172.21.15.167:0/2838305879 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f78f806eb50 0x7f78f80cd9b0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:01.935 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.934+0000 7f78fe1c8640 1 -- 172.21.15.167:0/2838305879 >> 172.21.15.167:0/2838305879 conn(0x7f78f80d1e40 msgr2=0x7f78f815c330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:07:01.935 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.934+0000 7f78fe1c8640 1 -- 172.21.15.167:0/2838305879 shutdown_connections 2024-05-11T08:07:01.935 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:01.934+0000 7f78fe1c8640 1 -- 172.21.15.167:0/2838305879 wait complete. 2024-05-11T08:07:01.945 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json flush journal 2024-05-11T08:07:02.081 INFO:teuthology.orchestra.run.smithi204.stdout:{"message":"","return_code":0} 2024-05-11T08:07:02.081 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-05-11T08:07:02.081 DEBUG:teuthology.orchestra.run.smithi167:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001fb.00000000 parent 2024-05-11T08:07:02.125 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.125+0000 7f6306f1af40 1 Processor -- start 2024-05-11T08:07:02.125 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.125+0000 7f6306f1af40 1 -- start start 2024-05-11T08:07:02.130 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.126+0000 7f6306f1af40 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564531f75b00 0x564531f74d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:02.130 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.126+0000 7f6306f1af40 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564531f75ee0 0x564531ee6450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:02.130 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.126+0000 7f6306f1af40 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x564531f90370 0x564531ee6990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:02.130 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.126+0000 7f6306f1af40 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x564531f01ff0 con 0x564531f75b00 2024-05-11T08:07:02.130 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.126+0000 7f6306f1af40 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x564531e62f50 con 0x564531f75ee0 2024-05-11T08:07:02.130 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.126+0000 7f6306f1af40 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x564531ebcb20 con 0x564531f90370 2024-05-11T08:07:02.130 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.126+0000 7f6303fb0640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564531f75b00 0x564531f74d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:02.130 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.126+0000 7f6303fb0640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564531f75b00 0x564531f74d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:43206/0 (socket says 172.21.15.167:43206) 2024-05-11T08:07:02.130 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.126+0000 7f6303fb0640 1 -- 172.21.15.167:0/1487814844 learned_addr learned my addr 172.21.15.167:0/1487814844 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:07:02.131 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.126+0000 7f62fbfff640 1 --2- 172.21.15.167:0/1487814844 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564531f75ee0 0x564531ee6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:02.131 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.126+0000 7f63047b1640 1 --2- 172.21.15.167:0/1487814844 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x564531f90370 0x564531ee6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:02.131 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.126+0000 7f6303fb0640 1 -- 172.21.15.167:0/1487814844 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x564531f90370 msgr2=0x564531ee6990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:07:02.131 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.126+0000 7f6303fb0640 1 --2- 172.21.15.167:0/1487814844 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x564531f90370 0x564531ee6990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:02.131 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.126+0000 7f6303fb0640 1 -- 172.21.15.167:0/1487814844 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564531f75ee0 msgr2=0x564531ee6450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:07:02.131 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.126+0000 7f6303fb0640 1 --2- 172.21.15.167:0/1487814844 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564531f75ee0 0x564531ee6450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:02.131 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.126+0000 7f6303fb0640 1 -- 172.21.15.167:0/1487814844 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564531ea8f00 con 0x564531f75b00 2024-05-11T08:07:02.131 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.126+0000 7f63047b1640 1 --2- 172.21.15.167:0/1487814844 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x564531f90370 0x564531ee6990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T08:07:02.131 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.126+0000 7f62fbfff640 1 --2- 172.21.15.167:0/1487814844 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564531f75ee0 0x564531ee6450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T08:07:02.131 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.126+0000 7f6303fb0640 1 --2- 172.21.15.167:0/1487814844 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564531f75b00 0x564531f74d20 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f62f400b210 tx=0x7f62f400b6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d6ef72d6a917426 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:02.131 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.126+0000 7f6302fae640 1 -- 172.21.15.167:0/1487814844 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62f4008070 con 0x564531f75b00 2024-05-11T08:07:02.131 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.126+0000 7f6302fae640 1 -- 172.21.15.167:0/1487814844 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62f400d500 con 0x564531f75b00 2024-05-11T08:07:02.131 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.126+0000 7f6302fae640 1 -- 172.21.15.167:0/1487814844 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62f400d8d0 con 0x564531f75b00 2024-05-11T08:07:02.131 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.127+0000 7f6306f1af40 1 -- 172.21.15.167:0/1487814844 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564531f75b00 msgr2=0x564531f74d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:02.131 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.127+0000 7f6306f1af40 1 --2- 172.21.15.167:0/1487814844 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564531f75b00 0x564531f74d20 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f62f400b210 tx=0x7f62f400b6d0 comp rx=0 tx=0).stop 2024-05-11T08:07:02.132 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.127+0000 7f6306f1af40 1 -- 172.21.15.167:0/1487814844 shutdown_connections 2024-05-11T08:07:02.132 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.127+0000 7f6306f1af40 1 --2- 172.21.15.167:0/1487814844 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x564531f90370 0x564531ee6990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:02.132 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.127+0000 7f6306f1af40 1 --2- 172.21.15.167:0/1487814844 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564531f75ee0 0x564531ee6450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:02.132 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.127+0000 7f6306f1af40 1 --2- 172.21.15.167:0/1487814844 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564531f75b00 0x564531f74d20 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:02.132 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.127+0000 7f6306f1af40 1 -- 172.21.15.167:0/1487814844 >> 172.21.15.167:0/1487814844 conn(0x564531eb9c70 msgr2=0x564531f92440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:07:02.132 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.127+0000 7f6306f1af40 1 -- 172.21.15.167:0/1487814844 shutdown_connections 2024-05-11T08:07:02.132 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.127+0000 7f6306f1af40 1 -- 172.21.15.167:0/1487814844 wait complete. 2024-05-11T08:07:02.132 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.128+0000 7f6306f1af40 1 Processor -- start 2024-05-11T08:07:02.132 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.128+0000 7f6306f1af40 1 -- start start 2024-05-11T08:07:02.132 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.128+0000 7f6306f1af40 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x564531f75b00 0x564531f73fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:02.132 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.128+0000 7f6306f1af40 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564531f75ee0 0x564531f8ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:02.132 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.128+0000 7f6306f1af40 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564531f90370 0x564531f8e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:02.132 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.128+0000 7f6306f1af40 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x564531ea8c20 con 0x564531f75ee0 2024-05-11T08:07:02.132 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.128+0000 7f6306f1af40 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x564531ee5d80 con 0x564531f90370 2024-05-11T08:07:02.133 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.128+0000 7f6306f1af40 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x564531ee5450 con 0x564531f75b00 2024-05-11T08:07:02.133 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.128+0000 7f62fbfff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564531f75ee0 0x564531f8ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:02.133 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.128+0000 7f62fbfff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564531f75ee0 0x564531f8ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:43208/0 (socket says 172.21.15.167:43208) 2024-05-11T08:07:02.133 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.128+0000 7f62fbfff640 1 -- 172.21.15.167:0/2648047634 learned_addr learned my addr 172.21.15.167:0/2648047634 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:07:02.133 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.128+0000 7f6303fb0640 1 --2- 172.21.15.167:0/2648047634 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x564531f75b00 0x564531f73fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:02.133 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.128+0000 7f63047b1640 1 --2- 172.21.15.167:0/2648047634 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564531f90370 0x564531f8e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:02.133 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.129+0000 7f62fbfff640 1 -- 172.21.15.167:0/2648047634 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x564531f75b00 msgr2=0x564531f73fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:02.133 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.129+0000 7f62fbfff640 1 --2- 172.21.15.167:0/2648047634 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x564531f75b00 0x564531f73fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:02.133 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.129+0000 7f62fbfff640 1 -- 172.21.15.167:0/2648047634 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564531f90370 msgr2=0x564531f8e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:02.133 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.129+0000 7f62fbfff640 1 --2- 172.21.15.167:0/2648047634 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564531f90370 0x564531f8e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:02.133 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.129+0000 7f62fbfff640 1 -- 172.21.15.167:0/2648047634 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564531f009a0 con 0x564531f75ee0 2024-05-11T08:07:02.133 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.129+0000 7f62fbfff640 1 --2- 172.21.15.167:0/2648047634 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564531f75ee0 0x564531f8ab60 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f62ec009ec0 tx=0x7f62ec008400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:02.133 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.129+0000 7f6301fac640 1 -- 172.21.15.167:0/2648047634 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62ec0145f0 con 0x564531f75ee0 2024-05-11T08:07:02.133 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.129+0000 7f6301fac640 1 -- 172.21.15.167:0/2648047634 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62ec014790 con 0x564531f75ee0 2024-05-11T08:07:02.133 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.129+0000 7f6306f1af40 1 -- 172.21.15.167:0/2648047634 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x564531f00c80 con 0x564531f75ee0 2024-05-11T08:07:02.133 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.129+0000 7f6301fac640 1 -- 172.21.15.167:0/2648047634 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62ec014b60 con 0x564531f75ee0 2024-05-11T08:07:02.134 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.129+0000 7f6306f1af40 1 -- 172.21.15.167:0/2648047634 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564531f01240 con 0x564531f75ee0 2024-05-11T08:07:02.135 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.130+0000 7f6301fac640 1 -- 172.21.15.167:0/2648047634 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f62ec014d00 con 0x564531f75ee0 2024-05-11T08:07:02.135 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.131+0000 7f6301fac640 1 --2- 172.21.15.167:0/2648047634 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f62e4074cb0 0x7f62e4077170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:02.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.131+0000 7f6301fac640 1 -- 172.21.15.167:0/2648047634 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6101+0+0 (secure 0 0 0) 0x7f62ec095a00 con 0x564531f75ee0 2024-05-11T08:07:02.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.131+0000 7f6303fb0640 1 --2- 172.21.15.167:0/2648047634 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f62e4074cb0 0x7f62e4077170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:02.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.131+0000 7f6306f1af40 1 --2- 172.21.15.167:0/2648047634 >> [v2:172.21.15.167:6824/453689293,v1:172.21.15.167:6825/453689293] conn(0x564531f8b0a0 0x564531ee67b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:02.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.131+0000 7f6306f1af40 1 -- 172.21.15.167:0/2648047634 --> [v2:172.21.15.167:6824/453689293,v1:172.21.15.167:6825/453689293] -- osd_op(unknown.0.0:1 6.13 6:ca8fdbba:::100000001fb.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41) -- 0x56453202c6c0 con 0x564531f8b0a0 2024-05-11T08:07:02.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.131+0000 7f63047b1640 1 --2- 172.21.15.167:0/2648047634 >> [v2:172.21.15.167:6824/453689293,v1:172.21.15.167:6825/453689293] conn(0x564531f8b0a0 0x564531ee67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:02.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.131+0000 7f6303fb0640 1 --2- 172.21.15.167:0/2648047634 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f62e4074cb0 0x7f62e4077170 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f62f400b210 tx=0x7f62f4008000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:02.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.132+0000 7f63047b1640 1 --2- 172.21.15.167:0/2648047634 >> [v2:172.21.15.167:6824/453689293,v1:172.21.15.167:6825/453689293] conn(0x564531f8b0a0 0x564531ee67b0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:02.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.132+0000 7f63047b1640 1 -- 172.21.15.167:0/2648047634 <== osd.3 v2:172.21.15.167:6824/453689293 1 ==== osd_op_reply(1 100000001fb.00000000 [getxattr (95) out=95b] v0'0 uv1 ondisk = 0) ==== 164+0+95 (crc 0 0 0) 0x7f62fc0538d0 con 0x564531f8b0a0 2024-05-11T08:07:02.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.132+0000 7f6306f1af40 1 -- 172.21.15.167:0/2648047634 >> [v2:172.21.15.167:6824/453689293,v1:172.21.15.167:6825/453689293] conn(0x564531f8b0a0 msgr2=0x564531ee67b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:02.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.132+0000 7f6306f1af40 1 --2- 172.21.15.167:0/2648047634 >> [v2:172.21.15.167:6824/453689293,v1:172.21.15.167:6825/453689293] conn(0x564531f8b0a0 0x564531ee67b0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:02.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.132+0000 7f6306f1af40 1 -- 172.21.15.167:0/2648047634 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f62e4074cb0 msgr2=0x7f62e4077170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:02.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.132+0000 7f6306f1af40 1 --2- 172.21.15.167:0/2648047634 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f62e4074cb0 0x7f62e4077170 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f62f400b210 tx=0x7f62f4008000 comp rx=0 tx=0).stop 2024-05-11T08:07:02.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.132+0000 7f6306f1af40 1 -- 172.21.15.167:0/2648047634 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564531f75ee0 msgr2=0x564531f8ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:02.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.132+0000 7f6306f1af40 1 --2- 172.21.15.167:0/2648047634 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564531f75ee0 0x564531f8ab60 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f62ec009ec0 tx=0x7f62ec008400 comp rx=0 tx=0).stop 2024-05-11T08:07:02.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.132+0000 7f63047b1640 1 -- 172.21.15.167:0/2648047634 reap_dead start 2024-05-11T08:07:02.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.132+0000 7f6306f1af40 1 -- 172.21.15.167:0/2648047634 shutdown_connections 2024-05-11T08:07:02.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.132+0000 7f6306f1af40 1 -- 172.21.15.167:0/2648047634 >> 172.21.15.167:0/2648047634 conn(0x564531eb9c70 msgr2=0x564531f74d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:07:02.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.132+0000 7f6306f1af40 1 -- 172.21.15.167:0/2648047634 shutdown_connections 2024-05-11T08:07:02.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.132+0000 7f6306f1af40 1 -- 172.21.15.167:0/2648047634 wait complete. 2024-05-11T08:07:02.137 INFO:teuthology.orchestra.run.smithi167.stdout:Y��alphaparent_a 2024-05-11T08:07:02.138 DEBUG:teuthology.orchestra.run.smithi167:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2024-05-11T08:07:02.228 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-05-11T08:07:02.228 INFO:teuthology.orchestra.run.smithi167.stdout: "ino": 1099511628283, 2024-05-11T08:07:02.228 INFO:teuthology.orchestra.run.smithi167.stdout: "ancestors": [ 2024-05-11T08:07:02.228 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T08:07:02.229 INFO:teuthology.orchestra.run.smithi167.stdout: "dirino": 1099511628282, 2024-05-11T08:07:02.229 INFO:teuthology.orchestra.run.smithi167.stdout: "dname": "alpha", 2024-05-11T08:07:02.229 INFO:teuthology.orchestra.run.smithi167.stdout: "version": 2 2024-05-11T08:07:02.229 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T08:07:02.229 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T08:07:02.229 INFO:teuthology.orchestra.run.smithi167.stdout: "dirino": 1, 2024-05-11T08:07:02.229 INFO:teuthology.orchestra.run.smithi167.stdout: "dname": "parent_a", 2024-05-11T08:07:02.229 INFO:teuthology.orchestra.run.smithi167.stdout: "version": 18 2024-05-11T08:07:02.229 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T08:07:02.229 INFO:teuthology.orchestra.run.smithi167.stdout: ], 2024-05-11T08:07:02.229 INFO:teuthology.orchestra.run.smithi167.stdout: "pool": 6, 2024-05-11T08:07:02.229 INFO:teuthology.orchestra.run.smithi167.stdout: "old_pools": [] 2024-05-11T08:07:02.229 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-05-11T08:07:02.230 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T08:07:02.231 DEBUG:teuthology.orchestra.run.smithi167:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001fb.00000000 layout 2024-05-11T08:07:02.319 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.314+0000 7f4bc873af40 1 Processor -- start 2024-05-11T08:07:02.319 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.314+0000 7f4bc873af40 1 -- start start 2024-05-11T08:07:02.319 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.315+0000 7f4bc873af40 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5597978b7720 0x559797922530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:02.319 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.315+0000 7f4bc873af40 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x5597979434d0 0x55979789d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:02.320 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.315+0000 7f4bc873af40 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5597979438b0 0x55979789dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:02.320 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.315+0000 7f4bc873af40 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x559797815f50 con 0x5597979438b0 2024-05-11T08:07:02.320 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.315+0000 7f4bc873af40 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x559797819f90 con 0x5597978b7720 2024-05-11T08:07:02.320 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.315+0000 7f4bc873af40 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x559797943e60 con 0x5597979434d0 2024-05-11T08:07:02.320 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.315+0000 7f4bc6a35640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5597979438b0 0x55979789dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:02.320 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.315+0000 7f4bc6a35640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5597979438b0 0x55979789dc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:43224/0 (socket says 172.21.15.167:43224) 2024-05-11T08:07:02.320 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.315+0000 7f4bc6a35640 1 -- 172.21.15.167:0/3042010758 learned_addr learned my addr 172.21.15.167:0/3042010758 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:07:02.320 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.315+0000 7f4bbffff640 1 --2- 172.21.15.167:0/3042010758 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x5597979434d0 0x55979789d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:02.320 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.315+0000 7f4bc6234640 1 --2- 172.21.15.167:0/3042010758 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5597978b7720 0x559797922530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:02.320 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.315+0000 7f4bc6a35640 1 -- 172.21.15.167:0/3042010758 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x5597979434d0 msgr2=0x55979789d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:07:02.320 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.315+0000 7f4bc6a35640 1 --2- 172.21.15.167:0/3042010758 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x5597979434d0 0x55979789d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:02.320 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.315+0000 7f4bc6a35640 1 -- 172.21.15.167:0/3042010758 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5597978b7720 msgr2=0x559797922530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:07:02.320 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.315+0000 7f4bc6a35640 1 --2- 172.21.15.167:0/3042010758 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5597978b7720 0x559797922530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:02.321 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.315+0000 7f4bc6a35640 1 -- 172.21.15.167:0/3042010758 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559797922a70 con 0x5597979438b0 2024-05-11T08:07:02.321 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.315+0000 7f4bbffff640 1 --2- 172.21.15.167:0/3042010758 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x5597979434d0 0x55979789d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T08:07:02.321 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.315+0000 7f4bc6234640 1 --2- 172.21.15.167:0/3042010758 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5597978b7720 0x559797922530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T08:07:02.321 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.316+0000 7f4bc6a35640 1 --2- 172.21.15.167:0/3042010758 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5597979438b0 0x55979789dc40 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f4bc0001650 tx=0x7f4bc0001b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96a4c4d25c5b026a server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:02.321 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.316+0000 7f4bc5232640 1 -- 172.21.15.167:0/3042010758 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bc0016070 con 0x5597979438b0 2024-05-11T08:07:02.321 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.316+0000 7f4bc5232640 1 -- 172.21.15.167:0/3042010758 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4bc000f730 con 0x5597979438b0 2024-05-11T08:07:02.321 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.316+0000 7f4bc5232640 1 -- 172.21.15.167:0/3042010758 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bc000fb20 con 0x5597979438b0 2024-05-11T08:07:02.321 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.316+0000 7f4bc873af40 1 -- 172.21.15.167:0/3042010758 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5597979438b0 msgr2=0x55979789dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:02.321 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.316+0000 7f4bc873af40 1 --2- 172.21.15.167:0/3042010758 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5597979438b0 0x55979789dc40 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f4bc0001650 tx=0x7f4bc0001b10 comp rx=0 tx=0).stop 2024-05-11T08:07:02.321 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.316+0000 7f4bc873af40 1 -- 172.21.15.167:0/3042010758 shutdown_connections 2024-05-11T08:07:02.321 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.316+0000 7f4bc873af40 1 --2- 172.21.15.167:0/3042010758 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5597979438b0 0x55979789dc40 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:02.321 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.316+0000 7f4bc873af40 1 --2- 172.21.15.167:0/3042010758 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x5597979434d0 0x55979789d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:02.321 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.316+0000 7f4bc873af40 1 --2- 172.21.15.167:0/3042010758 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5597978b7720 0x559797922530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:02.321 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.316+0000 7f4bc873af40 1 -- 172.21.15.167:0/3042010758 >> 172.21.15.167:0/3042010758 conn(0x55979789a950 msgr2=0x559797899690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:07:02.321 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.316+0000 7f4bc873af40 1 -- 172.21.15.167:0/3042010758 shutdown_connections 2024-05-11T08:07:02.322 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.316+0000 7f4bc873af40 1 -- 172.21.15.167:0/3042010758 wait complete. 2024-05-11T08:07:02.322 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.317+0000 7f4bc873af40 1 Processor -- start 2024-05-11T08:07:02.322 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.317+0000 7f4bc873af40 1 -- start start 2024-05-11T08:07:02.322 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.318+0000 7f4bc873af40 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x5597978b7720 0x5597978701b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:02.322 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.318+0000 7f4bc873af40 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5597979434d0 0x5597978706f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:02.322 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.318+0000 7f4bc873af40 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5597979438b0 0x5597979dfa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:02.322 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.318+0000 7f4bc873af40 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55979789c380 con 0x5597979434d0 2024-05-11T08:07:02.322 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.318+0000 7f4bc873af40 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55979789c0a0 con 0x5597979438b0 2024-05-11T08:07:02.322 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.318+0000 7f4bc873af40 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x55979789c660 con 0x5597978b7720 2024-05-11T08:07:02.322 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.318+0000 7f4bbffff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5597979434d0 0x5597978706f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:02.322 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.318+0000 7f4bbffff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5597979434d0 0x5597978706f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:43230/0 (socket says 172.21.15.167:43230) 2024-05-11T08:07:02.322 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.318+0000 7f4bbffff640 1 -- 172.21.15.167:0/4073617499 learned_addr learned my addr 172.21.15.167:0/4073617499 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:07:02.322 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.318+0000 7f4bc6234640 1 --2- 172.21.15.167:0/4073617499 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x5597978b7720 0x5597978701b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:02.322 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.318+0000 7f4bc6a35640 1 --2- 172.21.15.167:0/4073617499 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5597979438b0 0x5597979dfa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:02.323 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.318+0000 7f4bbffff640 1 -- 172.21.15.167:0/4073617499 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x5597978b7720 msgr2=0x5597978701b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:02.323 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.318+0000 7f4bbffff640 1 --2- 172.21.15.167:0/4073617499 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x5597978b7720 0x5597978701b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:02.323 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.318+0000 7f4bbffff640 1 -- 172.21.15.167:0/4073617499 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5597979438b0 msgr2=0x5597979dfa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:02.323 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.318+0000 7f4bbffff640 1 --2- 172.21.15.167:0/4073617499 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5597979438b0 0x5597979dfa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:02.323 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.318+0000 7f4bbffff640 1 -- 172.21.15.167:0/4073617499 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597979d76d0 con 0x5597979434d0 2024-05-11T08:07:02.323 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.318+0000 7f4bbffff640 1 --2- 172.21.15.167:0/4073617499 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5597979434d0 0x5597978706f0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x55979789a610 tx=0x7f4bb000caa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:02.323 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.318+0000 7f4bbf7fe640 1 -- 172.21.15.167:0/4073617499 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bb000f860 con 0x5597979434d0 2024-05-11T08:07:02.323 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.318+0000 7f4bbf7fe640 1 -- 172.21.15.167:0/4073617499 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4bb0013030 con 0x5597979434d0 2024-05-11T08:07:02.323 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.318+0000 7f4bc873af40 1 -- 172.21.15.167:0/4073617499 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5597979212e0 con 0x5597979434d0 2024-05-11T08:07:02.323 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.318+0000 7f4bbf7fe640 1 -- 172.21.15.167:0/4073617499 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bb000fc00 con 0x5597979434d0 2024-05-11T08:07:02.323 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.318+0000 7f4bc873af40 1 -- 172.21.15.167:0/4073617499 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559797934a20 con 0x5597979434d0 2024-05-11T08:07:02.325 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.320+0000 7f4bbf7fe640 1 -- 172.21.15.167:0/4073617499 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f4bb00173d0 con 0x5597979434d0 2024-05-11T08:07:02.325 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.320+0000 7f4bbf7fe640 1 --2- 172.21.15.167:0/4073617499 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f4ba8074cb0 0x7f4ba8077170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:02.325 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.320+0000 7f4bc6234640 1 --2- 172.21.15.167:0/4073617499 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f4ba8074cb0 0x7f4ba8077170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:02.326 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.320+0000 7f4bbf7fe640 1 -- 172.21.15.167:0/4073617499 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6101+0+0 (secure 0 0 0) 0x7f4bb0096d30 con 0x5597979434d0 2024-05-11T08:07:02.326 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.321+0000 7f4bc873af40 1 --2- 172.21.15.167:0/4073617499 >> [v2:172.21.15.167:6824/453689293,v1:172.21.15.167:6825/453689293] conn(0x5597979d4aa0 0x5597979d4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:02.326 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.321+0000 7f4bc873af40 1 -- 172.21.15.167:0/4073617499 --> [v2:172.21.15.167:6824/453689293,v1:172.21.15.167:6825/453689293] -- osd_op(unknown.0.0:1 6.13 6:ca8fdbba:::100000001fb.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41) -- 0x5597979d53f0 con 0x5597979d4aa0 2024-05-11T08:07:02.326 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.321+0000 7f4bc6a35640 1 --2- 172.21.15.167:0/4073617499 >> [v2:172.21.15.167:6824/453689293,v1:172.21.15.167:6825/453689293] conn(0x5597979d4aa0 0x5597979d4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:02.326 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.321+0000 7f4bc6234640 1 --2- 172.21.15.167:0/4073617499 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f4ba8074cb0 0x7f4ba8077170 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f4bb8056000 tx=0x7f4bb8050000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:02.326 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.321+0000 7f4bc6a35640 1 --2- 172.21.15.167:0/4073617499 >> [v2:172.21.15.167:6824/453689293,v1:172.21.15.167:6825/453689293] conn(0x5597979d4aa0 0x5597979d4eb0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:02.326 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.321+0000 7f4bc6a35640 1 -- 172.21.15.167:0/4073617499 <== osd.3 v2:172.21.15.167:6824/453689293 1 ==== osd_op_reply(1 100000001fb.00000000 [getxattr (30) out=30b] v0'0 uv1 ondisk = 0) ==== 164+0+30 (crc 0 0 0) 0x7f4bc0019040 con 0x5597979d4aa0 2024-05-11T08:07:02.326 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.321+0000 7f4bc873af40 1 -- 172.21.15.167:0/4073617499 >> [v2:172.21.15.167:6824/453689293,v1:172.21.15.167:6825/453689293] conn(0x5597979d4aa0 msgr2=0x5597979d4eb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:02.326 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.321+0000 7f4bc873af40 1 --2- 172.21.15.167:0/4073617499 >> [v2:172.21.15.167:6824/453689293,v1:172.21.15.167:6825/453689293] conn(0x5597979d4aa0 0x5597979d4eb0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:02.326 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.322+0000 7f4bc873af40 1 -- 172.21.15.167:0/4073617499 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f4ba8074cb0 msgr2=0x7f4ba8077170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:02.326 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.322+0000 7f4bc873af40 1 --2- 172.21.15.167:0/4073617499 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f4ba8074cb0 0x7f4ba8077170 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f4bb8056000 tx=0x7f4bb8050000 comp rx=0 tx=0).stop 2024-05-11T08:07:02.326 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.322+0000 7f4bc873af40 1 -- 172.21.15.167:0/4073617499 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5597979434d0 msgr2=0x5597978706f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:02.326 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.322+0000 7f4bc873af40 1 --2- 172.21.15.167:0/4073617499 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5597979434d0 0x5597978706f0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x55979789a610 tx=0x7f4bb000caa0 comp rx=0 tx=0).stop 2024-05-11T08:07:02.326 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.322+0000 7f4bc6a35640 1 -- 172.21.15.167:0/4073617499 reap_dead start 2024-05-11T08:07:02.327 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.322+0000 7f4bc873af40 1 -- 172.21.15.167:0/4073617499 shutdown_connections 2024-05-11T08:07:02.327 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.322+0000 7f4bc873af40 1 -- 172.21.15.167:0/4073617499 >> 172.21.15.167:0/4073617499 conn(0x55979789a950 msgr2=0x5597979427c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:07:02.327 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.322+0000 7f4bc873af40 1 -- 172.21.15.167:0/4073617499 shutdown_connections 2024-05-11T08:07:02.327 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.322+0000 7f4bc873af40 1 -- 172.21.15.167:0/4073617499 wait complete. 2024-05-11T08:07:02.327 INFO:teuthology.orchestra.run.smithi167.stdout:@@ 2024-05-11T08:07:02.328 DEBUG:teuthology.orchestra.run.smithi167:> ceph-dencoder type file_layout_t import - decode dump_json 2024-05-11T08:07:02.415 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-05-11T08:07:02.415 INFO:teuthology.orchestra.run.smithi167.stdout: "stripe_unit": 4194304, 2024-05-11T08:07:02.415 INFO:teuthology.orchestra.run.smithi167.stdout: "stripe_count": 1, 2024-05-11T08:07:02.415 INFO:teuthology.orchestra.run.smithi167.stdout: "object_size": 4194304, 2024-05-11T08:07:02.415 INFO:teuthology.orchestra.run.smithi167.stdout: "pool_id": 6, 2024-05-11T08:07:02.415 INFO:teuthology.orchestra.run.smithi167.stdout: "pool_ns": "" 2024-05-11T08:07:02.415 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-05-11T08:07:02.415 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T08:07:02.418 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T08:07:02.418 DEBUG:teuthology.orchestra.run.smithi167:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parent_b) 2024-05-11T08:07:02.475 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T08:07:02.475 DEBUG:teuthology.orchestra.run.smithi167:> (cd /home/ubuntu/cephtest/mnt.0 && exec mv parent_a/alpha parent_b/alpha) 2024-05-11T08:07:02.534 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-11T08:07:02.677 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.677+0000 7fa7bc22c640 1 Processor -- start 2024-05-11T08:07:02.677 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.677+0000 7fa7bc22c640 1 -- start start 2024-05-11T08:07:02.679 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.678+0000 7fa7bc22c640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fa7b41488a0 0x7fa7b415f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:02.679 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.678+0000 7fa7bc22c640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa7b415fe50 0x7fa7b40b5a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:02.679 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.678+0000 7fa7bc22c640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7b4163e60 con 0x7fa7b40b5f50 2024-05-11T08:07:02.679 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.678+0000 7fa7bc22c640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7fa7b40b5580 con 0x7fa7b41488a0 2024-05-11T08:07:02.679 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.678+0000 7fa7bc22c640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fa7b40b4f70 con 0x7fa7b415fe50 2024-05-11T08:07:02.679 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.678+0000 7fa7b9fc8640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fa7b41488a0 0x7fa7b415f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:02.679 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.678+0000 7fa7ba7c9640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7fa7b40b5f50 0x7fa7b40710f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:48400/0 (socket says 172.21.15.167:48400) 2024-05-11T08:07:02.680 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.678+0000 7fa7ba7c9640 1 -- 172.21.15.167:0/3779381351 learned_addr learned my addr 172.21.15.167:0/3779381351 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T08:07:02.680 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.678+0000 7fa7b97c7640 1 --2- 172.21.15.167:0/3779381351 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa7b415fe50 0x7fa7b40b5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:02.680 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.678+0000 7fa7b9fc8640 1 -- 172.21.15.167:0/3779381351 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa7b415fe50 msgr2=0x7fa7b40b5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:07:02.680 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.678+0000 7fa7b9fc8640 1 --2- 172.21.15.167:0/3779381351 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa7b415fe50 0x7fa7b40b5a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:02.680 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.678+0000 7fa7b9fc8640 1 -- 172.21.15.167:0/3779381351 >> v1:172.21.15.167:6789/0 conn(0x7fa7b40b5f50 legacy=0x7fa7b40710f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:02.680 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.678+0000 7fa7b9fc8640 1 -- 172.21.15.167:0/3779381351 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7b4163b80 con 0x7fa7b41488a0 2024-05-11T08:07:02.680 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.678+0000 7fa7b8fc6640 1 -- 172.21.15.167:0/3779381351 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3706622739 0 0) 0x7fa7b4163e60 con 0x7fa7b40b5f50 2024-05-11T08:07:02.680 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.678+0000 7fa7b97c7640 1 --2- 172.21.15.167:0/3779381351 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa7b415fe50 0x7fa7b40b5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T08:07:02.680 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.679+0000 7fa7b9fc8640 1 --2- 172.21.15.167:0/3779381351 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa7b41488a0 0x7fa7b415f910 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fa7b00097a0 tx=0x7fa7b002a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35da95336b8935ed server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:02.680 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.679+0000 7fa7b8fc6640 1 -- 172.21.15.167:0/3779381351 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7b0037070 con 0x7fa7b41488a0 2024-05-11T08:07:02.680 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.679+0000 7fa7b8fc6640 1 -- 172.21.15.167:0/3779381351 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7b002ae50 con 0x7fa7b41488a0 2024-05-11T08:07:02.680 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.679+0000 7fa7b8fc6640 1 -- 172.21.15.167:0/3779381351 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7b0032660 con 0x7fa7b41488a0 2024-05-11T08:07:02.683 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.680+0000 7fa7bc22c640 1 -- 172.21.15.167:0/3779381351 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa7b41488a0 msgr2=0x7fa7b415f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:02.683 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.680+0000 7fa7bc22c640 1 --2- 172.21.15.167:0/3779381351 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa7b41488a0 0x7fa7b415f910 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fa7b00097a0 tx=0x7fa7b002a100 comp rx=0 tx=0).stop 2024-05-11T08:07:02.683 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.680+0000 7fa7bc22c640 1 -- 172.21.15.167:0/3779381351 shutdown_connections 2024-05-11T08:07:02.683 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.680+0000 7fa7bc22c640 1 --2- 172.21.15.167:0/3779381351 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa7b415fe50 0x7fa7b40b5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:02.683 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.680+0000 7fa7bc22c640 1 --2- 172.21.15.167:0/3779381351 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa7b41488a0 0x7fa7b415f910 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:02.683 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.680+0000 7fa7bc22c640 1 -- 172.21.15.167:0/3779381351 >> 172.21.15.167:0/3779381351 conn(0x7fa7b40d2550 msgr2=0x7fa7b40ce890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:07:02.683 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.680+0000 7fa7bc22c640 1 -- 172.21.15.167:0/3779381351 shutdown_connections 2024-05-11T08:07:02.684 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.680+0000 7fa7bc22c640 1 -- 172.21.15.167:0/3779381351 wait complete. 2024-05-11T08:07:02.684 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.681+0000 7fa7bc22c640 1 Processor -- start 2024-05-11T08:07:02.684 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.681+0000 7fa7bc22c640 1 -- start start 2024-05-11T08:07:02.684 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.682+0000 7fa7bc22c640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa7b40b5f50 0x7fa7b41f5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:02.684 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.682+0000 7fa7bc22c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa7b41488a0 0x7fa7b4142bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:02.684 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.682+0000 7fa7bc22c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa7b415fe50 0x7fa7b40b07b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:02.684 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.682+0000 7fa7bc22c640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa7b4148e00 con 0x7fa7b415fe50 2024-05-11T08:07:02.684 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.682+0000 7fa7bc22c640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa7b4148c80 con 0x7fa7b41488a0 2024-05-11T08:07:02.684 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.682+0000 7fa7bc22c640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fa7b4071800 con 0x7fa7b40b5f50 2024-05-11T08:07:02.684 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.682+0000 7fa7ba7c9640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa7b415fe50 0x7fa7b40b07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:02.684 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.682+0000 7fa7ba7c9640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa7b415fe50 0x7fa7b40b07b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:43246/0 (socket says 172.21.15.167:43246) 2024-05-11T08:07:02.684 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.682+0000 7fa7ba7c9640 1 -- 172.21.15.167:0/2332543341 learned_addr learned my addr 172.21.15.167:0/2332543341 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:07:02.684 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.682+0000 7fa7b9fc8640 1 --2- 172.21.15.167:0/2332543341 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa7b40b5f50 0x7fa7b41f5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:02.684 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.682+0000 7fa7b97c7640 1 --2- 172.21.15.167:0/2332543341 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa7b41488a0 0x7fa7b4142bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:02.685 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.682+0000 7fa7ba7c9640 1 -- 172.21.15.167:0/2332543341 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa7b40b5f50 msgr2=0x7fa7b41f5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:02.685 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.682+0000 7fa7ba7c9640 1 --2- 172.21.15.167:0/2332543341 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa7b40b5f50 0x7fa7b41f5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:02.685 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.682+0000 7fa7ba7c9640 1 -- 172.21.15.167:0/2332543341 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa7b41488a0 msgr2=0x7fa7b4142bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:02.685 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.682+0000 7fa7ba7c9640 1 --2- 172.21.15.167:0/2332543341 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa7b41488a0 0x7fa7b4142bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:02.685 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.682+0000 7fa7ba7c9640 1 -- 172.21.15.167:0/2332543341 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7b4143100 con 0x7fa7b415fe50 2024-05-11T08:07:02.685 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.682+0000 7fa7b9fc8640 1 --2- 172.21.15.167:0/2332543341 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa7b40b5f50 0x7fa7b41f5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T08:07:02.685 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.682+0000 7fa7ba7c9640 1 --2- 172.21.15.167:0/2332543341 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa7b415fe50 0x7fa7b40b07b0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac00c790 tx=0x7fa7ac00cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:02.685 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.682+0000 7fa7aa7fc640 1 -- 172.21.15.167:0/2332543341 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7ac018070 con 0x7fa7b415fe50 2024-05-11T08:07:02.685 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.682+0000 7fa7aa7fc640 1 -- 172.21.15.167:0/2332543341 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7ac00a640 con 0x7fa7b415fe50 2024-05-11T08:07:02.685 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.682+0000 7fa7bc22c640 1 -- 172.21.15.167:0/2332543341 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7b41433d0 con 0x7fa7b415fe50 2024-05-11T08:07:02.685 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.682+0000 7fa7aa7fc640 1 -- 172.21.15.167:0/2332543341 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7ac00aa50 con 0x7fa7b415fe50 2024-05-11T08:07:02.686 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.682+0000 7fa7bc22c640 1 -- 172.21.15.167:0/2332543341 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7b413d5f0 con 0x7fa7b415fe50 2024-05-11T08:07:02.686 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.683+0000 7fa7bc22c640 1 -- 172.21.15.167:0/2332543341 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7b4069590 con 0x7fa7b415fe50 2024-05-11T08:07:02.687 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.684+0000 7fa7aa7fc640 1 -- 172.21.15.167:0/2332543341 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fa7ac01b070 con 0x7fa7b415fe50 2024-05-11T08:07:02.687 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.684+0000 7fa7aa7fc640 1 --2- 172.21.15.167:0/2332543341 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fa7980762d0 0x7fa798078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:02.687 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.685+0000 7fa7aa7fc640 1 -- 172.21.15.167:0/2332543341 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6101+0+0 (secure 0 0 0) 0x7fa7ac096d60 con 0x7fa7b415fe50 2024-05-11T08:07:02.687 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.685+0000 7fa7b9fc8640 1 --2- 172.21.15.167:0/2332543341 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fa7980762d0 0x7fa798078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:02.687 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.685+0000 7fa7b9fc8640 1 --2- 172.21.15.167:0/2332543341 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fa7980762d0 0x7fa798078790 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa7b0034000 tx=0x7fa7b002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:02.689 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.689+0000 7fa7aa7fc640 1 -- 172.21.15.167:0/2332543341 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fa7ac060530 con 0x7fa7b415fe50 2024-05-11T08:07:02.904 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.904+0000 7fa7bc22c640 1 -- 172.21.15.167:0/2332543341 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa7b40d1220 con 0x7fa7b415fe50 2024-05-11T08:07:02.905 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.905+0000 7fa7aa7fc640 1 -- 172.21.15.167:0/2332543341 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5190 (secure 0 0 0) 0x7fa7ac05fe30 con 0x7fa7b415fe50 2024-05-11T08:07:02.905 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T08:07:02.905 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4604,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2674983673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":2674983673},{"type":"v1","addr":"172.21.15.167:6837","nonce":2674983673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4607,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6834/1018548066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1018548066},{"type":"v1","addr":"172.21.15.204:6834","nonce":1018548066}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4729,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3369722249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3369722249},{"type":"v1","addr":"172.21.15.167:6836","nonce":3369722249}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-11T08:06:47.943939+0000","modified":"2024-05-11T08:06:49.955430+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4750},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4750":{"gid":4750,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.204:6835/469864006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6833","nonce":469864006},{"type":"v1","addr":"172.21.15.204:6835","nonce":469864006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4750,"qdb_cluster":[4750]},"id":2}]} 2024-05-11T08:07:02.905 INFO:teuthology.orchestra.run.smithi167.stderr:dumped fsmap epoch 16 2024-05-11T08:07:02.907 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.907+0000 7fa7bc22c640 1 -- 172.21.15.167:0/2332543341 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fa7980762d0 msgr2=0x7fa798078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:02.907 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.907+0000 7fa7bc22c640 1 --2- 172.21.15.167:0/2332543341 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fa7980762d0 0x7fa798078790 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa7b0034000 tx=0x7fa7b002e000 comp rx=0 tx=0).stop 2024-05-11T08:07:02.907 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.907+0000 7fa7bc22c640 1 -- 172.21.15.167:0/2332543341 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa7b415fe50 msgr2=0x7fa7b40b07b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:02.907 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.907+0000 7fa7bc22c640 1 --2- 172.21.15.167:0/2332543341 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa7b415fe50 0x7fa7b40b07b0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac00c790 tx=0x7fa7ac00cc50 comp rx=0 tx=0).stop 2024-05-11T08:07:02.907 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.907+0000 7fa7bc22c640 1 -- 172.21.15.167:0/2332543341 shutdown_connections 2024-05-11T08:07:02.907 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.907+0000 7fa7bc22c640 1 --2- 172.21.15.167:0/2332543341 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fa7980762d0 0x7fa798078790 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:02.907 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.907+0000 7fa7bc22c640 1 --2- 172.21.15.167:0/2332543341 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa7b415fe50 0x7fa7b40b07b0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:02.908 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.907+0000 7fa7bc22c640 1 --2- 172.21.15.167:0/2332543341 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa7b41488a0 0x7fa7b4142bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:02.908 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.907+0000 7fa7bc22c640 1 --2- 172.21.15.167:0/2332543341 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa7b40b5f50 0x7fa7b41f5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:02.908 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.907+0000 7fa7bc22c640 1 -- 172.21.15.167:0/2332543341 >> 172.21.15.167:0/2332543341 conn(0x7fa7b40d2550 msgr2=0x7fa7b41464a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:07:02.908 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.907+0000 7fa7bc22c640 1 -- 172.21.15.167:0/2332543341 shutdown_connections 2024-05-11T08:07:02.908 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:02.907+0000 7fa7bc22c640 1 -- 172.21.15.167:0/2332543341 wait complete. 2024-05-11T08:07:02.918 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json flush journal 2024-05-11T08:07:03.046 INFO:teuthology.orchestra.run.smithi204.stdout:{"message":"","return_code":0} 2024-05-11T08:07:03.046 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-05-11T08:07:03.047 DEBUG:teuthology.orchestra.run.smithi167:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001fb.00000000 parent 2024-05-11T08:07:03.090 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.090+0000 7f7401470f40 1 Processor -- start 2024-05-11T08:07:03.090 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.090+0000 7f7401470f40 1 -- start start 2024-05-11T08:07:03.094 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.091+0000 7f7401470f40 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5638ec030fc0 0x5638ec101b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:03.095 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.091+0000 7f7401470f40 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5638ec033d30 0x5638ec0f0560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:03.095 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.091+0000 7f7401470f40 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x5638ec109760 0x5638ec0613b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:03.095 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.091+0000 7f7401470f40 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5638ec0313f0 con 0x5638ec033d30 2024-05-11T08:07:03.095 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.091+0000 7f7401470f40 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5638ebfd9f50 con 0x5638ec030fc0 2024-05-11T08:07:03.095 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.091+0000 7f7401470f40 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x5638ec07b8e0 con 0x5638ec109760 2024-05-11T08:07:03.095 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.091+0000 7f73f7fff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5638ec033d30 0x5638ec0f0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:03.095 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.091+0000 7f73f7fff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5638ec033d30 0x5638ec0f0560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:43258/0 (socket says 172.21.15.167:43258) 2024-05-11T08:07:03.095 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.091+0000 7f73f7fff640 1 -- 172.21.15.167:0/1999679121 learned_addr learned my addr 172.21.15.167:0/1999679121 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:07:03.095 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.091+0000 7f73fef6a640 1 --2- 172.21.15.167:0/1999679121 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5638ec030fc0 0x5638ec101b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:03.095 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.091+0000 7f73f7fff640 1 -- 172.21.15.167:0/1999679121 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x5638ec109760 msgr2=0x5638ec0613b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:07:03.095 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.091+0000 7f73f7fff640 1 --2- 172.21.15.167:0/1999679121 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x5638ec109760 0x5638ec0613b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:03.095 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.091+0000 7f73f7fff640 1 -- 172.21.15.167:0/1999679121 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5638ec030fc0 msgr2=0x5638ec101b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:07:03.095 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.091+0000 7f73f7fff640 1 --2- 172.21.15.167:0/1999679121 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5638ec030fc0 0x5638ec101b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:03.096 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.091+0000 7f73f7fff640 1 -- 172.21.15.167:0/1999679121 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638ec076590 con 0x5638ec033d30 2024-05-11T08:07:03.096 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.091+0000 7f73f7fff640 1 --2- 172.21.15.167:0/1999679121 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5638ec033d30 0x5638ec0f0560 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f73e8000c00 tx=0x7f73e8051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e40c6c0446f0a5d server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:03.096 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.091+0000 7f73fdf68640 1 -- 172.21.15.167:0/1999679121 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e8052a90 con 0x5638ec033d30 2024-05-11T08:07:03.096 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.091+0000 7f73fdf68640 1 -- 172.21.15.167:0/1999679121 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73e8052c50 con 0x5638ec033d30 2024-05-11T08:07:03.096 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.091+0000 7f73fdf68640 1 -- 172.21.15.167:0/1999679121 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e8067480 con 0x5638ec033d30 2024-05-11T08:07:03.096 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.092+0000 7f7401470f40 1 -- 172.21.15.167:0/1999679121 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5638ec033d30 msgr2=0x5638ec0f0560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:03.096 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.092+0000 7f7401470f40 1 --2- 172.21.15.167:0/1999679121 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5638ec033d30 0x5638ec0f0560 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f73e8000c00 tx=0x7f73e8051000 comp rx=0 tx=0).stop 2024-05-11T08:07:03.096 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.092+0000 7f7401470f40 1 -- 172.21.15.167:0/1999679121 shutdown_connections 2024-05-11T08:07:03.096 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.092+0000 7f7401470f40 1 --2- 172.21.15.167:0/1999679121 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x5638ec109760 0x5638ec0613b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:03.096 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.092+0000 7f7401470f40 1 --2- 172.21.15.167:0/1999679121 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5638ec033d30 0x5638ec0f0560 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:03.096 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.092+0000 7f7401470f40 1 --2- 172.21.15.167:0/1999679121 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5638ec030fc0 0x5638ec101b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:03.096 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.092+0000 7f7401470f40 1 -- 172.21.15.167:0/1999679121 >> 172.21.15.167:0/1999679121 conn(0x5638ec05e950 msgr2=0x5638ec05df80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:07:03.096 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.092+0000 7f7401470f40 1 -- 172.21.15.167:0/1999679121 shutdown_connections 2024-05-11T08:07:03.096 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.092+0000 7f7401470f40 1 -- 172.21.15.167:0/1999679121 wait complete. 2024-05-11T08:07:03.097 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.092+0000 7f7401470f40 1 Processor -- start 2024-05-11T08:07:03.097 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.092+0000 7f7401470f40 1 -- start start 2024-05-11T08:07:03.097 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.093+0000 7f7401470f40 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5638ec030fc0 0x5638ec0f14d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:03.097 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.093+0000 7f7401470f40 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x5638ec033d30 0x5638ec1a39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:03.097 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.093+0000 7f7401470f40 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5638ec109760 0x5638ec1a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:03.097 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.093+0000 7f7401470f40 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5638ec0339c0 con 0x5638ec109760 2024-05-11T08:07:03.097 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.093+0000 7f7401470f40 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5638ec033530 con 0x5638ec030fc0 2024-05-11T08:07:03.097 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.093+0000 7f7401470f40 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x5638ec108d70 con 0x5638ec033d30 2024-05-11T08:07:03.097 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.093+0000 7f73ff76b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5638ec109760 0x5638ec1a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:03.097 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.093+0000 7f73ff76b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5638ec109760 0x5638ec1a3ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:43264/0 (socket says 172.21.15.167:43264) 2024-05-11T08:07:03.097 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.093+0000 7f73ff76b640 1 -- 172.21.15.167:0/2525024175 learned_addr learned my addr 172.21.15.167:0/2525024175 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:07:03.097 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.093+0000 7f73fef6a640 1 --2- 172.21.15.167:0/2525024175 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5638ec030fc0 0x5638ec0f14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:03.097 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.093+0000 7f73f7fff640 1 --2- 172.21.15.167:0/2525024175 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x5638ec033d30 0x5638ec1a39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:03.097 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.093+0000 7f73ff76b640 1 -- 172.21.15.167:0/2525024175 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x5638ec033d30 msgr2=0x5638ec1a39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:03.098 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.093+0000 7f73ff76b640 1 --2- 172.21.15.167:0/2525024175 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x5638ec033d30 0x5638ec1a39b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:03.098 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.094+0000 7f73ff76b640 1 -- 172.21.15.167:0/2525024175 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5638ec030fc0 msgr2=0x5638ec0f14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:03.098 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.094+0000 7f73ff76b640 1 --2- 172.21.15.167:0/2525024175 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5638ec030fc0 0x5638ec0f14d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:03.098 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.094+0000 7f73ff76b640 1 -- 172.21.15.167:0/2525024175 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638ec0ef190 con 0x5638ec109760 2024-05-11T08:07:03.098 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.094+0000 7f73ff76b640 1 --2- 172.21.15.167:0/2525024175 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5638ec109760 0x5638ec1a3ef0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f73f8005f60 tx=0x7f73f8005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:03.098 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.094+0000 7f73fcf66640 1 -- 172.21.15.167:0/2525024175 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73f8008e60 con 0x5638ec109760 2024-05-11T08:07:03.098 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.094+0000 7f73fcf66640 1 -- 172.21.15.167:0/2525024175 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73f8015070 con 0x5638ec109760 2024-05-11T08:07:03.098 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.094+0000 7f7401470f40 1 -- 172.21.15.167:0/2525024175 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5638ec101ac0 con 0x5638ec109760 2024-05-11T08:07:03.098 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.094+0000 7f73fcf66640 1 -- 172.21.15.167:0/2525024175 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73f80108a0 con 0x5638ec109760 2024-05-11T08:07:03.098 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.094+0000 7f7401470f40 1 -- 172.21.15.167:0/2525024175 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638ec0ef540 con 0x5638ec109760 2024-05-11T08:07:03.100 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.095+0000 7f73fcf66640 1 -- 172.21.15.167:0/2525024175 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f73f8010a40 con 0x5638ec109760 2024-05-11T08:07:03.100 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.096+0000 7f73fcf66640 1 --2- 172.21.15.167:0/2525024175 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f73e0074cb0 0x7f73e0077170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:03.100 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.096+0000 7f73fcf66640 1 -- 172.21.15.167:0/2525024175 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6101+0+0 (secure 0 0 0) 0x7f73f8096140 con 0x5638ec109760 2024-05-11T08:07:03.100 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.096+0000 7f73fef6a640 1 --2- 172.21.15.167:0/2525024175 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f73e0074cb0 0x7f73e0077170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:03.100 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.096+0000 7f7401470f40 1 --2- 172.21.15.167:0/2525024175 >> [v2:172.21.15.167:6824/453689293,v1:172.21.15.167:6825/453689293] conn(0x5638ec0f1a10 0x5638ec10aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:03.100 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.096+0000 7f7401470f40 1 -- 172.21.15.167:0/2525024175 --> [v2:172.21.15.167:6824/453689293,v1:172.21.15.167:6825/453689293] -- osd_op(unknown.0.0:1 6.13 6:ca8fdbba:::100000001fb.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41) -- 0x5638ec0efa90 con 0x5638ec0f1a10 2024-05-11T08:07:03.100 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.096+0000 7f73f7fff640 1 --2- 172.21.15.167:0/2525024175 >> [v2:172.21.15.167:6824/453689293,v1:172.21.15.167:6825/453689293] conn(0x5638ec0f1a10 0x5638ec10aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:03.101 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.096+0000 7f73fef6a640 1 --2- 172.21.15.167:0/2525024175 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f73e0074cb0 0x7f73e0077170 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x5638ec076a80 tx=0x7f73f000d040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:03.101 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.097+0000 7f73f7fff640 1 --2- 172.21.15.167:0/2525024175 >> [v2:172.21.15.167:6824/453689293,v1:172.21.15.167:6825/453689293] conn(0x5638ec0f1a10 0x5638ec10aa60 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:03.101 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.097+0000 7f73f7fff640 1 -- 172.21.15.167:0/2525024175 <== osd.3 v2:172.21.15.167:6824/453689293 1 ==== osd_op_reply(1 100000001fb.00000000 [getxattr (95) out=95b] v0'0 uv2 ondisk = 0) ==== 164+0+95 (crc 0 0 0) 0x7f73e8050050 con 0x5638ec0f1a10 2024-05-11T08:07:03.101 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.097+0000 7f7401470f40 1 -- 172.21.15.167:0/2525024175 >> [v2:172.21.15.167:6824/453689293,v1:172.21.15.167:6825/453689293] conn(0x5638ec0f1a10 msgr2=0x5638ec10aa60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:03.101 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.097+0000 7f7401470f40 1 --2- 172.21.15.167:0/2525024175 >> [v2:172.21.15.167:6824/453689293,v1:172.21.15.167:6825/453689293] conn(0x5638ec0f1a10 0x5638ec10aa60 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:03.101 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.097+0000 7f7401470f40 1 -- 172.21.15.167:0/2525024175 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f73e0074cb0 msgr2=0x7f73e0077170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:03.101 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.097+0000 7f7401470f40 1 --2- 172.21.15.167:0/2525024175 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f73e0074cb0 0x7f73e0077170 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x5638ec076a80 tx=0x7f73f000d040 comp rx=0 tx=0).stop 2024-05-11T08:07:03.101 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.097+0000 7f7401470f40 1 -- 172.21.15.167:0/2525024175 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5638ec109760 msgr2=0x5638ec1a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:03.101 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.097+0000 7f7401470f40 1 --2- 172.21.15.167:0/2525024175 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5638ec109760 0x5638ec1a3ef0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f73f8005f60 tx=0x7f73f8005f90 comp rx=0 tx=0).stop 2024-05-11T08:07:03.101 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.097+0000 7f73ff76b640 1 -- 172.21.15.167:0/2525024175 reap_dead start 2024-05-11T08:07:03.101 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.097+0000 7f7401470f40 1 -- 172.21.15.167:0/2525024175 shutdown_connections 2024-05-11T08:07:03.101 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.097+0000 7f7401470f40 1 -- 172.21.15.167:0/2525024175 >> 172.21.15.167:0/2525024175 conn(0x5638ec05e950 msgr2=0x5638ec107e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:07:03.101 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.097+0000 7f7401470f40 1 -- 172.21.15.167:0/2525024175 shutdown_connections 2024-05-11T08:07:03.101 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.097+0000 7f7401470f40 1 -- 172.21.15.167:0/2525024175 wait complete. 2024-05-11T08:07:03.102 INFO:teuthology.orchestra.run.smithi167.stdout:Y��alphaparent_b 2024-05-11T08:07:03.102 DEBUG:teuthology.orchestra.run.smithi167:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2024-05-11T08:07:03.190 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-05-11T08:07:03.190 INFO:teuthology.orchestra.run.smithi167.stdout: "ino": 1099511628283, 2024-05-11T08:07:03.190 INFO:teuthology.orchestra.run.smithi167.stdout: "ancestors": [ 2024-05-11T08:07:03.190 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T08:07:03.190 INFO:teuthology.orchestra.run.smithi167.stdout: "dirino": 1099511628284, 2024-05-11T08:07:03.190 INFO:teuthology.orchestra.run.smithi167.stdout: "dname": "alpha", 2024-05-11T08:07:03.190 INFO:teuthology.orchestra.run.smithi167.stdout: "version": 3 2024-05-11T08:07:03.191 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T08:07:03.191 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T08:07:03.191 INFO:teuthology.orchestra.run.smithi167.stdout: "dirino": 1, 2024-05-11T08:07:03.191 INFO:teuthology.orchestra.run.smithi167.stdout: "dname": "parent_b", 2024-05-11T08:07:03.191 INFO:teuthology.orchestra.run.smithi167.stdout: "version": 25 2024-05-11T08:07:03.191 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T08:07:03.191 INFO:teuthology.orchestra.run.smithi167.stdout: ], 2024-05-11T08:07:03.191 INFO:teuthology.orchestra.run.smithi167.stdout: "pool": 6, 2024-05-11T08:07:03.191 INFO:teuthology.orchestra.run.smithi167.stdout: "old_pools": [] 2024-05-11T08:07:03.191 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-05-11T08:07:03.191 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T08:07:03.193 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create data_new --pg_num_min 64 2024-05-11T08:07:03.338 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.337+0000 7f8a6d0cf640 1 Processor -- start 2024-05-11T08:07:03.338 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.337+0000 7f8a6d0cf640 1 -- start start 2024-05-11T08:07:03.340 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.338+0000 7f8a6d0cf640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f8a68148d50 0x7f8a680cd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:03.340 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.338+0000 7f8a6d0cf640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8a680d2630 0x7f8a68162e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:03.340 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.338+0000 7f8a6d0cf640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f8a681633b0 0x7f8a680c7d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:03.340 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.338+0000 7f8a6d0cf640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f8a680cda40 con 0x7f8a68148d50 2024-05-11T08:07:03.340 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.338+0000 7f8a6d0cf640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f8a681637e0 con 0x7f8a681633b0 2024-05-11T08:07:03.340 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.338+0000 7f8a6d0cf640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f8a680cfad0 con 0x7f8a680d2630 2024-05-11T08:07:03.340 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.338+0000 7f8a66d9d640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f8a68148d50 0x7f8a680cd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:03.340 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.339+0000 7f8a66d9d640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f8a68148d50 0x7f8a680cd500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:43266/0 (socket says 172.21.15.167:43266) 2024-05-11T08:07:03.340 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.339+0000 7f8a66d9d640 1 -- 172.21.15.167:0/3801370274 learned_addr learned my addr 172.21.15.167:0/3801370274 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:07:03.340 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.339+0000 7f8a6659c640 1 --2- 172.21.15.167:0/3801370274 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8a680d2630 0x7f8a68162e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:03.340 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.339+0000 7f8a6759e640 1 --2- 172.21.15.167:0/3801370274 >> v2:172.21.15.204:3300/0 conn(0x7f8a681633b0 0x7f8a680c7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:03.341 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.339+0000 7f8a66d9d640 1 -- 172.21.15.167:0/3801370274 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8a680d2630 msgr2=0x7f8a68162e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:07:03.341 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.339+0000 7f8a66d9d640 1 --2- 172.21.15.167:0/3801370274 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8a680d2630 0x7f8a68162e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:03.341 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.339+0000 7f8a66d9d640 1 -- 172.21.15.167:0/3801370274 >> v2:172.21.15.204:3300/0 conn(0x7f8a681633b0 msgr2=0x7f8a680c7d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:07:03.341 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.339+0000 7f8a66d9d640 1 --2- 172.21.15.167:0/3801370274 >> v2:172.21.15.204:3300/0 conn(0x7f8a681633b0 0x7f8a680c7d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:03.341 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.339+0000 7f8a66d9d640 1 -- 172.21.15.167:0/3801370274 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a680cfdb0 con 0x7f8a68148d50 2024-05-11T08:07:03.341 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.340+0000 7f8a66d9d640 1 --2- 172.21.15.167:0/3801370274 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8a68148d50 0x7f8a680cd500 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f8a680c8f80 tx=0x7f8a5802a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cdcd83a3a7e8acc server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:03.341 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.340+0000 7f8a65d9b640 1 -- 172.21.15.167:0/3801370274 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a58037070 con 0x7f8a68148d50 2024-05-11T08:07:03.341 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.340+0000 7f8a65d9b640 1 -- 172.21.15.167:0/3801370274 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a5802ae10 con 0x7f8a68148d50 2024-05-11T08:07:03.341 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.340+0000 7f8a65d9b640 1 -- 172.21.15.167:0/3801370274 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a58032660 con 0x7f8a68148d50 2024-05-11T08:07:03.344 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.340+0000 7f8a6d0cf640 1 -- 172.21.15.167:0/3801370274 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8a68148d50 msgr2=0x7f8a680cd500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:03.345 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.340+0000 7f8a6d0cf640 1 --2- 172.21.15.167:0/3801370274 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8a68148d50 0x7f8a680cd500 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f8a680c8f80 tx=0x7f8a5802a100 comp rx=0 tx=0).stop 2024-05-11T08:07:03.345 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.340+0000 7f8a6d0cf640 1 -- 172.21.15.167:0/3801370274 shutdown_connections 2024-05-11T08:07:03.345 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.340+0000 7f8a6d0cf640 1 --2- 172.21.15.167:0/3801370274 >> v2:172.21.15.204:3300/0 conn(0x7f8a681633b0 0x7f8a680c7d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:03.345 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.340+0000 7f8a6d0cf640 1 --2- 172.21.15.167:0/3801370274 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8a680d2630 0x7f8a68162e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:03.345 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.340+0000 7f8a6d0cf640 1 --2- 172.21.15.167:0/3801370274 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8a68148d50 0x7f8a680cd500 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:03.345 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.340+0000 7f8a6d0cf640 1 -- 172.21.15.167:0/3801370274 >> 172.21.15.167:0/3801370274 conn(0x7f8a680b4ae0 msgr2=0x7f8a680b4ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:07:03.345 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.340+0000 7f8a6d0cf640 1 -- 172.21.15.167:0/3801370274 shutdown_connections 2024-05-11T08:07:03.345 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.340+0000 7f8a6d0cf640 1 -- 172.21.15.167:0/3801370274 wait complete. 2024-05-11T08:07:03.345 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.342+0000 7f8a6d0cf640 1 Processor -- start 2024-05-11T08:07:03.345 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.342+0000 7f8a6d0cf640 1 -- start start 2024-05-11T08:07:03.345 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.342+0000 7f8a6d0cf640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a680d2630 0x7f8a6815e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:03.345 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.342+0000 7f8a6d0cf640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8a68148d50 0x7f8a680cbea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:03.345 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.342+0000 7f8a6d0cf640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8a681633b0 0x7f8a680cafc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:03.346 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.343+0000 7f8a6d0cf640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8a680b5de0 con 0x7f8a68148d50 2024-05-11T08:07:03.346 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.343+0000 7f8a6d0cf640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8a680d2a90 con 0x7f8a680d2630 2024-05-11T08:07:03.346 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.343+0000 7f8a6d0cf640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f8a680c8290 con 0x7f8a681633b0 2024-05-11T08:07:03.346 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.343+0000 7f8a6659c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8a68148d50 0x7f8a680cbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:03.346 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.343+0000 7f8a6659c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8a68148d50 0x7f8a680cbea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:43270/0 (socket says 172.21.15.167:43270) 2024-05-11T08:07:03.346 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.343+0000 7f8a66d9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a680d2630 0x7f8a6815e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:03.346 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.343+0000 7f8a6659c640 1 -- 172.21.15.167:0/193095790 learned_addr learned my addr 172.21.15.167:0/193095790 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:07:03.346 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.343+0000 7f8a6759e640 1 --2- 172.21.15.167:0/193095790 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8a681633b0 0x7f8a680cafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:03.346 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.343+0000 7f8a6659c640 1 -- 172.21.15.167:0/193095790 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8a681633b0 msgr2=0x7f8a680cafc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:03.346 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.343+0000 7f8a6659c640 1 --2- 172.21.15.167:0/193095790 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8a681633b0 0x7f8a680cafc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:03.346 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.343+0000 7f8a6659c640 1 -- 172.21.15.167:0/193095790 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a680d2630 msgr2=0x7f8a6815e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:03.346 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.343+0000 7f8a6659c640 1 --2- 172.21.15.167:0/193095790 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a680d2630 0x7f8a6815e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:03.346 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.343+0000 7f8a6659c640 1 -- 172.21.15.167:0/193095790 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a6815fb90 con 0x7f8a68148d50 2024-05-11T08:07:03.347 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.343+0000 7f8a6659c640 1 --2- 172.21.15.167:0/193095790 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8a68148d50 0x7f8a680cbea0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f8a5c00e890 tx=0x7f8a5c00ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:03.347 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.343+0000 7f8a4f7fe640 1 -- 172.21.15.167:0/193095790 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a5c017070 con 0x7f8a68148d50 2024-05-11T08:07:03.347 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.343+0000 7f8a4f7fe640 1 -- 172.21.15.167:0/193095790 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a5c00c940 con 0x7f8a68148d50 2024-05-11T08:07:03.347 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.343+0000 7f8a4f7fe640 1 -- 172.21.15.167:0/193095790 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a5c00cd10 con 0x7f8a68148d50 2024-05-11T08:07:03.347 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.343+0000 7f8a6d0cf640 1 -- 172.21.15.167:0/193095790 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a6815fe70 con 0x7f8a68148d50 2024-05-11T08:07:03.347 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.344+0000 7f8a6d0cf640 1 -- 172.21.15.167:0/193095790 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a68160430 con 0x7f8a68148d50 2024-05-11T08:07:03.347 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.344+0000 7f8a6d0cf640 1 -- 172.21.15.167:0/193095790 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a680752e0 con 0x7f8a68148d50 2024-05-11T08:07:03.348 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.345+0000 7f8a4f7fe640 1 -- 172.21.15.167:0/193095790 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f8a5c01a070 con 0x7f8a68148d50 2024-05-11T08:07:03.348 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.346+0000 7f8a4f7fe640 1 --2- 172.21.15.167:0/193095790 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f8a540762d0 0x7f8a54078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:03.348 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.346+0000 7f8a4f7fe640 1 -- 172.21.15.167:0/193095790 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6101+0+0 (secure 0 0 0) 0x7f8a5c097a90 con 0x7f8a68148d50 2024-05-11T08:07:03.348 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.346+0000 7f8a66d9d640 1 --2- 172.21.15.167:0/193095790 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f8a540762d0 0x7f8a54078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:03.348 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.346+0000 7f8a66d9d640 1 --2- 172.21.15.167:0/193095790 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f8a540762d0 0x7f8a54078790 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f8a58034000 tx=0x7f8a5802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:03.350 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.350+0000 7f8a4f7fe640 1 -- 172.21.15.167:0/193095790 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f8a5c060c10 con 0x7f8a68148d50 2024-05-11T08:07:03.537 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.537+0000 7f8a6d0cf640 1 -- 172.21.15.167:0/193095790 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "data_new", "pg_num_min": 64} v 0) -- 0x7f8a680b8e60 con 0x7f8a68148d50 2024-05-11T08:07:03.655 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.654+0000 7f8a4f7fe640 1 -- 172.21.15.167:0/193095790 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "data_new", "pg_num_min": 64}]=0 pool 'data_new' created v42) ==== 124+0+0 (secure 0 0 0) 0x7f8a5c064f10 con 0x7f8a68148d50 2024-05-11T08:07:03.655 INFO:teuthology.orchestra.run.smithi167.stderr:pool 'data_new' created 2024-05-11T08:07:03.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.658+0000 7f8a6d0cf640 1 -- 172.21.15.167:0/193095790 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f8a540762d0 msgr2=0x7f8a54078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:03.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.658+0000 7f8a6d0cf640 1 --2- 172.21.15.167:0/193095790 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f8a540762d0 0x7f8a54078790 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f8a58034000 tx=0x7f8a5802e000 comp rx=0 tx=0).stop 2024-05-11T08:07:03.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.658+0000 7f8a6d0cf640 1 -- 172.21.15.167:0/193095790 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8a68148d50 msgr2=0x7f8a680cbea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:03.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.658+0000 7f8a6d0cf640 1 --2- 172.21.15.167:0/193095790 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8a68148d50 0x7f8a680cbea0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f8a5c00e890 tx=0x7f8a5c00ed50 comp rx=0 tx=0).stop 2024-05-11T08:07:03.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.658+0000 7f8a6d0cf640 1 -- 172.21.15.167:0/193095790 shutdown_connections 2024-05-11T08:07:03.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.658+0000 7f8a6d0cf640 1 --2- 172.21.15.167:0/193095790 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f8a540762d0 0x7f8a54078790 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:03.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.658+0000 7f8a6d0cf640 1 --2- 172.21.15.167:0/193095790 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8a681633b0 0x7f8a680cafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:03.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.658+0000 7f8a6d0cf640 1 --2- 172.21.15.167:0/193095790 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8a68148d50 0x7f8a680cbea0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:03.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.658+0000 7f8a6d0cf640 1 --2- 172.21.15.167:0/193095790 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a680d2630 0x7f8a6815e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:03.659 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.658+0000 7f8a6d0cf640 1 -- 172.21.15.167:0/193095790 >> 172.21.15.167:0/193095790 conn(0x7f8a680b4ae0 msgr2=0x7f8a680b37a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:07:03.659 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.658+0000 7f8a6d0cf640 1 -- 172.21.15.167:0/193095790 shutdown_connections 2024-05-11T08:07:03.659 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.658+0000 7f8a6d0cf640 1 -- 172.21.15.167:0/193095790 wait complete. 2024-05-11T08:07:03.672 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs data_new 2024-05-11T08:07:03.774 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.774+0000 7f959c39d640 1 Processor -- start 2024-05-11T08:07:03.774 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.774+0000 7f959c39d640 1 -- start start 2024-05-11T08:07:03.776 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.775+0000 7f959c39d640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f95941627f0 0x7f9594160020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:03.776 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.775+0000 7f959c39d640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f9594071800 0x7f9594160930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:03.776 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.775+0000 7f959c39d640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95940b4fe0 con 0x7f9594163a20 2024-05-11T08:07:03.776 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.775+0000 7f959c39d640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f95940b52c0 con 0x7f95941627f0 2024-05-11T08:07:03.776 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.775+0000 7f959c39d640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f95940b55a0 con 0x7f9594071800 2024-05-11T08:07:03.776 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.775+0000 7f959a139640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f95941627f0 0x7f9594160020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:03.776 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.775+0000 7f9599938640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f9594163a20 0x7f95940710f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:48414/0 (socket says 172.21.15.167:48414) 2024-05-11T08:07:03.776 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.775+0000 7f9599938640 1 -- 172.21.15.167:0/911451564 learned_addr learned my addr 172.21.15.167:0/911451564 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T08:07:03.777 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.775+0000 7f959a93a640 1 --2- 172.21.15.167:0/911451564 >> v2:172.21.15.204:3300/0 conn(0x7f9594071800 0x7f9594160930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:03.777 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.775+0000 7f959a139640 1 -- 172.21.15.167:0/911451564 >> v2:172.21.15.204:3300/0 conn(0x7f9594071800 msgr2=0x7f9594160930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:07:03.777 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.775+0000 7f959a139640 1 --2- 172.21.15.167:0/911451564 >> v2:172.21.15.204:3300/0 conn(0x7f9594071800 0x7f9594160930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:03.777 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.775+0000 7f959a139640 1 -- 172.21.15.167:0/911451564 >> v1:172.21.15.167:6789/0 conn(0x7f9594163a20 legacy=0x7f95940710f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:03.777 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.775+0000 7f9599137640 1 -- 172.21.15.167:0/911451564 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1322178368 0 0) 0x7f95940b4fe0 con 0x7f9594163a20 2024-05-11T08:07:03.777 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.775+0000 7f959a139640 1 -- 172.21.15.167:0/911451564 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95940b5880 con 0x7f95941627f0 2024-05-11T08:07:03.777 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.775+0000 7f959a93a640 1 --2- 172.21.15.167:0/911451564 >> v2:172.21.15.204:3300/0 conn(0x7f9594071800 0x7f9594160930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T08:07:03.777 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.776+0000 7f959a139640 1 --2- 172.21.15.167:0/911451564 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f95941627f0 0x7f9594160020 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f959415d330 tx=0x7f959002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=252b98310d7bba97 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:03.777 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.776+0000 7f9599137640 1 -- 172.21.15.167:0/911451564 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9590037070 con 0x7f95941627f0 2024-05-11T08:07:03.777 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.776+0000 7f9599137640 1 -- 172.21.15.167:0/911451564 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9590032440 con 0x7f95941627f0 2024-05-11T08:07:03.777 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.776+0000 7f9599137640 1 -- 172.21.15.167:0/911451564 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f959002ae60 con 0x7f95941627f0 2024-05-11T08:07:03.780 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.777+0000 7f959c39d640 1 -- 172.21.15.167:0/911451564 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f95941627f0 msgr2=0x7f9594160020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:03.780 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.777+0000 7f959c39d640 1 --2- 172.21.15.167:0/911451564 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f95941627f0 0x7f9594160020 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f959415d330 tx=0x7f959002a110 comp rx=0 tx=0).stop 2024-05-11T08:07:03.780 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.777+0000 7f959c39d640 1 -- 172.21.15.167:0/911451564 shutdown_connections 2024-05-11T08:07:03.780 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.777+0000 7f959c39d640 1 --2- 172.21.15.167:0/911451564 >> v2:172.21.15.204:3300/0 conn(0x7f9594071800 0x7f9594160930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:03.780 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.777+0000 7f959c39d640 1 --2- 172.21.15.167:0/911451564 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f95941627f0 0x7f9594160020 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:03.780 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.777+0000 7f959c39d640 1 -- 172.21.15.167:0/911451564 >> 172.21.15.167:0/911451564 conn(0x7f95940d2000 msgr2=0x7f95940d2410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:07:03.780 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.777+0000 7f959c39d640 1 -- 172.21.15.167:0/911451564 shutdown_connections 2024-05-11T08:07:03.781 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.777+0000 7f959c39d640 1 -- 172.21.15.167:0/911451564 wait complete. 2024-05-11T08:07:03.781 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.778+0000 7f959c39d640 1 Processor -- start 2024-05-11T08:07:03.781 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.778+0000 7f959c39d640 1 -- start start 2024-05-11T08:07:03.781 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.779+0000 7f959c39d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9594071800 0x7f95941534a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:03.781 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.779+0000 7f959c39d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f95941627f0 0x7f95941f5520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:03.781 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.779+0000 7f959c39d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9594163a20 0x7f9594151e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:03.781 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.779+0000 7f959c39d640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f95941606e0 con 0x7f9594071800 2024-05-11T08:07:03.781 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.779+0000 7f959c39d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f9594160560 con 0x7f9594163a20 2024-05-11T08:07:03.781 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.779+0000 7f959c39d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f9594160e70 con 0x7f95941627f0 2024-05-11T08:07:03.781 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.779+0000 7f959a139640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9594071800 0x7f95941534a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:03.781 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.779+0000 7f959a139640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9594071800 0x7f95941534a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:43278/0 (socket says 172.21.15.167:43278) 2024-05-11T08:07:03.781 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.779+0000 7f959a139640 1 -- 172.21.15.167:0/3415953974 learned_addr learned my addr 172.21.15.167:0/3415953974 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:07:03.781 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.779+0000 7f9599938640 1 --2- 172.21.15.167:0/3415953974 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f95941627f0 0x7f95941f5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:03.781 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.779+0000 7f959a93a640 1 --2- 172.21.15.167:0/3415953974 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9594163a20 0x7f9594151e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:03.782 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.779+0000 7f959a139640 1 -- 172.21.15.167:0/3415953974 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f95941627f0 msgr2=0x7f95941f5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:03.782 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.779+0000 7f959a139640 1 --2- 172.21.15.167:0/3415953974 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f95941627f0 0x7f95941f5520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:03.782 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.779+0000 7f959a139640 1 -- 172.21.15.167:0/3415953974 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9594163a20 msgr2=0x7f9594151e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:03.782 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.779+0000 7f959a139640 1 --2- 172.21.15.167:0/3415953974 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9594163a20 0x7f9594151e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:03.782 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.779+0000 7f959a139640 1 -- 172.21.15.167:0/3415953974 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9594153b60 con 0x7f9594071800 2024-05-11T08:07:03.782 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.779+0000 7f959a139640 1 --2- 172.21.15.167:0/3415953974 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9594071800 0x7f95941534a0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f9590009890 tx=0x7f95900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:03.782 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.779+0000 7f958affd640 1 -- 172.21.15.167:0/3415953974 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9590037040 con 0x7f9594071800 2024-05-11T08:07:03.782 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.779+0000 7f958affd640 1 -- 172.21.15.167:0/3415953974 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9590042070 con 0x7f9594071800 2024-05-11T08:07:03.782 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.779+0000 7f959c39d640 1 -- 172.21.15.167:0/3415953974 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9594153e40 con 0x7f9594071800 2024-05-11T08:07:03.782 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.779+0000 7f958affd640 1 -- 172.21.15.167:0/3415953974 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9590030bd0 con 0x7f9594071800 2024-05-11T08:07:03.782 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.780+0000 7f959c39d640 1 -- 172.21.15.167:0/3415953974 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9594154400 con 0x7f9594071800 2024-05-11T08:07:03.782 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.780+0000 7f959c39d640 1 -- 172.21.15.167:0/3415953974 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9594069520 con 0x7f9594071800 2024-05-11T08:07:03.783 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.781+0000 7f958affd640 1 -- 172.21.15.167:0/3415953974 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f9590047460 con 0x7f9594071800 2024-05-11T08:07:03.783 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.782+0000 7f958affd640 1 --2- 172.21.15.167:0/3415953974 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f95780762d0 0x7f9578078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:03.783 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.782+0000 7f958affd640 1 -- 172.21.15.167:0/3415953974 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6464+0+0 (secure 0 0 0) 0x7f95900b8440 con 0x7f9594071800 2024-05-11T08:07:03.783 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.782+0000 7f9599938640 1 --2- 172.21.15.167:0/3415953974 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f95780762d0 0x7f9578078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:03.783 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.782+0000 7f9599938640 1 --2- 172.21.15.167:0/3415953974 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f95780762d0 0x7f9578078790 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f959414fed0 tx=0x7f9584007de0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:03.786 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:03.786+0000 7f958affd640 1 -- 172.21.15.167:0/3415953974 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f9590032e20 con 0x7f9594071800 2024-05-11T08:07:04.005 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:04.005+0000 7f959c39d640 1 -- 172.21.15.167:0/3415953974 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "data_new"} v 0) -- 0x7f95940b6cd0 con 0x7f9594071800 2024-05-11T08:07:04.667 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:04.666+0000 7f958affd640 1 -- 172.21.15.167:0/3415953974 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "data_new"}]=0 added data pool 7 to fsmap v17) ==== 131+0+0 (secure 0 0 0) 0x7f9590081460 con 0x7f9594071800 2024-05-11T08:07:04.667 INFO:teuthology.orchestra.run.smithi167.stderr:added data pool 7 to fsmap 2024-05-11T08:07:04.671 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:04.671+0000 7f959c39d640 1 -- 172.21.15.167:0/3415953974 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f95780762d0 msgr2=0x7f9578078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:04.671 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:04.671+0000 7f959c39d640 1 --2- 172.21.15.167:0/3415953974 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f95780762d0 0x7f9578078790 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f959414fed0 tx=0x7f9584007de0 comp rx=0 tx=0).stop 2024-05-11T08:07:04.671 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:04.671+0000 7f959c39d640 1 -- 172.21.15.167:0/3415953974 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9594071800 msgr2=0x7f95941534a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:04.671 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:04.671+0000 7f959c39d640 1 --2- 172.21.15.167:0/3415953974 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9594071800 0x7f95941534a0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f9590009890 tx=0x7f95900046c0 comp rx=0 tx=0).stop 2024-05-11T08:07:04.673 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:04.672+0000 7f959c39d640 1 -- 172.21.15.167:0/3415953974 shutdown_connections 2024-05-11T08:07:04.673 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:04.672+0000 7f959c39d640 1 --2- 172.21.15.167:0/3415953974 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f95780762d0 0x7f9578078790 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:04.673 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:04.672+0000 7f959c39d640 1 --2- 172.21.15.167:0/3415953974 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9594163a20 0x7f9594151e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:04.673 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:04.672+0000 7f959c39d640 1 --2- 172.21.15.167:0/3415953974 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f95941627f0 0x7f95941f5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:04.673 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:04.672+0000 7f959c39d640 1 --2- 172.21.15.167:0/3415953974 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9594071800 0x7f95941534a0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:04.673 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:04.672+0000 7f959c39d640 1 -- 172.21.15.167:0/3415953974 >> 172.21.15.167:0/3415953974 conn(0x7f95940d2000 msgr2=0x7f959415ecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:07:04.673 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:04.673+0000 7f959c39d640 1 -- 172.21.15.167:0/3415953974 shutdown_connections 2024-05-11T08:07:04.673 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:04.673+0000 7f959c39d640 1 -- 172.21.15.167:0/3415953974 wait complete. 2024-05-11T08:07:04.685 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-11T08:07:04.786 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:04.786+0000 7fbe839da640 1 Processor -- start 2024-05-11T08:07:04.786 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:04.786+0000 7fbe839da640 1 -- start start 2024-05-11T08:07:04.788 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:04.787+0000 7fbe839da640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fbe7c0b7150 0x7fbe7c0d0890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:04.788 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:04.787+0000 7fbe839da640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fbe7c0b7530 0x7fbe7c15d650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:04.788 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:04.787+0000 7fbe839da640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe7c0d1980 con 0x7fbe7c15db90 2024-05-11T08:07:04.789 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:04.787+0000 7fbe839da640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7fbe7c15b090 con 0x7fbe7c0b7150 2024-05-11T08:07:04.789 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:04.787+0000 7fbe839da640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fbe7c15e2f0 con 0x7fbe7c0b7530 2024-05-11T08:07:04.789 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:04.787+0000 7fbe81776640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fbe7c0b7150 0x7fbe7c0d0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:04.789 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:04.787+0000 7fbe81f77640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7fbe7c15db90 0x7fbe7c070730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:47352/0 (socket says 172.21.15.167:47352) 2024-05-11T08:07:04.789 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:04.787+0000 7fbe81f77640 1 -- 172.21.15.167:0/1376586490 learned_addr learned my addr 172.21.15.167:0/1376586490 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T08:07:04.789 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:04.787+0000 7fbe80f75640 1 --2- 172.21.15.167:0/1376586490 >> v2:172.21.15.204:3300/0 conn(0x7fbe7c0b7530 0x7fbe7c15d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:04.789 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:04.788+0000 7fbe81776640 1 -- 172.21.15.167:0/1376586490 >> v2:172.21.15.204:3300/0 conn(0x7fbe7c0b7530 msgr2=0x7fbe7c15d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:07:04.789 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:04.788+0000 7fbe81776640 1 --2- 172.21.15.167:0/1376586490 >> v2:172.21.15.204:3300/0 conn(0x7fbe7c0b7530 0x7fbe7c15d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:04.789 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:04.788+0000 7fbe81776640 1 -- 172.21.15.167:0/1376586490 >> v1:172.21.15.167:6789/0 conn(0x7fbe7c15db90 legacy=0x7fbe7c070730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:04.789 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:04.788+0000 7fbe73fff640 1 -- 172.21.15.167:0/1376586490 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2546346043 0 0) 0x7fbe74001110 con 0x7fbe7c15db90 2024-05-11T08:07:04.789 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:04.788+0000 7fbe81776640 1 -- 172.21.15.167:0/1376586490 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe7c15e5d0 con 0x7fbe7c0b7150 2024-05-11T08:07:04.789 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:04.788+0000 7fbe81776640 1 --2- 172.21.15.167:0/1376586490 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe7c0b7150 0x7fbe7c0d0890 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fbe7c0d2cd0 tx=0x7fbe7802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdc0fe673bfba250 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:04.789 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:04.788+0000 7fbe73fff640 1 -- 172.21.15.167:0/1376586490 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe78007740 con 0x7fbe7c0b7150 2024-05-11T08:07:04.789 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:04.788+0000 7fbe73fff640 1 -- 172.21.15.167:0/1376586490 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe780078e0 con 0x7fbe7c0b7150 2024-05-11T08:07:04.790 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:04.788+0000 7fbe73fff640 1 -- 172.21.15.167:0/1376586490 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe78007cb0 con 0x7fbe7c0b7150 2024-05-11T08:07:04.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:04.789+0000 7fbe839da640 1 -- 172.21.15.167:0/1376586490 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe7c0b7150 msgr2=0x7fbe7c0d0890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:04.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:04.789+0000 7fbe839da640 1 --2- 172.21.15.167:0/1376586490 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe7c0b7150 0x7fbe7c0d0890 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fbe7c0d2cd0 tx=0x7fbe7802a680 comp rx=0 tx=0).stop 2024-05-11T08:07:04.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:04.789+0000 7fbe839da640 1 -- 172.21.15.167:0/1376586490 shutdown_connections 2024-05-11T08:07:04.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:04.789+0000 7fbe839da640 1 --2- 172.21.15.167:0/1376586490 >> v2:172.21.15.204:3300/0 conn(0x7fbe7c0b7530 0x7fbe7c15d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:04.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:04.789+0000 7fbe839da640 1 --2- 172.21.15.167:0/1376586490 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe7c0b7150 0x7fbe7c0d0890 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:04.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:04.789+0000 7fbe839da640 1 -- 172.21.15.167:0/1376586490 >> 172.21.15.167:0/1376586490 conn(0x7fbe7c0d2550 msgr2=0x7fbe7c0ce890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:07:04.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:04.789+0000 7fbe839da640 1 -- 172.21.15.167:0/1376586490 shutdown_connections 2024-05-11T08:07:04.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:04.789+0000 7fbe839da640 1 -- 172.21.15.167:0/1376586490 wait complete. 2024-05-11T08:07:04.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:04.790+0000 7fbe839da640 1 Processor -- start 2024-05-11T08:07:04.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:04.790+0000 7fbe839da640 1 -- start start 2024-05-11T08:07:04.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:04.791+0000 7fbe839da640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe7c0b7150 0x7fbe7c150260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:04.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:04.791+0000 7fbe839da640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbe7c0b7530 0x7fbe7c06d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:04.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:04.791+0000 7fbe839da640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe7c15db90 0x7fbe7c06d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:04.794 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:04.791+0000 7fbe839da640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fbe7c0d0370 con 0x7fbe7c0b7150 2024-05-11T08:07:04.794 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:04.791+0000 7fbe839da640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fbe7c0d0090 con 0x7fbe7c15db90 2024-05-11T08:07:04.794 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:04.791+0000 7fbe839da640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fbe7c15e000 con 0x7fbe7c0b7530 2024-05-11T08:07:04.794 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:04.791+0000 7fbe81776640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe7c0b7150 0x7fbe7c150260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:04.794 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:04.791+0000 7fbe81776640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe7c0b7150 0x7fbe7c150260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:55600/0 (socket says 172.21.15.167:55600) 2024-05-11T08:07:04.794 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:04.791+0000 7fbe81776640 1 -- 172.21.15.167:0/1561876401 learned_addr learned my addr 172.21.15.167:0/1561876401 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:07:04.794 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:04.791+0000 7fbe81f77640 1 --2- 172.21.15.167:0/1561876401 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe7c15db90 0x7fbe7c06d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:04.794 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:04.791+0000 7fbe80f75640 1 --2- 172.21.15.167:0/1561876401 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbe7c0b7530 0x7fbe7c06d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:04.794 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:04.792+0000 7fbe81776640 1 -- 172.21.15.167:0/1561876401 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbe7c0b7530 msgr2=0x7fbe7c06d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:04.794 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:04.792+0000 7fbe81776640 1 --2- 172.21.15.167:0/1561876401 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbe7c0b7530 0x7fbe7c06d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:04.794 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:04.792+0000 7fbe81776640 1 -- 172.21.15.167:0/1561876401 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe7c15db90 msgr2=0x7fbe7c06d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:04.794 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:04.792+0000 7fbe81776640 1 --2- 172.21.15.167:0/1561876401 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe7c15db90 0x7fbe7c06d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:04.794 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:04.792+0000 7fbe81776640 1 -- 172.21.15.167:0/1561876401 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe7c157910 con 0x7fbe7c0b7150 2024-05-11T08:07:04.794 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:04.792+0000 7fbe81776640 1 --2- 172.21.15.167:0/1561876401 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe7c0b7150 0x7fbe7c150260 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fbe78009e80 tx=0x7fbe78004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:04.795 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:04.792+0000 7fbe71ffb640 1 -- 172.21.15.167:0/1561876401 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe7803f070 con 0x7fbe7c0b7150 2024-05-11T08:07:04.795 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:04.792+0000 7fbe71ffb640 1 -- 172.21.15.167:0/1561876401 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe78031c40 con 0x7fbe7c0b7150 2024-05-11T08:07:04.795 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:04.792+0000 7fbe71ffb640 1 -- 172.21.15.167:0/1561876401 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe78044460 con 0x7fbe7c0b7150 2024-05-11T08:07:04.795 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:04.792+0000 7fbe839da640 1 -- 172.21.15.167:0/1561876401 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe7c156870 con 0x7fbe7c0b7150 2024-05-11T08:07:04.795 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:04.792+0000 7fbe839da640 1 -- 172.21.15.167:0/1561876401 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe7c156e30 con 0x7fbe7c0b7150 2024-05-11T08:07:04.796 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:04.793+0000 7fbe839da640 1 -- 172.21.15.167:0/1561876401 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe7c069590 con 0x7fbe7c0b7150 2024-05-11T08:07:04.796 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:04.794+0000 7fbe71ffb640 1 -- 172.21.15.167:0/1561876401 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fbe7803b410 con 0x7fbe7c0b7150 2024-05-11T08:07:04.796 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:04.794+0000 7fbe71ffb640 1 --2- 172.21.15.167:0/1561876401 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fbe600762d0 0x7fbe60078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:04.796 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:04.794+0000 7fbe71ffb640 1 -- 172.21.15.167:0/1561876401 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6496+0+0 (secure 0 0 0) 0x7fbe780b8f10 con 0x7fbe7c0b7150 2024-05-11T08:07:04.796 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:04.794+0000 7fbe80f75640 1 --2- 172.21.15.167:0/1561876401 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fbe600762d0 0x7fbe60078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:04.796 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:04.794+0000 7fbe80f75640 1 --2- 172.21.15.167:0/1561876401 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fbe600762d0 0x7fbe60078790 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fbe7c158f40 tx=0x7fbe6c009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:04.798 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:04.798+0000 7fbe71ffb640 1 -- 172.21.15.167:0/1561876401 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fbe78081fa0 con 0x7fbe7c0b7150 2024-05-11T08:07:05.018 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.018+0000 7fbe839da640 1 -- 172.21.15.167:0/1561876401 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbe7c0d1430 con 0x7fbe7c0b7150 2024-05-11T08:07:05.018 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.018+0000 7fbe71ffb640 1 -- 172.21.15.167:0/1561876401 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5192 (secure 0 0 0) 0x7fbe780862a0 con 0x7fbe7c0b7150 2024-05-11T08:07:05.019 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T08:07:05.019 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4604,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2674983673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":2674983673},{"type":"v1","addr":"172.21.15.167:6837","nonce":2674983673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4607,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6834/1018548066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1018548066},{"type":"v1","addr":"172.21.15.204:6834","nonce":1018548066}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4729,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3369722249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3369722249},{"type":"v1","addr":"172.21.15.167:6836","nonce":3369722249}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-11T08:06:47.943939+0000","modified":"2024-05-11T08:07:04.006894+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4750},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4750":{"gid":4750,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.204:6835/469864006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6833","nonce":469864006},{"type":"v1","addr":"172.21.15.204:6835","nonce":469864006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6,7],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4750,"qdb_cluster":[4750]},"id":2}]} 2024-05-11T08:07:05.019 INFO:teuthology.orchestra.run.smithi167.stderr:dumped fsmap epoch 17 2024-05-11T08:07:05.021 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.021+0000 7fbe839da640 1 -- 172.21.15.167:0/1561876401 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fbe600762d0 msgr2=0x7fbe60078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:05.021 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.021+0000 7fbe839da640 1 --2- 172.21.15.167:0/1561876401 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fbe600762d0 0x7fbe60078790 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fbe7c158f40 tx=0x7fbe6c009190 comp rx=0 tx=0).stop 2024-05-11T08:07:05.021 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.021+0000 7fbe839da640 1 -- 172.21.15.167:0/1561876401 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe7c0b7150 msgr2=0x7fbe7c150260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:05.021 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.021+0000 7fbe839da640 1 --2- 172.21.15.167:0/1561876401 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe7c0b7150 0x7fbe7c150260 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fbe78009e80 tx=0x7fbe78004a40 comp rx=0 tx=0).stop 2024-05-11T08:07:05.021 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.021+0000 7fbe839da640 1 -- 172.21.15.167:0/1561876401 shutdown_connections 2024-05-11T08:07:05.021 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.021+0000 7fbe839da640 1 --2- 172.21.15.167:0/1561876401 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fbe600762d0 0x7fbe60078790 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:05.022 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.021+0000 7fbe839da640 1 --2- 172.21.15.167:0/1561876401 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe7c15db90 0x7fbe7c06d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:05.022 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.021+0000 7fbe839da640 1 --2- 172.21.15.167:0/1561876401 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbe7c0b7530 0x7fbe7c06d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:05.022 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.021+0000 7fbe839da640 1 --2- 172.21.15.167:0/1561876401 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe7c0b7150 0x7fbe7c150260 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:05.022 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.021+0000 7fbe839da640 1 -- 172.21.15.167:0/1561876401 >> 172.21.15.167:0/1561876401 conn(0x7fbe7c0d2550 msgr2=0x7fbe7c152850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:07:05.022 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.021+0000 7fbe839da640 1 -- 172.21.15.167:0/1561876401 shutdown_connections 2024-05-11T08:07:05.022 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.021+0000 7fbe839da640 1 -- 172.21.15.167:0/1561876401 wait complete. 2024-05-11T08:07:05.032 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-11T08:07:05.132 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.132+0000 7f53923da640 1 Processor -- start 2024-05-11T08:07:05.132 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.132+0000 7f53923da640 1 -- start start 2024-05-11T08:07:05.134 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.133+0000 7f53923da640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f538c1598d0 0x7f538c0d2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:05.134 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.133+0000 7f53923da640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f538c0d0640 0x7f538c0d0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:05.134 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.133+0000 7f53923da640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f538c0d1190 con 0x7f538c0cafb0 2024-05-11T08:07:05.134 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.133+0000 7f53923da640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f538c06d500 con 0x7f538c0d0640 2024-05-11T08:07:05.134 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.133+0000 7f53923da640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f538c06d7e0 con 0x7f538c1598d0 2024-05-11T08:07:05.134 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.133+0000 7f5390977640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f538c0cafb0 0x7f538c06dfe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:47368/0 (socket says 172.21.15.167:47368) 2024-05-11T08:07:05.134 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.133+0000 7f5390977640 1 -- 172.21.15.167:0/2024716343 learned_addr learned my addr 172.21.15.167:0/2024716343 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T08:07:05.134 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.133+0000 7f538b7fe640 1 --2- 172.21.15.167:0/2024716343 >> v2:172.21.15.204:3300/0 conn(0x7f538c0d0640 0x7f538c0d0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:05.134 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.133+0000 7f538bfff640 1 --2- 172.21.15.167:0/2024716343 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f538c1598d0 0x7f538c0d2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:05.134 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.133+0000 7f538affd640 1 -- 172.21.15.167:0/2024716343 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1918351396 0 0) 0x7f538c0d1190 con 0x7f538c0cafb0 2024-05-11T08:07:05.134 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.133+0000 7f538affd640 1 -- 172.21.15.167:0/2024716343 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5374003520 con 0x7f538c0cafb0 2024-05-11T08:07:05.135 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.133+0000 7f538affd640 1 -- 172.21.15.167:0/2024716343 <== mon.0 v1:172.21.15.167:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 184600685 0 0) 0x7f5374003520 con 0x7f538c0cafb0 2024-05-11T08:07:05.135 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.133+0000 7f538b7fe640 1 -- 172.21.15.167:0/2024716343 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f538c1598d0 msgr2=0x7f538c0d2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:07:05.135 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.133+0000 7f538b7fe640 1 --2- 172.21.15.167:0/2024716343 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f538c1598d0 0x7f538c0d2cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:05.135 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.133+0000 7f538b7fe640 1 -- 172.21.15.167:0/2024716343 >> v1:172.21.15.167:6789/0 conn(0x7f538c0cafb0 legacy=0x7f538c06dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:05.135 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.133+0000 7f538b7fe640 1 -- 172.21.15.167:0/2024716343 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f538c06dac0 con 0x7f538c0d0640 2024-05-11T08:07:05.135 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.133+0000 7f538bfff640 1 --2- 172.21.15.167:0/2024716343 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f538c1598d0 0x7f538c0d2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T08:07:05.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.134+0000 7f538b7fe640 1 --2- 172.21.15.167:0/2024716343 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f538c0d0640 0x7f538c0d0aa0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f53800097a0 tx=0x7f538002a110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=31c4513f5853b134 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:05.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.134+0000 7f538affd640 1 -- 172.21.15.167:0/2024716343 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5380037070 con 0x7f538c0d0640 2024-05-11T08:07:05.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.134+0000 7f538affd640 1 -- 172.21.15.167:0/2024716343 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f538002aba0 con 0x7f538c0d0640 2024-05-11T08:07:05.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.134+0000 7f538affd640 1 -- 172.21.15.167:0/2024716343 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5384001a70 con 0x7f538c0d0640 2024-05-11T08:07:05.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.135+0000 7f53923da640 1 -- 172.21.15.167:0/2024716343 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f538c0d0640 msgr2=0x7f538c0d0aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:05.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.135+0000 7f53923da640 1 --2- 172.21.15.167:0/2024716343 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f538c0d0640 0x7f538c0d0aa0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f53800097a0 tx=0x7f538002a110 comp rx=0 tx=0).stop 2024-05-11T08:07:05.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.135+0000 7f53923da640 1 -- 172.21.15.167:0/2024716343 shutdown_connections 2024-05-11T08:07:05.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.135+0000 7f53923da640 1 --2- 172.21.15.167:0/2024716343 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f538c0d0640 0x7f538c0d0aa0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:05.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.135+0000 7f53923da640 1 --2- 172.21.15.167:0/2024716343 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f538c1598d0 0x7f538c0d2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:05.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.135+0000 7f53923da640 1 -- 172.21.15.167:0/2024716343 >> 172.21.15.167:0/2024716343 conn(0x7f538c0d1ba0 msgr2=0x7f538c0d1fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:07:05.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.135+0000 7f53923da640 1 -- 172.21.15.167:0/2024716343 shutdown_connections 2024-05-11T08:07:05.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.135+0000 7f53923da640 1 -- 172.21.15.167:0/2024716343 wait complete. 2024-05-11T08:07:05.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.136+0000 7f53923da640 1 Processor -- start 2024-05-11T08:07:05.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.136+0000 7f53923da640 1 -- start start 2024-05-11T08:07:05.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.137+0000 7f53923da640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f538c0cafb0 0x7f538c14df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:05.140 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.137+0000 7f53923da640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f538c0d0640 0x7f538c14ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:05.140 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.137+0000 7f53923da640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f538c1598d0 0x7f538c1633e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:05.140 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.137+0000 7f53923da640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f538c06d1b0 con 0x7f538c1598d0 2024-05-11T08:07:05.140 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.137+0000 7f53923da640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f538c159cb0 con 0x7f538c0cafb0 2024-05-11T08:07:05.140 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.137+0000 7f53923da640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f538c0cb400 con 0x7f538c0d0640 2024-05-11T08:07:05.140 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.137+0000 7f5390977640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f538c1598d0 0x7f538c1633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:05.140 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.137+0000 7f5390977640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f538c1598d0 0x7f538c1633e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:55604/0 (socket says 172.21.15.167:55604) 2024-05-11T08:07:05.140 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.137+0000 7f5390977640 1 -- 172.21.15.167:0/2421480877 learned_addr learned my addr 172.21.15.167:0/2421480877 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:07:05.140 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.138+0000 7f538bfff640 1 --2- 172.21.15.167:0/2421480877 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f538c0cafb0 0x7f538c14df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:05.140 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.138+0000 7f538b7fe640 1 --2- 172.21.15.167:0/2421480877 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f538c0d0640 0x7f538c14ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:05.140 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.138+0000 7f5390977640 1 -- 172.21.15.167:0/2421480877 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f538c0d0640 msgr2=0x7f538c14ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:05.140 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.138+0000 7f5390977640 1 --2- 172.21.15.167:0/2421480877 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f538c0d0640 0x7f538c14ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:05.140 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.138+0000 7f5390977640 1 -- 172.21.15.167:0/2421480877 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f538c0cafb0 msgr2=0x7f538c14df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:05.141 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.138+0000 7f5390977640 1 --2- 172.21.15.167:0/2421480877 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f538c0cafb0 0x7f538c14df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:05.141 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.138+0000 7f5390977640 1 -- 172.21.15.167:0/2421480877 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f538c14e4c0 con 0x7f538c1598d0 2024-05-11T08:07:05.141 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.138+0000 7f538bfff640 1 --2- 172.21.15.167:0/2421480877 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f538c0cafb0 0x7f538c14df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T08:07:05.141 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.138+0000 7f538b7fe640 1 --2- 172.21.15.167:0/2421480877 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f538c0d0640 0x7f538c14ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T08:07:05.141 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.138+0000 7f5390977640 1 --2- 172.21.15.167:0/2421480877 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f538c1598d0 0x7f538c1633e0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f538400cbf0 tx=0x7f538400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:05.141 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.138+0000 7f5388ff9640 1 -- 172.21.15.167:0/2421480877 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5384017070 con 0x7f538c1598d0 2024-05-11T08:07:05.141 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.138+0000 7f5388ff9640 1 -- 172.21.15.167:0/2421480877 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f538400ad50 con 0x7f538c1598d0 2024-05-11T08:07:05.141 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.138+0000 7f53923da640 1 -- 172.21.15.167:0/2421480877 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f538c150be0 con 0x7f538c1598d0 2024-05-11T08:07:05.141 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.138+0000 7f5388ff9640 1 -- 172.21.15.167:0/2421480877 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5384012620 con 0x7f538c1598d0 2024-05-11T08:07:05.141 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.138+0000 7f53923da640 1 -- 172.21.15.167:0/2421480877 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f538c1511a0 con 0x7f538c1598d0 2024-05-11T08:07:05.141 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.139+0000 7f53923da640 1 -- 172.21.15.167:0/2421480877 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f538c069590 con 0x7f538c1598d0 2024-05-11T08:07:05.141 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.140+0000 7f5388ff9640 1 -- 172.21.15.167:0/2421480877 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f538401a070 con 0x7f538c1598d0 2024-05-11T08:07:05.142 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.140+0000 7f5388ff9640 1 --2- 172.21.15.167:0/2421480877 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f53780762d0 0x7f5378078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:05.142 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.140+0000 7f5388ff9640 1 -- 172.21.15.167:0/2421480877 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6496+0+0 (secure 0 0 0) 0x7f5384097780 con 0x7f538c1598d0 2024-05-11T08:07:05.142 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.140+0000 7f538bfff640 1 --2- 172.21.15.167:0/2421480877 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f53780762d0 0x7f5378078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:05.142 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.141+0000 7f538bfff640 1 --2- 172.21.15.167:0/2421480877 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f53780762d0 0x7f5378078790 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f537c009a20 tx=0x7f537c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:05.144 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.145+0000 7f5388ff9640 1 -- 172.21.15.167:0/2421480877 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f5384060e60 con 0x7f538c1598d0 2024-05-11T08:07:05.325 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.325+0000 7f53923da640 1 -- 172.21.15.167:0/2421480877 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f538c1623e0 con 0x7f538c1598d0 2024-05-11T08:07:05.326 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.326+0000 7f5388ff9640 1 -- 172.21.15.167:0/2421480877 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) ==== 74+0+18815 (secure 0 0 0) 0x7f5384060760 con 0x7f538c1598d0 2024-05-11T08:07:05.327 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T08:07:05.327 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":43,"fsid":"2034778d-8ff1-4823-aa44-ded32b518241","created":"2024-05-11T08:05:31.612421+0000","modified":"2024-05-11T08:07:04.006691+0000","last_up_change":"2024-05-11T08:05:42.557382+0000","last_in_change":"2024-05-11T08:05:33.757311+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-11T08:05:43.931263+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-05-11T08:06:45.857156+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-05-11T08:06:46.898591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"data_new","create_time":"2024-05-11T08:07:03.541639+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"587eaced-5ded-406d-884b-30c1caa8686b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":3863307356},{"type":"v1","addr":"172.21.15.167:6806","nonce":3863307356}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":3863307356},{"type":"v1","addr":"172.21.15.167:6810","nonce":3863307356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":3863307356},{"type":"v1","addr":"172.21.15.167:6815","nonce":3863307356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":3863307356},{"type":"v1","addr":"172.21.15.167:6813","nonce":3863307356}]},"public_addr":"172.21.15.167:6806/3863307356","cluster_addr":"172.21.15.167:6810/3863307356","heartbeat_back_addr":"172.21.15.167:6815/3863307356","heartbeat_front_addr":"172.21.15.167:6813/3863307356","state":["exists","up"]},{"osd":1,"uuid":"37c59104-2dd0-4cf9-85ea-56636f6b02a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":1585024048},{"type":"v1","addr":"172.21.15.167:6817","nonce":1585024048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1585024048},{"type":"v1","addr":"172.21.15.167:6819","nonce":1585024048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1585024048},{"type":"v1","addr":"172.21.15.167:6823","nonce":1585024048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":1585024048},{"type":"v1","addr":"172.21.15.167:6821","nonce":1585024048}]},"public_addr":"172.21.15.167:6817/1585024048","cluster_addr":"172.21.15.167:6819/1585024048","heartbeat_back_addr":"172.21.15.167:6823/1585024048","heartbeat_front_addr":"172.21.15.167:6821/1585024048","state":["exists","up"]},{"osd":2,"uuid":"2bb4f7f5-c698-451b-b4bb-78e716921291","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":3242166900},{"type":"v1","addr":"172.21.15.167:6801","nonce":3242166900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":3242166900},{"type":"v1","addr":"172.21.15.167:6803","nonce":3242166900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6809","nonce":3242166900},{"type":"v1","addr":"172.21.15.167:6811","nonce":3242166900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6805","nonce":3242166900},{"type":"v1","addr":"172.21.15.167:6807","nonce":3242166900}]},"public_addr":"172.21.15.167:6801/3242166900","cluster_addr":"172.21.15.167:6803/3242166900","heartbeat_back_addr":"172.21.15.167:6811/3242166900","heartbeat_front_addr":"172.21.15.167:6807/3242166900","state":["exists","up"]},{"osd":3,"uuid":"9dd26759-a338-49d2-8103-45d068b9c000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":453689293},{"type":"v1","addr":"172.21.15.167:6825","nonce":453689293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":453689293},{"type":"v1","addr":"172.21.15.167:6827","nonce":453689293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":453689293},{"type":"v1","addr":"172.21.15.167:6831","nonce":453689293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":453689293},{"type":"v1","addr":"172.21.15.167:6829","nonce":453689293}]},"public_addr":"172.21.15.167:6825/453689293","cluster_addr":"172.21.15.167:6827/453689293","heartbeat_back_addr":"172.21.15.167:6831/453689293","heartbeat_front_addr":"172.21.15.167:6829/453689293","state":["exists","up"]},{"osd":4,"uuid":"cb27250d-2a80-4c7b-8cef-3a7af05c8a16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1311831500},{"type":"v1","addr":"172.21.15.204:6801","nonce":1311831500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1311831500},{"type":"v1","addr":"172.21.15.204:6803","nonce":1311831500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1311831500},{"type":"v1","addr":"172.21.15.204:6807","nonce":1311831500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1311831500},{"type":"v1","addr":"172.21.15.204:6805","nonce":1311831500}]},"public_addr":"172.21.15.204:6801/1311831500","cluster_addr":"172.21.15.204:6803/1311831500","heartbeat_back_addr":"172.21.15.204:6807/1311831500","heartbeat_front_addr":"172.21.15.204:6805/1311831500","state":["exists","up"]},{"osd":5,"uuid":"92998541-ffec-430b-ab76-8c3cb7c2752d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2921847521},{"type":"v1","addr":"172.21.15.204:6825","nonce":2921847521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2921847521},{"type":"v1","addr":"172.21.15.204:6827","nonce":2921847521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2921847521},{"type":"v1","addr":"172.21.15.204:6831","nonce":2921847521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2921847521},{"type":"v1","addr":"172.21.15.204:6829","nonce":2921847521}]},"public_addr":"172.21.15.204:6825/2921847521","cluster_addr":"172.21.15.204:6827/2921847521","heartbeat_back_addr":"172.21.15.204:6831/2921847521","heartbeat_front_addr":"172.21.15.204:6829/2921847521","state":["exists","up"]},{"osd":6,"uuid":"9543479c-2ec5-42a8-8933-707bfed2e048","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":2889851077},{"type":"v1","addr":"172.21.15.204:6817","nonce":2889851077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":2889851077},{"type":"v1","addr":"172.21.15.204:6819","nonce":2889851077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":2889851077},{"type":"v1","addr":"172.21.15.204:6823","nonce":2889851077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":2889851077},{"type":"v1","addr":"172.21.15.204:6821","nonce":2889851077}]},"public_addr":"172.21.15.204:6817/2889851077","cluster_addr":"172.21.15.204:6819/2889851077","heartbeat_back_addr":"172.21.15.204:6823/2889851077","heartbeat_front_addr":"172.21.15.204:6821/2889851077","state":["exists","up"]},{"osd":7,"uuid":"c45a5647-8358-409e-af91-c05dda65f0b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":1949534184},{"type":"v1","addr":"172.21.15.204:6809","nonce":1949534184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":1949534184},{"type":"v1","addr":"172.21.15.204:6811","nonce":1949534184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1949534184},{"type":"v1","addr":"172.21.15.204:6815","nonce":1949534184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1949534184},{"type":"v1","addr":"172.21.15.204:6813","nonce":1949534184}]},"public_addr":"172.21.15.204:6809/1949534184","cluster_addr":"172.21.15.204:6811/1949534184","heartbeat_back_addr":"172.21.15.204:6815/1949534184","heartbeat_front_addr":"172.21.15.204:6813/1949534184","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.952253+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.940689+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.860682+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.929350+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.901627+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.957451+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.987249+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.951197+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-11T08:07:05.329 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.328+0000 7f53923da640 1 -- 172.21.15.167:0/2421480877 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f53780762d0 msgr2=0x7f5378078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:05.329 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.328+0000 7f53923da640 1 --2- 172.21.15.167:0/2421480877 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f53780762d0 0x7f5378078790 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f537c009a20 tx=0x7f537c008040 comp rx=0 tx=0).stop 2024-05-11T08:07:05.329 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.328+0000 7f53923da640 1 -- 172.21.15.167:0/2421480877 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f538c1598d0 msgr2=0x7f538c1633e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:05.329 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.328+0000 7f53923da640 1 --2- 172.21.15.167:0/2421480877 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f538c1598d0 0x7f538c1633e0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f538400cbf0 tx=0x7f538400a430 comp rx=0 tx=0).stop 2024-05-11T08:07:05.329 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.328+0000 7f53923da640 1 -- 172.21.15.167:0/2421480877 shutdown_connections 2024-05-11T08:07:05.329 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.328+0000 7f53923da640 1 --2- 172.21.15.167:0/2421480877 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f53780762d0 0x7f5378078790 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:05.329 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.328+0000 7f53923da640 1 --2- 172.21.15.167:0/2421480877 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f538c1598d0 0x7f538c1633e0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:05.329 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.328+0000 7f53923da640 1 --2- 172.21.15.167:0/2421480877 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f538c0d0640 0x7f538c14ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:05.329 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.328+0000 7f53923da640 1 --2- 172.21.15.167:0/2421480877 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f538c0cafb0 0x7f538c14df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:05.329 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.328+0000 7f53923da640 1 -- 172.21.15.167:0/2421480877 >> 172.21.15.167:0/2421480877 conn(0x7f538c0d1ba0 msgr2=0x7f538c0c9fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:07:05.330 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.328+0000 7f53923da640 1 -- 172.21.15.167:0/2421480877 shutdown_connections 2024-05-11T08:07:05.330 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.328+0000 7f53923da640 1 -- 172.21.15.167:0/2421480877 wait complete. 2024-05-11T08:07:05.340 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T08:07:05.340 DEBUG:teuthology.orchestra.run.smithi167:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.file.layout.pool -v data_new ./parent_b/alpha) 2024-05-11T08:07:05.360 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-11T08:07:05.501 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.500+0000 7f6067719640 1 Processor -- start 2024-05-11T08:07:05.501 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.501+0000 7f6067719640 1 -- start start 2024-05-11T08:07:05.503 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.501+0000 7f6067719640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f60600d1cc0 0x7f60600cd580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:05.503 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.501+0000 7f6067719640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f60600d25f0 0x7f6060166b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:05.503 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.501+0000 7f6067719640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60600cf950 con 0x7f6060070000 2024-05-11T08:07:05.503 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.501+0000 7f6067719640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f6060071ea0 con 0x7f60600d1cc0 2024-05-11T08:07:05.503 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.501+0000 7f6067719640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f60600d0220 con 0x7f60600d25f0 2024-05-11T08:07:05.503 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.502+0000 7f6064cb4640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f60600d1cc0 0x7f60600cd580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:05.503 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.502+0000 7f60654b5640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f6060070000 0x7f60600703e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:47378/0 (socket says 172.21.15.167:47378) 2024-05-11T08:07:05.503 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.502+0000 7f60654b5640 1 -- 172.21.15.167:0/4246952433 learned_addr learned my addr 172.21.15.167:0/4246952433 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T08:07:05.503 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.502+0000 7f6065cb6640 1 --2- 172.21.15.167:0/4246952433 >> v2:172.21.15.204:3300/0 conn(0x7f60600d25f0 0x7f6060166b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:05.503 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.502+0000 7f6064cb4640 1 -- 172.21.15.167:0/4246952433 >> v2:172.21.15.204:3300/0 conn(0x7f60600d25f0 msgr2=0x7f6060166b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:07:05.503 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.502+0000 7f6064cb4640 1 --2- 172.21.15.167:0/4246952433 >> v2:172.21.15.204:3300/0 conn(0x7f60600d25f0 0x7f6060166b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:05.503 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.502+0000 7f6064cb4640 1 -- 172.21.15.167:0/4246952433 >> v1:172.21.15.167:6789/0 conn(0x7f6060070000 legacy=0x7f60600703e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:05.503 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.502+0000 7f6064cb4640 1 -- 172.21.15.167:0/4246952433 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60600d2b70 con 0x7f60600d1cc0 2024-05-11T08:07:05.504 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.502+0000 7f605ffff640 1 -- 172.21.15.167:0/4246952433 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2293037709 0 0) 0x7f6058001110 con 0x7f6060070000 2024-05-11T08:07:05.504 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.502+0000 7f6065cb6640 1 --2- 172.21.15.167:0/4246952433 >> v2:172.21.15.204:3300/0 conn(0x7f60600d25f0 0x7f6060166b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T08:07:05.504 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.503+0000 7f6064cb4640 1 --2- 172.21.15.167:0/4246952433 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f60600d1cc0 0x7f60600cd580 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f606015d550 tx=0x7f605402a080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40453bcf019f7f5c server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:05.504 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.503+0000 7f605ffff640 1 -- 172.21.15.167:0/4246952433 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6054037070 con 0x7f60600d1cc0 2024-05-11T08:07:05.504 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.503+0000 7f605ffff640 1 -- 172.21.15.167:0/4246952433 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f605402add0 con 0x7f60600d1cc0 2024-05-11T08:07:05.504 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.503+0000 7f605ffff640 1 -- 172.21.15.167:0/4246952433 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6054032660 con 0x7f60600d1cc0 2024-05-11T08:07:05.507 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.503+0000 7f6067719640 1 -- 172.21.15.167:0/4246952433 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f60600d1cc0 msgr2=0x7f60600cd580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:05.507 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.503+0000 7f6067719640 1 --2- 172.21.15.167:0/4246952433 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f60600d1cc0 0x7f60600cd580 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f606015d550 tx=0x7f605402a080 comp rx=0 tx=0).stop 2024-05-11T08:07:05.507 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.503+0000 7f6067719640 1 -- 172.21.15.167:0/4246952433 shutdown_connections 2024-05-11T08:07:05.507 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.503+0000 7f6067719640 1 --2- 172.21.15.167:0/4246952433 >> v2:172.21.15.204:3300/0 conn(0x7f60600d25f0 0x7f6060166b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:05.507 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.503+0000 7f6067719640 1 --2- 172.21.15.167:0/4246952433 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f60600d1cc0 0x7f60600cd580 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:05.507 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.503+0000 7f6067719640 1 -- 172.21.15.167:0/4246952433 >> 172.21.15.167:0/4246952433 conn(0x7f60600b4ae0 msgr2=0x7f60600b4ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:07:05.507 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.503+0000 7f6067719640 1 -- 172.21.15.167:0/4246952433 shutdown_connections 2024-05-11T08:07:05.507 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.503+0000 7f6067719640 1 -- 172.21.15.167:0/4246952433 wait complete. 2024-05-11T08:07:05.507 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.505+0000 7f6067719640 1 Processor -- start 2024-05-11T08:07:05.508 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.505+0000 7f6067719640 1 -- start start 2024-05-11T08:07:05.508 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.505+0000 7f6067719640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6060070000 0x7f60600caa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:05.508 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.505+0000 7f6067719640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f60600d1cc0 0x7f606014e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:05.508 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.505+0000 7f6067719640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f60600d25f0 0x7f606015b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:05.508 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.505+0000 7f6067719640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6060070af0 con 0x7f60600d1cc0 2024-05-11T08:07:05.508 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.505+0000 7f6067719640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f60600cfeb0 con 0x7f60600d25f0 2024-05-11T08:07:05.508 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.505+0000 7f6067719640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f6060070c70 con 0x7f6060070000 2024-05-11T08:07:05.508 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.505+0000 7f6064cb4640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f60600d1cc0 0x7f606014e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:05.508 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.505+0000 7f6064cb4640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f60600d1cc0 0x7f606014e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:55612/0 (socket says 172.21.15.167:55612) 2024-05-11T08:07:05.508 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.505+0000 7f6064cb4640 1 -- 172.21.15.167:0/894816500 learned_addr learned my addr 172.21.15.167:0/894816500 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:07:05.508 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.506+0000 7f6064cb4640 1 -- 172.21.15.167:0/894816500 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6060070000 msgr2=0x7f60600caa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:05.508 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.506+0000 7f6064cb4640 1 --2- 172.21.15.167:0/894816500 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6060070000 0x7f60600caa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:05.508 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.506+0000 7f6064cb4640 1 -- 172.21.15.167:0/894816500 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f60600d25f0 msgr2=0x7f606015b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:05.509 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.506+0000 7f6065cb6640 1 --2- 172.21.15.167:0/894816500 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f60600d25f0 0x7f606015b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:05.509 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.506+0000 7f6064cb4640 1 --2- 172.21.15.167:0/894816500 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f60600d25f0 0x7f606015b190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:05.509 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.506+0000 7f6064cb4640 1 -- 172.21.15.167:0/894816500 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60600c6d30 con 0x7f60600d1cc0 2024-05-11T08:07:05.509 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.506+0000 7f6064cb4640 1 --2- 172.21.15.167:0/894816500 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f60600d1cc0 0x7f606014e720 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f6054009890 tx=0x7f60540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:05.509 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.506+0000 7f605dffb640 1 -- 172.21.15.167:0/894816500 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6054041020 con 0x7f60600d1cc0 2024-05-11T08:07:05.509 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.506+0000 7f605dffb640 1 -- 172.21.15.167:0/894816500 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f605403d640 con 0x7f60600d1cc0 2024-05-11T08:07:05.509 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.506+0000 7f6067719640 1 -- 172.21.15.167:0/894816500 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60600c7010 con 0x7f60600d1cc0 2024-05-11T08:07:05.509 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.506+0000 7f605dffb640 1 -- 172.21.15.167:0/894816500 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6054037040 con 0x7f60600d1cc0 2024-05-11T08:07:05.509 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.506+0000 7f6067719640 1 -- 172.21.15.167:0/894816500 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60600c75d0 con 0x7f60600d1cc0 2024-05-11T08:07:05.510 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.507+0000 7f6067719640 1 -- 172.21.15.167:0/894816500 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6060069bc0 con 0x7f60600d1cc0 2024-05-11T08:07:05.510 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.508+0000 7f605dffb640 1 -- 172.21.15.167:0/894816500 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f6054044020 con 0x7f60600d1cc0 2024-05-11T08:07:05.510 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.508+0000 7f605dffb640 1 --2- 172.21.15.167:0/894816500 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f604c0762d0 0x7f604c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:05.510 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.509+0000 7f605dffb640 1 -- 172.21.15.167:0/894816500 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6496+0+0 (secure 0 0 0) 0x7f60540b7dd0 con 0x7f60600d1cc0 2024-05-11T08:07:05.510 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.509+0000 7f60654b5640 1 --2- 172.21.15.167:0/894816500 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f604c0762d0 0x7f604c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:05.510 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.509+0000 7f60654b5640 1 --2- 172.21.15.167:0/894816500 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f604c0762d0 0x7f604c078790 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f6058007fd0 tx=0x7f6058007960 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:05.513 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.513+0000 7f605dffb640 1 -- 172.21.15.167:0/894816500 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f605404b030 con 0x7f60600d1cc0 2024-05-11T08:07:05.731 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.731+0000 7f6067719640 1 -- 172.21.15.167:0/894816500 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f60600b4fd0 con 0x7f60600d1cc0 2024-05-11T08:07:05.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.732+0000 7f605dffb640 1 -- 172.21.15.167:0/894816500 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5192 (secure 0 0 0) 0x7f60540814b0 con 0x7f60600d1cc0 2024-05-11T08:07:05.732 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T08:07:05.732 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4604,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2674983673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":2674983673},{"type":"v1","addr":"172.21.15.167:6837","nonce":2674983673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4607,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6834/1018548066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1018548066},{"type":"v1","addr":"172.21.15.204:6834","nonce":1018548066}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4729,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3369722249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3369722249},{"type":"v1","addr":"172.21.15.167:6836","nonce":3369722249}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-11T08:06:47.943939+0000","modified":"2024-05-11T08:07:04.006894+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4750},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4750":{"gid":4750,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.204:6835/469864006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6833","nonce":469864006},{"type":"v1","addr":"172.21.15.204:6835","nonce":469864006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6,7],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4750,"qdb_cluster":[4750]},"id":2}]} 2024-05-11T08:07:05.732 INFO:teuthology.orchestra.run.smithi167.stderr:dumped fsmap epoch 17 2024-05-11T08:07:05.734 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.734+0000 7f6067719640 1 -- 172.21.15.167:0/894816500 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f604c0762d0 msgr2=0x7f604c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:05.734 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.734+0000 7f6067719640 1 --2- 172.21.15.167:0/894816500 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f604c0762d0 0x7f604c078790 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f6058007fd0 tx=0x7f6058007960 comp rx=0 tx=0).stop 2024-05-11T08:07:05.734 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.734+0000 7f6067719640 1 -- 172.21.15.167:0/894816500 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f60600d1cc0 msgr2=0x7f606014e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:05.734 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.734+0000 7f6067719640 1 --2- 172.21.15.167:0/894816500 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f60600d1cc0 0x7f606014e720 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f6054009890 tx=0x7f60540046c0 comp rx=0 tx=0).stop 2024-05-11T08:07:05.734 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.734+0000 7f6067719640 1 -- 172.21.15.167:0/894816500 shutdown_connections 2024-05-11T08:07:05.735 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.734+0000 7f6067719640 1 --2- 172.21.15.167:0/894816500 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f604c0762d0 0x7f604c078790 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:05.735 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.734+0000 7f6067719640 1 --2- 172.21.15.167:0/894816500 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f60600d25f0 0x7f606015b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:05.735 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.734+0000 7f6067719640 1 --2- 172.21.15.167:0/894816500 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f60600d1cc0 0x7f606014e720 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:05.735 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.734+0000 7f6067719640 1 --2- 172.21.15.167:0/894816500 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6060070000 0x7f60600caa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:05.735 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.734+0000 7f6067719640 1 -- 172.21.15.167:0/894816500 >> 172.21.15.167:0/894816500 conn(0x7f60600b4ae0 msgr2=0x7f60600b5ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:07:05.735 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.734+0000 7f6067719640 1 -- 172.21.15.167:0/894816500 shutdown_connections 2024-05-11T08:07:05.735 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.734+0000 7f6067719640 1 -- 172.21.15.167:0/894816500 wait complete. 2024-05-11T08:07:05.745 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json flush journal 2024-05-11T08:07:05.886 INFO:teuthology.orchestra.run.smithi204.stdout:{"message":"","return_code":0} 2024-05-11T08:07:05.886 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-05-11T08:07:05.886 DEBUG:teuthology.orchestra.run.smithi167:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001fb.00000000 parent 2024-05-11T08:07:05.930 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.930+0000 7fda6011ff40 1 Processor -- start 2024-05-11T08:07:05.931 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.931+0000 7fda6011ff40 1 -- start start 2024-05-11T08:07:05.933 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.931+0000 7fda6011ff40 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x557c801d3090 0x557c802a3860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:05.933 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.931+0000 7fda6011ff40 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c801ffd60 0x557c802a76e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:05.933 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.931+0000 7fda6011ff40 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557c802a6100 0x557c8029b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:05.933 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.931+0000 7fda6011ff40 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x557c802006f0 con 0x557c801ffd60 2024-05-11T08:07:05.933 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.931+0000 7fda6011ff40 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x557c80178f50 con 0x557c802a6100 2024-05-11T08:07:05.933 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.931+0000 7fda6011ff40 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x557c802a0130 con 0x557c801d3090 2024-05-11T08:07:05.933 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.931+0000 7fda57fff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c801ffd60 0x557c802a76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:05.933 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.931+0000 7fda57fff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c801ffd60 0x557c802a76e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:55642/0 (socket says 172.21.15.167:55642) 2024-05-11T08:07:05.934 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.931+0000 7fda57fff640 1 -- 172.21.15.167:0/3003175610 learned_addr learned my addr 172.21.15.167:0/3003175610 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:07:05.934 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.931+0000 7fda5d994640 1 --2- 172.21.15.167:0/3003175610 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557c802a6100 0x557c8029b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:05.934 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.931+0000 7fda5d193640 1 --2- 172.21.15.167:0/3003175610 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x557c801d3090 0x557c802a3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:05.934 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.932+0000 7fda57fff640 1 -- 172.21.15.167:0/3003175610 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x557c801d3090 msgr2=0x557c802a3860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:07:05.934 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.932+0000 7fda57fff640 1 --2- 172.21.15.167:0/3003175610 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x557c801d3090 0x557c802a3860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:05.934 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.932+0000 7fda57fff640 1 -- 172.21.15.167:0/3003175610 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557c802a6100 msgr2=0x557c8029b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:07:05.934 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.932+0000 7fda57fff640 1 --2- 172.21.15.167:0/3003175610 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557c802a6100 0x557c8029b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:05.934 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.932+0000 7fda57fff640 1 -- 172.21.15.167:0/3003175610 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c801fd330 con 0x557c801ffd60 2024-05-11T08:07:05.934 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.932+0000 7fda57fff640 1 --2- 172.21.15.167:0/3003175610 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c801ffd60 0x557c802a76e0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fda48001d50 tx=0x7fda4800b5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2ef8b086d66a0c1 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:05.934 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.932+0000 7fda577fe640 1 -- 172.21.15.167:0/3003175610 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda4800bd80 con 0x557c801ffd60 2024-05-11T08:07:05.934 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.932+0000 7fda577fe640 1 -- 172.21.15.167:0/3003175610 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda4800f400 con 0x557c801ffd60 2024-05-11T08:07:05.935 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.932+0000 7fda577fe640 1 -- 172.21.15.167:0/3003175610 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda4800f7f0 con 0x557c801ffd60 2024-05-11T08:07:05.935 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.932+0000 7fda6011ff40 1 -- 172.21.15.167:0/3003175610 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c801ffd60 msgr2=0x557c802a76e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:05.935 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.932+0000 7fda6011ff40 1 --2- 172.21.15.167:0/3003175610 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c801ffd60 0x557c802a76e0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fda48001d50 tx=0x7fda4800b5b0 comp rx=0 tx=0).stop 2024-05-11T08:07:05.935 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.932+0000 7fda6011ff40 1 -- 172.21.15.167:0/3003175610 shutdown_connections 2024-05-11T08:07:05.935 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.932+0000 7fda6011ff40 1 --2- 172.21.15.167:0/3003175610 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557c802a6100 0x557c8029b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:05.935 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.932+0000 7fda6011ff40 1 --2- 172.21.15.167:0/3003175610 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c801ffd60 0x557c802a76e0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:05.935 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.932+0000 7fda6011ff40 1 --2- 172.21.15.167:0/3003175610 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x557c801d3090 0x557c802a3860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:05.935 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.932+0000 7fda6011ff40 1 -- 172.21.15.167:0/3003175610 >> 172.21.15.167:0/3003175610 conn(0x557c801fd950 msgr2=0x557c801d25e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:07:05.935 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.932+0000 7fda6011ff40 1 -- 172.21.15.167:0/3003175610 shutdown_connections 2024-05-11T08:07:05.935 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.932+0000 7fda6011ff40 1 -- 172.21.15.167:0/3003175610 wait complete. 2024-05-11T08:07:05.935 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.933+0000 7fda6011ff40 1 Processor -- start 2024-05-11T08:07:05.935 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.933+0000 7fda6011ff40 1 -- start start 2024-05-11T08:07:05.941 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.934+0000 7fda6011ff40 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557c801d3090 0x557c80299000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:05.941 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.934+0000 7fda6011ff40 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c801ffd60 0x557c8029d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:05.941 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.934+0000 7fda6011ff40 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x557c802a6100 0x557c8029e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:05.941 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.934+0000 7fda6011ff40 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x557c801fb6b0 con 0x557c801ffd60 2024-05-11T08:07:05.942 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.934+0000 7fda6011ff40 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x557c801fb460 con 0x557c801d3090 2024-05-11T08:07:05.942 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.934+0000 7fda6011ff40 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x557c801fddd0 con 0x557c802a6100 2024-05-11T08:07:05.942 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.934+0000 7fda57fff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c801ffd60 0x557c8029d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:05.942 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.934+0000 7fda57fff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c801ffd60 0x557c8029d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:55648/0 (socket says 172.21.15.167:55648) 2024-05-11T08:07:05.942 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.934+0000 7fda57fff640 1 -- 172.21.15.167:0/1913633011 learned_addr learned my addr 172.21.15.167:0/1913633011 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:07:05.942 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.934+0000 7fda5d193640 1 --2- 172.21.15.167:0/1913633011 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557c801d3090 0x557c80299000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:05.942 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.934+0000 7fda5d994640 1 --2- 172.21.15.167:0/1913633011 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x557c802a6100 0x557c8029e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:05.942 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.934+0000 7fda57fff640 1 -- 172.21.15.167:0/1913633011 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x557c802a6100 msgr2=0x557c8029e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:05.942 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.934+0000 7fda57fff640 1 --2- 172.21.15.167:0/1913633011 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x557c802a6100 0x557c8029e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:05.942 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.934+0000 7fda57fff640 1 -- 172.21.15.167:0/1913633011 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557c801d3090 msgr2=0x557c80299000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:05.942 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.934+0000 7fda57fff640 1 --2- 172.21.15.167:0/1913633011 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557c801d3090 0x557c80299000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:05.942 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.934+0000 7fda57fff640 1 -- 172.21.15.167:0/1913633011 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c802a5aa0 con 0x557c801ffd60 2024-05-11T08:07:05.942 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.934+0000 7fda5d193640 1 --2- 172.21.15.167:0/1913633011 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557c801d3090 0x557c80299000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T08:07:05.943 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.934+0000 7fda57fff640 1 --2- 172.21.15.167:0/1913633011 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c801ffd60 0x557c8029d940 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fda48001d20 tx=0x7fda48000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:05.943 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.934+0000 7fda567fc640 1 -- 172.21.15.167:0/1913633011 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda4801c440 con 0x557c801ffd60 2024-05-11T08:07:05.943 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.935+0000 7fda567fc640 1 -- 172.21.15.167:0/1913633011 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda4801c5e0 con 0x557c801ffd60 2024-05-11T08:07:05.943 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.935+0000 7fda6011ff40 1 -- 172.21.15.167:0/1913633011 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x557c8029b000 con 0x557c801ffd60 2024-05-11T08:07:05.943 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.935+0000 7fda567fc640 1 -- 172.21.15.167:0/1913633011 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda4801c9f0 con 0x557c801ffd60 2024-05-11T08:07:05.943 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.935+0000 7fda6011ff40 1 -- 172.21.15.167:0/1913633011 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c8029b5c0 con 0x557c801ffd60 2024-05-11T08:07:05.943 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.936+0000 7fda567fc640 1 -- 172.21.15.167:0/1913633011 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fda48017030 con 0x557c801ffd60 2024-05-11T08:07:05.943 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.936+0000 7fda567fc640 1 --2- 172.21.15.167:0/1913633011 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fda40074cb0 0x7fda40077170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:05.943 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.937+0000 7fda5d193640 1 --2- 172.21.15.167:0/1913633011 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fda40074cb0 0x7fda40077170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:05.943 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.937+0000 7fda567fc640 1 -- 172.21.15.167:0/1913633011 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 6496+0+0 (secure 0 0 0) 0x7fda48097b90 con 0x557c801ffd60 2024-05-11T08:07:05.943 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.937+0000 7fda6011ff40 1 --2- 172.21.15.167:0/1913633011 >> [v2:172.21.15.167:6824/453689293,v1:172.21.15.167:6825/453689293] conn(0x557c801d0100 0x557c80214fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:05.943 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.937+0000 7fda6011ff40 1 -- 172.21.15.167:0/1913633011 --> [v2:172.21.15.167:6824/453689293,v1:172.21.15.167:6825/453689293] -- osd_op(unknown.0.0:1 6.13 6:ca8fdbba:::100000001fb.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44) -- 0x557c8029de80 con 0x557c801d0100 2024-05-11T08:07:05.943 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.937+0000 7fda5d994640 1 --2- 172.21.15.167:0/1913633011 >> [v2:172.21.15.167:6824/453689293,v1:172.21.15.167:6825/453689293] conn(0x557c801d0100 0x557c80214fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:05.943 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.937+0000 7fda5d193640 1 --2- 172.21.15.167:0/1913633011 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fda40074cb0 0x7fda40077170 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fda50005400 tx=0x7fda5000c040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:05.943 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.937+0000 7fda5d994640 1 --2- 172.21.15.167:0/1913633011 >> [v2:172.21.15.167:6824/453689293,v1:172.21.15.167:6825/453689293] conn(0x557c801d0100 0x557c80214fd0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:05.944 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.937+0000 7fda5d994640 1 -- 172.21.15.167:0/1913633011 <== osd.3 v2:172.21.15.167:6824/453689293 1 ==== osd_op_reply(1 100000001fb.00000000 [getxattr (103) out=103b] v0'0 uv3 ondisk = 0) ==== 164+0+103 (crc 0 0 0) 0x7fda58056030 con 0x557c801d0100 2024-05-11T08:07:05.944 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.937+0000 7fda6011ff40 1 -- 172.21.15.167:0/1913633011 >> [v2:172.21.15.167:6824/453689293,v1:172.21.15.167:6825/453689293] conn(0x557c801d0100 msgr2=0x557c80214fd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:05.944 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.938+0000 7fda6011ff40 1 --2- 172.21.15.167:0/1913633011 >> [v2:172.21.15.167:6824/453689293,v1:172.21.15.167:6825/453689293] conn(0x557c801d0100 0x557c80214fd0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:05.944 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.938+0000 7fda6011ff40 1 -- 172.21.15.167:0/1913633011 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fda40074cb0 msgr2=0x7fda40077170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:05.944 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.938+0000 7fda6011ff40 1 --2- 172.21.15.167:0/1913633011 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fda40074cb0 0x7fda40077170 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fda50005400 tx=0x7fda5000c040 comp rx=0 tx=0).stop 2024-05-11T08:07:05.944 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.938+0000 7fda6011ff40 1 -- 172.21.15.167:0/1913633011 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c801ffd60 msgr2=0x557c8029d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:05.944 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.938+0000 7fda6011ff40 1 --2- 172.21.15.167:0/1913633011 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c801ffd60 0x557c8029d940 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fda48001d20 tx=0x7fda48000bc0 comp rx=0 tx=0).stop 2024-05-11T08:07:05.944 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.938+0000 7fda5d994640 1 -- 172.21.15.167:0/1913633011 reap_dead start 2024-05-11T08:07:05.944 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.938+0000 7fda6011ff40 1 -- 172.21.15.167:0/1913633011 shutdown_connections 2024-05-11T08:07:05.944 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.938+0000 7fda6011ff40 1 -- 172.21.15.167:0/1913633011 >> 172.21.15.167:0/1913633011 conn(0x557c801fd950 msgr2=0x557c802a27b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:07:05.944 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.938+0000 7fda6011ff40 1 -- 172.21.15.167:0/1913633011 shutdown_connections 2024-05-11T08:07:05.944 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:05.938+0000 7fda6011ff40 1 -- 172.21.15.167:0/1913633011 wait complete. 2024-05-11T08:07:05.945 INFO:teuthology.orchestra.run.smithi167.stdout:a��alphaparent_b 2024-05-11T08:07:05.945 DEBUG:teuthology.orchestra.run.smithi167:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2024-05-11T08:07:05.992 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-05-11T08:07:05.992 INFO:teuthology.orchestra.run.smithi167.stdout: "ino": 1099511628283, 2024-05-11T08:07:05.993 INFO:teuthology.orchestra.run.smithi167.stdout: "ancestors": [ 2024-05-11T08:07:05.993 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T08:07:05.993 INFO:teuthology.orchestra.run.smithi167.stdout: "dirino": 1099511628284, 2024-05-11T08:07:05.993 INFO:teuthology.orchestra.run.smithi167.stdout: "dname": "alpha", 2024-05-11T08:07:05.993 INFO:teuthology.orchestra.run.smithi167.stdout: "version": 7 2024-05-11T08:07:05.993 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T08:07:05.993 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T08:07:05.993 INFO:teuthology.orchestra.run.smithi167.stdout: "dirino": 1, 2024-05-11T08:07:05.994 INFO:teuthology.orchestra.run.smithi167.stdout: "dname": "parent_b", 2024-05-11T08:07:05.994 INFO:teuthology.orchestra.run.smithi167.stdout: "version": 30 2024-05-11T08:07:05.994 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T08:07:05.994 INFO:teuthology.orchestra.run.smithi167.stdout: ], 2024-05-11T08:07:05.994 INFO:teuthology.orchestra.run.smithi167.stdout: "pool": 7, 2024-05-11T08:07:05.994 INFO:teuthology.orchestra.run.smithi167.stdout: "old_pools": [ 2024-05-11T08:07:05.994 INFO:teuthology.orchestra.run.smithi167.stdout: 6 2024-05-11T08:07:05.994 INFO:teuthology.orchestra.run.smithi167.stdout: ] 2024-05-11T08:07:05.994 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-05-11T08:07:05.994 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T08:07:05.995 DEBUG:teuthology.orchestra.run.smithi167:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool data_new getxattr 100000001fb.00000000 parent 2024-05-11T08:07:06.079 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.079+0000 7f6b943c4f40 1 Processor -- start 2024-05-11T08:07:06.079 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.079+0000 7f6b943c4f40 1 -- start start 2024-05-11T08:07:06.083 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.080+0000 7f6b943c4f40 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55ccde6434f0 0x55ccde6c8070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:06.084 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.080+0000 7f6b943c4f40 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ccde6d3080 0x55ccde777480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:06.084 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.080+0000 7f6b943c4f40 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ccde6d3460 0x55ccde6156c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:06.084 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.080+0000 7f6b943c4f40 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55ccde6eb270 con 0x55ccde6d3080 2024-05-11T08:07:06.084 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.080+0000 7f6b943c4f40 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55ccde6eaf90 con 0x55ccde6d3460 2024-05-11T08:07:06.084 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.080+0000 7f6b943c4f40 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x55ccde6eb550 con 0x55ccde6434f0 2024-05-11T08:07:06.084 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.080+0000 7f6b896bd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ccde6d3080 0x55ccde777480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:06.084 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.080+0000 7f6b896bd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ccde6d3080 0x55ccde777480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:55662/0 (socket says 172.21.15.167:55662) 2024-05-11T08:07:06.084 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.080+0000 7f6b896bd640 1 -- 172.21.15.167:0/2200696088 learned_addr learned my addr 172.21.15.167:0/2200696088 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:07:06.084 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.080+0000 7f6b91ebe640 1 --2- 172.21.15.167:0/2200696088 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55ccde6434f0 0x55ccde6c8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:06.084 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.080+0000 7f6b926bf640 1 --2- 172.21.15.167:0/2200696088 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ccde6d3460 0x55ccde6156c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:06.084 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.080+0000 7f6b896bd640 1 -- 172.21.15.167:0/2200696088 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55ccde6434f0 msgr2=0x55ccde6c8070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:07:06.084 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.080+0000 7f6b896bd640 1 --2- 172.21.15.167:0/2200696088 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55ccde6434f0 0x55ccde6c8070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:06.085 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.080+0000 7f6b896bd640 1 -- 172.21.15.167:0/2200696088 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ccde6d3460 msgr2=0x55ccde6156c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:07:06.085 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.080+0000 7f6b896bd640 1 --2- 172.21.15.167:0/2200696088 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ccde6d3460 0x55ccde6156c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:06.085 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.080+0000 7f6b896bd640 1 -- 172.21.15.167:0/2200696088 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ccde57e350 con 0x55ccde6d3080 2024-05-11T08:07:06.085 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.080+0000 7f6b91ebe640 1 --2- 172.21.15.167:0/2200696088 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55ccde6434f0 0x55ccde6c8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T08:07:06.085 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.080+0000 7f6b896bd640 1 --2- 172.21.15.167:0/2200696088 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ccde6d3080 0x55ccde777480 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f6b7c00a7a0 tx=0x7f6b7c00a890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ef4f8b76e17859a server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:06.085 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.080+0000 7f6b90ebc640 1 -- 172.21.15.167:0/2200696088 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b7c018070 con 0x55ccde6d3080 2024-05-11T08:07:06.085 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.080+0000 7f6b90ebc640 1 -- 172.21.15.167:0/2200696088 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b7c007600 con 0x55ccde6d3080 2024-05-11T08:07:06.085 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.080+0000 7f6b90ebc640 1 -- 172.21.15.167:0/2200696088 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b7c0079b0 con 0x55ccde6d3080 2024-05-11T08:07:06.085 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.081+0000 7f6b943c4f40 1 -- 172.21.15.167:0/2200696088 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ccde6d3080 msgr2=0x55ccde777480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:06.085 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.081+0000 7f6b943c4f40 1 --2- 172.21.15.167:0/2200696088 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ccde6d3080 0x55ccde777480 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f6b7c00a7a0 tx=0x7f6b7c00a890 comp rx=0 tx=0).stop 2024-05-11T08:07:06.085 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.081+0000 7f6b943c4f40 1 -- 172.21.15.167:0/2200696088 shutdown_connections 2024-05-11T08:07:06.085 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.081+0000 7f6b943c4f40 1 --2- 172.21.15.167:0/2200696088 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ccde6d3460 0x55ccde6156c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:06.085 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.081+0000 7f6b943c4f40 1 --2- 172.21.15.167:0/2200696088 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ccde6d3080 0x55ccde777480 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:06.085 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.081+0000 7f6b943c4f40 1 --2- 172.21.15.167:0/2200696088 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55ccde6434f0 0x55ccde6c8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:06.085 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.081+0000 7f6b943c4f40 1 -- 172.21.15.167:0/2200696088 >> 172.21.15.167:0/2200696088 conn(0x55ccde613c70 msgr2=0x55ccde6cf530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:07:06.086 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.081+0000 7f6b943c4f40 1 -- 172.21.15.167:0/2200696088 shutdown_connections 2024-05-11T08:07:06.086 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.081+0000 7f6b943c4f40 1 -- 172.21.15.167:0/2200696088 wait complete. 2024-05-11T08:07:06.086 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.081+0000 7f6b943c4f40 1 Processor -- start 2024-05-11T08:07:06.086 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.082+0000 7f6b943c4f40 1 -- start start 2024-05-11T08:07:06.086 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.082+0000 7f6b943c4f40 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ccde6434f0 0x55ccde6e99d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:06.086 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.082+0000 7f6b943c4f40 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ccde6d3080 0x55ccde6d0830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:06.086 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.082+0000 7f6b943c4f40 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55ccde6d3460 0x55ccde6c9690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:06.086 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.082+0000 7f6b943c4f40 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55ccde617210 con 0x55ccde6d3080 2024-05-11T08:07:06.086 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.082+0000 7f6b943c4f40 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55ccde6eddd0 con 0x55ccde6434f0 2024-05-11T08:07:06.086 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.082+0000 7f6b943c4f40 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x55ccde6c7a30 con 0x55ccde6d3460 2024-05-11T08:07:06.086 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.082+0000 7f6b896bd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ccde6d3080 0x55ccde6d0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:06.087 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.082+0000 7f6b896bd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ccde6d3080 0x55ccde6d0830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:55674/0 (socket says 172.21.15.167:55674) 2024-05-11T08:07:06.087 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.082+0000 7f6b896bd640 1 -- 172.21.15.167:0/4275478078 learned_addr learned my addr 172.21.15.167:0/4275478078 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:07:06.087 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.082+0000 7f6b91ebe640 1 --2- 172.21.15.167:0/4275478078 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ccde6434f0 0x55ccde6e99d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:06.087 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.082+0000 7f6b926bf640 1 --2- 172.21.15.167:0/4275478078 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55ccde6d3460 0x55ccde6c9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:06.087 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.083+0000 7f6b896bd640 1 -- 172.21.15.167:0/4275478078 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55ccde6d3460 msgr2=0x55ccde6c9690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:06.087 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.083+0000 7f6b896bd640 1 --2- 172.21.15.167:0/4275478078 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55ccde6d3460 0x55ccde6c9690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:06.087 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.083+0000 7f6b896bd640 1 -- 172.21.15.167:0/4275478078 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ccde6434f0 msgr2=0x55ccde6e99d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:06.087 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.083+0000 7f6b896bd640 1 --2- 172.21.15.167:0/4275478078 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ccde6434f0 0x55ccde6e99d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:06.087 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.083+0000 7f6b896bd640 1 -- 172.21.15.167:0/4275478078 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ccde6cb690 con 0x55ccde6d3080 2024-05-11T08:07:06.087 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.083+0000 7f6b896bd640 1 --2- 172.21.15.167:0/4275478078 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ccde6d3080 0x55ccde6d0830 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f6b7c016040 tx=0x7f6b7c003020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:06.087 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.083+0000 7f6b8b7fe640 1 -- 172.21.15.167:0/4275478078 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b7c022020 con 0x55ccde6d3080 2024-05-11T08:07:06.087 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.083+0000 7f6b8b7fe640 1 -- 172.21.15.167:0/4275478078 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b7c01e670 con 0x55ccde6d3080 2024-05-11T08:07:06.087 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.083+0000 7f6b943c4f40 1 -- 172.21.15.167:0/4275478078 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55ccde6cab40 con 0x55ccde6d3080 2024-05-11T08:07:06.087 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.083+0000 7f6b8b7fe640 1 -- 172.21.15.167:0/4275478078 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b7c018040 con 0x55ccde6d3080 2024-05-11T08:07:06.088 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.083+0000 7f6b943c4f40 1 -- 172.21.15.167:0/4275478078 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ccde6cbc10 con 0x55ccde6d3080 2024-05-11T08:07:06.090 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.084+0000 7f6b8b7fe640 1 -- 172.21.15.167:0/4275478078 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f6b7c025020 con 0x55ccde6d3080 2024-05-11T08:07:06.090 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.085+0000 7f6b8b7fe640 1 --2- 172.21.15.167:0/4275478078 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f6b74074cb0 0x7f6b74077170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:06.090 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.085+0000 7f6b91ebe640 1 --2- 172.21.15.167:0/4275478078 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f6b74074cb0 0x7f6b74077170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:06.090 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.085+0000 7f6b8b7fe640 1 -- 172.21.15.167:0/4275478078 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 6496+0+0 (secure 0 0 0) 0x7f6b7c097650 con 0x55ccde6d3080 2024-05-11T08:07:06.090 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.085+0000 7f6b943c4f40 1 --2- 172.21.15.167:0/4275478078 >> [v2:172.21.15.167:6804/3863307356,v1:172.21.15.167:6806/3863307356] conn(0x55ccde6d0d70 0x55ccde6ca0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:06.090 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.085+0000 7f6b943c4f40 1 -- 172.21.15.167:0/4275478078 --> [v2:172.21.15.167:6804/3863307356,v1:172.21.15.167:6806/3863307356] -- osd_op(unknown.0.0:1 7.13 7:ca8fdbba:::100000001fb.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44) -- 0x55ccde6cc130 con 0x55ccde6d0d70 2024-05-11T08:07:06.090 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.085+0000 7f6b926bf640 1 --2- 172.21.15.167:0/4275478078 >> [v2:172.21.15.167:6804/3863307356,v1:172.21.15.167:6806/3863307356] conn(0x55ccde6d0d70 0x55ccde6ca0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:06.090 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.085+0000 7f6b91ebe640 1 --2- 172.21.15.167:0/4275478078 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f6b74074cb0 0x7f6b74077170 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x55ccde63e8d0 tx=0x7f6b84005ac0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:06.090 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.085+0000 7f6b926bf640 1 --2- 172.21.15.167:0/4275478078 >> [v2:172.21.15.167:6804/3863307356,v1:172.21.15.167:6806/3863307356] conn(0x55ccde6d0d70 0x55ccde6ca0e0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:06.091 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.086+0000 7f6b926bf640 1 -- 172.21.15.167:0/4275478078 <== osd.0 v2:172.21.15.167:6804/3863307356 1 ==== osd_op_reply(1 100000001fb.00000000 [getxattr (103) out=103b] v0'0 uv1 ondisk = 0) ==== 164+0+103 (crc 0 0 0) 0x7f6b8c051860 con 0x55ccde6d0d70 2024-05-11T08:07:06.091 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.086+0000 7f6b943c4f40 1 -- 172.21.15.167:0/4275478078 >> [v2:172.21.15.167:6804/3863307356,v1:172.21.15.167:6806/3863307356] conn(0x55ccde6d0d70 msgr2=0x55ccde6ca0e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:06.091 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.086+0000 7f6b943c4f40 1 --2- 172.21.15.167:0/4275478078 >> [v2:172.21.15.167:6804/3863307356,v1:172.21.15.167:6806/3863307356] conn(0x55ccde6d0d70 0x55ccde6ca0e0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:06.091 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.086+0000 7f6b943c4f40 1 -- 172.21.15.167:0/4275478078 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f6b74074cb0 msgr2=0x7f6b74077170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:06.091 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.086+0000 7f6b943c4f40 1 --2- 172.21.15.167:0/4275478078 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f6b74074cb0 0x7f6b74077170 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x55ccde63e8d0 tx=0x7f6b84005ac0 comp rx=0 tx=0).stop 2024-05-11T08:07:06.091 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.086+0000 7f6b943c4f40 1 -- 172.21.15.167:0/4275478078 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ccde6d3080 msgr2=0x55ccde6d0830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:06.091 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.086+0000 7f6b943c4f40 1 --2- 172.21.15.167:0/4275478078 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ccde6d3080 0x55ccde6d0830 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f6b7c016040 tx=0x7f6b7c003020 comp rx=0 tx=0).stop 2024-05-11T08:07:06.091 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.086+0000 7f6b926bf640 1 -- 172.21.15.167:0/4275478078 reap_dead start 2024-05-11T08:07:06.091 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.086+0000 7f6b943c4f40 1 -- 172.21.15.167:0/4275478078 shutdown_connections 2024-05-11T08:07:06.091 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.086+0000 7f6b943c4f40 1 -- 172.21.15.167:0/4275478078 >> 172.21.15.167:0/4275478078 conn(0x55ccde613c70 msgr2=0x55ccde6cfdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:07:06.091 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.086+0000 7f6b943c4f40 1 -- 172.21.15.167:0/4275478078 shutdown_connections 2024-05-11T08:07:06.091 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.086+0000 7f6b943c4f40 1 -- 172.21.15.167:0/4275478078 wait complete. 2024-05-11T08:07:06.092 INFO:teuthology.orchestra.run.smithi167.stdout:a��alphaparent_b 2024-05-11T08:07:06.093 DEBUG:teuthology.orchestra.run.smithi167:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2024-05-11T08:07:06.180 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-05-11T08:07:06.180 INFO:teuthology.orchestra.run.smithi167.stdout: "ino": 1099511628283, 2024-05-11T08:07:06.180 INFO:teuthology.orchestra.run.smithi167.stdout: "ancestors": [ 2024-05-11T08:07:06.180 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T08:07:06.180 INFO:teuthology.orchestra.run.smithi167.stdout: "dirino": 1099511628284, 2024-05-11T08:07:06.180 INFO:teuthology.orchestra.run.smithi167.stdout: "dname": "alpha", 2024-05-11T08:07:06.181 INFO:teuthology.orchestra.run.smithi167.stdout: "version": 7 2024-05-11T08:07:06.181 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T08:07:06.181 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T08:07:06.181 INFO:teuthology.orchestra.run.smithi167.stdout: "dirino": 1, 2024-05-11T08:07:06.181 INFO:teuthology.orchestra.run.smithi167.stdout: "dname": "parent_b", 2024-05-11T08:07:06.181 INFO:teuthology.orchestra.run.smithi167.stdout: "version": 30 2024-05-11T08:07:06.181 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T08:07:06.181 INFO:teuthology.orchestra.run.smithi167.stdout: ], 2024-05-11T08:07:06.181 INFO:teuthology.orchestra.run.smithi167.stdout: "pool": 7, 2024-05-11T08:07:06.181 INFO:teuthology.orchestra.run.smithi167.stdout: "old_pools": [ 2024-05-11T08:07:06.181 INFO:teuthology.orchestra.run.smithi167.stdout: 6 2024-05-11T08:07:06.181 INFO:teuthology.orchestra.run.smithi167.stdout: ] 2024-05-11T08:07:06.181 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-05-11T08:07:06.181 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T08:07:06.182 DEBUG:teuthology.orchestra.run.smithi167:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool data_new getxattr 100000001fb.00000000 layout 2024-05-11T08:07:06.266 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.266+0000 7fd88bad1f40 1 Processor -- start 2024-05-11T08:07:06.267 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.266+0000 7fd88bad1f40 1 -- start start 2024-05-11T08:07:06.269 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.267+0000 7fd88bad1f40 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5648bdb5b1f0 0x5648bdc1c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:06.269 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.267+0000 7fd88bad1f40 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5648bdc32750 0x5648bdb88de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:06.269 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.267+0000 7fd88bad1f40 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x5648bdc340e0 0x5648bdba61c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:06.269 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.267+0000 7fd88bad1f40 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5648bdb5bba0 con 0x5648bdb5b1f0 2024-05-11T08:07:06.270 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.267+0000 7fd88bad1f40 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5648bdb04f50 con 0x5648bdc32750 2024-05-11T08:07:06.270 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.267+0000 7fd88bad1f40 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x5648bdba6ea0 con 0x5648bdc340e0 2024-05-11T08:07:06.270 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.267+0000 7fd8895cb640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5648bdb5b1f0 0x5648bdc1c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:06.270 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.267+0000 7fd8895cb640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5648bdb5b1f0 0x5648bdc1c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:55682/0 (socket says 172.21.15.167:55682) 2024-05-11T08:07:06.270 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.267+0000 7fd8895cb640 1 -- 172.21.15.167:0/681698197 learned_addr learned my addr 172.21.15.167:0/681698197 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:07:06.270 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.267+0000 7fd888dca640 1 --2- 172.21.15.167:0/681698197 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5648bdc32750 0x5648bdb88de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:06.270 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.267+0000 7fd889dcc640 1 --2- 172.21.15.167:0/681698197 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x5648bdc340e0 0x5648bdba61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:06.270 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.268+0000 7fd8895cb640 1 -- 172.21.15.167:0/681698197 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x5648bdc340e0 msgr2=0x5648bdba61c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:07:06.270 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.268+0000 7fd8895cb640 1 --2- 172.21.15.167:0/681698197 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x5648bdc340e0 0x5648bdba61c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:06.270 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.268+0000 7fd8895cb640 1 -- 172.21.15.167:0/681698197 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5648bdc32750 msgr2=0x5648bdb88de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:07:06.270 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.268+0000 7fd8895cb640 1 --2- 172.21.15.167:0/681698197 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5648bdc32750 0x5648bdb88de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:06.270 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.268+0000 7fd8895cb640 1 -- 172.21.15.167:0/681698197 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648bdc2a220 con 0x5648bdb5b1f0 2024-05-11T08:07:06.270 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.268+0000 7fd8895cb640 1 --2- 172.21.15.167:0/681698197 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5648bdb5b1f0 0x5648bdc1c7f0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fd8840259c0 tx=0x7fd884051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26df71b893c8f401 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:06.271 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.268+0000 7fd8837fe640 1 -- 172.21.15.167:0/681698197 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd884056070 con 0x5648bdb5b1f0 2024-05-11T08:07:06.271 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.268+0000 7fd8837fe640 1 -- 172.21.15.167:0/681698197 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd884052ac0 con 0x5648bdb5b1f0 2024-05-11T08:07:06.271 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.268+0000 7fd8837fe640 1 -- 172.21.15.167:0/681698197 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd884065440 con 0x5648bdb5b1f0 2024-05-11T08:07:06.271 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.268+0000 7fd88bad1f40 1 -- 172.21.15.167:0/681698197 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5648bdb5b1f0 msgr2=0x5648bdc1c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:06.271 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.268+0000 7fd88bad1f40 1 --2- 172.21.15.167:0/681698197 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5648bdb5b1f0 0x5648bdc1c7f0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fd8840259c0 tx=0x7fd884051000 comp rx=0 tx=0).stop 2024-05-11T08:07:06.271 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.268+0000 7fd88bad1f40 1 -- 172.21.15.167:0/681698197 shutdown_connections 2024-05-11T08:07:06.271 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.268+0000 7fd88bad1f40 1 --2- 172.21.15.167:0/681698197 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x5648bdc340e0 0x5648bdba61c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:06.271 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.268+0000 7fd88bad1f40 1 --2- 172.21.15.167:0/681698197 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5648bdc32750 0x5648bdb88de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:06.271 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.268+0000 7fd88bad1f40 1 --2- 172.21.15.167:0/681698197 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5648bdb5b1f0 0x5648bdc1c7f0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:06.271 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.268+0000 7fd88bad1f40 1 -- 172.21.15.167:0/681698197 >> 172.21.15.167:0/681698197 conn(0x5648bdb5c4f0 msgr2=0x5648bdb5c8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:07:06.271 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.269+0000 7fd88bad1f40 1 -- 172.21.15.167:0/681698197 shutdown_connections 2024-05-11T08:07:06.271 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.269+0000 7fd88bad1f40 1 -- 172.21.15.167:0/681698197 wait complete. 2024-05-11T08:07:06.271 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.269+0000 7fd88bad1f40 1 Processor -- start 2024-05-11T08:07:06.272 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.269+0000 7fd88bad1f40 1 -- start start 2024-05-11T08:07:06.275 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.270+0000 7fd88bad1f40 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x5648bdb5b1f0 0x5648bdc1b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:06.275 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.270+0000 7fd88bad1f40 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5648bdc32750 0x5648bdb9e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:06.276 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.270+0000 7fd88bad1f40 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5648bdc340e0 0x5648bdba61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:06.276 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.270+0000 7fd88bad1f40 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5648bdc345d0 con 0x5648bdc340e0 2024-05-11T08:07:06.276 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.270+0000 7fd88bad1f40 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5648bdb89320 con 0x5648bdc32750 2024-05-11T08:07:06.276 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.270+0000 7fd88bad1f40 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x5648bdc348b0 con 0x5648bdb5b1f0 2024-05-11T08:07:06.276 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.270+0000 7fd889dcc640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5648bdc340e0 0x5648bdba61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:06.276 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.270+0000 7fd889dcc640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5648bdc340e0 0x5648bdba61c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:55690/0 (socket says 172.21.15.167:55690) 2024-05-11T08:07:06.276 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.270+0000 7fd889dcc640 1 -- 172.21.15.167:0/2659814453 learned_addr learned my addr 172.21.15.167:0/2659814453 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:07:06.276 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.270+0000 7fd8895cb640 1 --2- 172.21.15.167:0/2659814453 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x5648bdb5b1f0 0x5648bdc1b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:06.276 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.270+0000 7fd888dca640 1 --2- 172.21.15.167:0/2659814453 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5648bdc32750 0x5648bdb9e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:06.276 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.270+0000 7fd889dcc640 1 -- 172.21.15.167:0/2659814453 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x5648bdb5b1f0 msgr2=0x5648bdc1b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:06.276 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.270+0000 7fd889dcc640 1 --2- 172.21.15.167:0/2659814453 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x5648bdb5b1f0 0x5648bdc1b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:06.276 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.270+0000 7fd889dcc640 1 -- 172.21.15.167:0/2659814453 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5648bdc32750 msgr2=0x5648bdb9e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:06.276 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.270+0000 7fd889dcc640 1 --2- 172.21.15.167:0/2659814453 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5648bdc32750 0x5648bdb9e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:06.276 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.270+0000 7fd889dcc640 1 -- 172.21.15.167:0/2659814453 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648bdb9f000 con 0x5648bdc340e0 2024-05-11T08:07:06.277 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.270+0000 7fd889dcc640 1 --2- 172.21.15.167:0/2659814453 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5648bdc340e0 0x5648bdba61c0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fd87c008630 tx=0x7fd87c008720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:06.277 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.270+0000 7fd8827fc640 1 -- 172.21.15.167:0/2659814453 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd87c019020 con 0x5648bdc340e0 2024-05-11T08:07:06.277 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.270+0000 7fd8827fc640 1 -- 172.21.15.167:0/2659814453 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd87c008e50 con 0x5648bdc340e0 2024-05-11T08:07:06.277 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.270+0000 7fd88bad1f40 1 -- 172.21.15.167:0/2659814453 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5648bdb9f2e0 con 0x5648bdc340e0 2024-05-11T08:07:06.277 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.271+0000 7fd8827fc640 1 -- 172.21.15.167:0/2659814453 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd87c015ae0 con 0x5648bdc340e0 2024-05-11T08:07:06.277 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.271+0000 7fd88bad1f40 1 -- 172.21.15.167:0/2659814453 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648bdc1ba70 con 0x5648bdc340e0 2024-05-11T08:07:06.277 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.272+0000 7fd8827fc640 1 -- 172.21.15.167:0/2659814453 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fd87c01c020 con 0x5648bdc340e0 2024-05-11T08:07:06.277 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.272+0000 7fd8827fc640 1 --2- 172.21.15.167:0/2659814453 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fd86c074ce0 0x7fd86c0771a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:06.277 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.273+0000 7fd8827fc640 1 -- 172.21.15.167:0/2659814453 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 6496+0+0 (secure 0 0 0) 0x7fd87c096d50 con 0x5648bdc340e0 2024-05-11T08:07:06.277 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.273+0000 7fd8895cb640 1 --2- 172.21.15.167:0/2659814453 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fd86c074ce0 0x7fd86c0771a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:06.277 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.273+0000 7fd88bad1f40 1 --2- 172.21.15.167:0/2659814453 >> [v2:172.21.15.167:6804/3863307356,v1:172.21.15.167:6806/3863307356] conn(0x5648bdb9ea60 0x5648bdba0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:06.277 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.273+0000 7fd88bad1f40 1 -- 172.21.15.167:0/2659814453 --> [v2:172.21.15.167:6804/3863307356,v1:172.21.15.167:6806/3863307356] -- osd_op(unknown.0.0:1 7.13 7:ca8fdbba:::100000001fb.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44) -- 0x5648bdba10b0 con 0x5648bdb9ea60 2024-05-11T08:07:06.277 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.273+0000 7fd888dca640 1 --2- 172.21.15.167:0/2659814453 >> [v2:172.21.15.167:6804/3863307356,v1:172.21.15.167:6806/3863307356] conn(0x5648bdb9ea60 0x5648bdba0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:06.277 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.273+0000 7fd8895cb640 1 --2- 172.21.15.167:0/2659814453 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fd86c074ce0 0x7fd86c0771a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd884005fd0 tx=0x7fd884052000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:06.278 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.273+0000 7fd888dca640 1 --2- 172.21.15.167:0/2659814453 >> [v2:172.21.15.167:6804/3863307356,v1:172.21.15.167:6806/3863307356] conn(0x5648bdb9ea60 0x5648bdba0b70 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:06.278 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.274+0000 7fd888dca640 1 -- 172.21.15.167:0/2659814453 <== osd.0 v2:172.21.15.167:6804/3863307356 1 ==== osd_op_reply(1 100000001fb.00000000 [getxattr (30) out=30b] v0'0 uv1 ondisk = 0) ==== 164+0+30 (crc 0 0 0) 0x7fd874009040 con 0x5648bdb9ea60 2024-05-11T08:07:06.278 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.274+0000 7fd88bad1f40 1 -- 172.21.15.167:0/2659814453 >> [v2:172.21.15.167:6804/3863307356,v1:172.21.15.167:6806/3863307356] conn(0x5648bdb9ea60 msgr2=0x5648bdba0b70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:06.278 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.274+0000 7fd88bad1f40 1 --2- 172.21.15.167:0/2659814453 >> [v2:172.21.15.167:6804/3863307356,v1:172.21.15.167:6806/3863307356] conn(0x5648bdb9ea60 0x5648bdba0b70 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:06.278 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.274+0000 7fd88bad1f40 1 -- 172.21.15.167:0/2659814453 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fd86c074ce0 msgr2=0x7fd86c0771a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:06.278 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.274+0000 7fd88bad1f40 1 --2- 172.21.15.167:0/2659814453 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fd86c074ce0 0x7fd86c0771a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd884005fd0 tx=0x7fd884052000 comp rx=0 tx=0).stop 2024-05-11T08:07:06.278 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.274+0000 7fd88bad1f40 1 -- 172.21.15.167:0/2659814453 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5648bdc340e0 msgr2=0x5648bdba61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:06.278 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.274+0000 7fd88bad1f40 1 --2- 172.21.15.167:0/2659814453 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5648bdc340e0 0x5648bdba61c0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fd87c008630 tx=0x7fd87c008720 comp rx=0 tx=0).stop 2024-05-11T08:07:06.278 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.274+0000 7fd889dcc640 1 -- 172.21.15.167:0/2659814453 reap_dead start 2024-05-11T08:07:06.278 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.274+0000 7fd88bad1f40 1 -- 172.21.15.167:0/2659814453 shutdown_connections 2024-05-11T08:07:06.278 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.274+0000 7fd88bad1f40 1 -- 172.21.15.167:0/2659814453 >> 172.21.15.167:0/2659814453 conn(0x5648bdb5c4f0 msgr2=0x5648bdb8cc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:07:06.278 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.274+0000 7fd88bad1f40 1 -- 172.21.15.167:0/2659814453 shutdown_connections 2024-05-11T08:07:06.278 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.274+0000 7fd88bad1f40 1 -- 172.21.15.167:0/2659814453 wait complete. 2024-05-11T08:07:06.279 INFO:teuthology.orchestra.run.smithi167.stdout:@@ 2024-05-11T08:07:06.280 DEBUG:teuthology.orchestra.run.smithi167:> ceph-dencoder type file_layout_t import - decode dump_json 2024-05-11T08:07:06.371 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-05-11T08:07:06.371 INFO:teuthology.orchestra.run.smithi167.stdout: "stripe_unit": 4194304, 2024-05-11T08:07:06.371 INFO:teuthology.orchestra.run.smithi167.stdout: "stripe_count": 1, 2024-05-11T08:07:06.372 INFO:teuthology.orchestra.run.smithi167.stdout: "object_size": 4194304, 2024-05-11T08:07:06.372 INFO:teuthology.orchestra.run.smithi167.stdout: "pool_id": 7, 2024-05-11T08:07:06.372 INFO:teuthology.orchestra.run.smithi167.stdout: "pool_ns": "" 2024-05-11T08:07:06.372 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-05-11T08:07:06.372 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T08:07:06.374 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T08:07:06.374 DEBUG:teuthology.orchestra.run.smithi167:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parent_c) 2024-05-11T08:07:06.431 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T08:07:06.431 DEBUG:teuthology.orchestra.run.smithi167:> (cd /home/ubuntu/cephtest/mnt.0 && exec mv parent_b/alpha parent_c/alpha) 2024-05-11T08:07:06.491 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-11T08:07:06.632 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.632+0000 7f377e2c8640 1 Processor -- start 2024-05-11T08:07:06.632 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.632+0000 7f377e2c8640 1 -- start start 2024-05-11T08:07:06.634 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.633+0000 7f377e2c8640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f3778071800 0x7f377815ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:06.634 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.633+0000 7f377e2c8640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37780b4f50 con 0x7f3778160470 2024-05-11T08:07:06.634 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.633+0000 7f377e2c8640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37780b5230 con 0x7f3778148900 2024-05-11T08:07:06.634 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.633+0000 7f377e2c8640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f37780b5510 con 0x7f3778071800 2024-05-11T08:07:06.634 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.633+0000 7f377c865640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f3778160470 0x7f37781608b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:47388/0 (socket says 172.21.15.167:47388) 2024-05-11T08:07:06.634 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.633+0000 7f377c865640 1 -- 172.21.15.167:0/3844336200 learned_addr learned my addr 172.21.15.167:0/3844336200 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T08:07:06.634 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.633+0000 7f37777fe640 1 --2- 172.21.15.167:0/3844336200 >> v2:172.21.15.204:3300/0 conn(0x7f3778071800 0x7f377815ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:06.634 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.633+0000 7f3776ffd640 1 -- 172.21.15.167:0/3844336200 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3294363689 0 0) 0x7f37780b4f50 con 0x7f3778160470 2024-05-11T08:07:06.634 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.633+0000 7f3776ffd640 1 -- 172.21.15.167:0/3844336200 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3760003520 con 0x7f3778160470 2024-05-11T08:07:06.634 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.633+0000 7f3776ffd640 1 -- 172.21.15.167:0/3844336200 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 626287049 0 0) 0x7f37780b5230 con 0x7f3778148900 2024-05-11T08:07:06.634 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.633+0000 7f3776ffd640 1 -- 172.21.15.167:0/3844336200 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f37780b4f50 con 0x7f3778148900 2024-05-11T08:07:06.635 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.633+0000 7f3776ffd640 1 -- 172.21.15.167:0/3844336200 <== mon.0 v1:172.21.15.167:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1370861907 0 0) 0x7f3760003520 con 0x7f3778160470 2024-05-11T08:07:06.635 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.633+0000 7f37777fe640 1 -- 172.21.15.167:0/3844336200 >> v1:172.21.15.204:6789/0 conn(0x7f3778148900 legacy=0x7f37780710f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:06.635 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.634+0000 7f37777fe640 1 -- 172.21.15.167:0/3844336200 >> v1:172.21.15.167:6789/0 conn(0x7f3778160470 legacy=0x7f37781608b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:06.635 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.634+0000 7f37777fe640 1 -- 172.21.15.167:0/3844336200 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37780b57f0 con 0x7f3778071800 2024-05-11T08:07:06.636 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.634+0000 7f37777fe640 1 --2- 172.21.15.167:0/3844336200 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3778071800 0x7f377815ff30 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f376c0097a0 tx=0x7f376c02a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=42f39e46846b1f27 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:06.636 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.634+0000 7f3776ffd640 1 -- 172.21.15.167:0/3844336200 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f376c037070 con 0x7f3778071800 2024-05-11T08:07:06.636 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.634+0000 7f3776ffd640 1 -- 172.21.15.167:0/3844336200 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f376c02ac50 con 0x7f3778071800 2024-05-11T08:07:06.636 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.634+0000 7f3776ffd640 1 -- 172.21.15.167:0/3844336200 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3770001a70 con 0x7f3778071800 2024-05-11T08:07:06.636 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.635+0000 7f377e2c8640 1 -- 172.21.15.167:0/3844336200 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3778071800 msgr2=0x7f377815ff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:06.636 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.635+0000 7f377e2c8640 1 --2- 172.21.15.167:0/3844336200 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3778071800 0x7f377815ff30 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f376c0097a0 tx=0x7f376c02a1c0 comp rx=0 tx=0).stop 2024-05-11T08:07:06.637 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.635+0000 7f377e2c8640 1 -- 172.21.15.167:0/3844336200 shutdown_connections 2024-05-11T08:07:06.637 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.635+0000 7f377e2c8640 1 --2- 172.21.15.167:0/3844336200 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3778071800 0x7f377815ff30 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:06.637 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.635+0000 7f377e2c8640 1 -- 172.21.15.167:0/3844336200 >> 172.21.15.167:0/3844336200 conn(0x7f37780d1e40 msgr2=0x7f37780d2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:07:06.637 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.635+0000 7f377e2c8640 1 -- 172.21.15.167:0/3844336200 shutdown_connections 2024-05-11T08:07:06.637 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.635+0000 7f377e2c8640 1 -- 172.21.15.167:0/3844336200 wait complete. 2024-05-11T08:07:06.637 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.636+0000 7f377e2c8640 1 Processor -- start 2024-05-11T08:07:06.637 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.636+0000 7f377e2c8640 1 -- start start 2024-05-11T08:07:06.639 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.637+0000 7f377e2c8640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3778071800 0x7f3778142970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:06.639 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.637+0000 7f377e2c8640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3778148900 0x7f377815c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:06.639 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.637+0000 7f377e2c8640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3778160470 0x7f37781e8b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:06.639 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.637+0000 7f377e2c8640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f377815f900 con 0x7f3778071800 2024-05-11T08:07:06.640 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.637+0000 7f377e2c8640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f3778148ce0 con 0x7f3778148900 2024-05-11T08:07:06.640 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.637+0000 7f377e2c8640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f377815fa80 con 0x7f3778160470 2024-05-11T08:07:06.640 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.637+0000 7f3777fff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3778071800 0x7f3778142970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:06.640 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.637+0000 7f3777fff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3778071800 0x7f3778142970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:55702/0 (socket says 172.21.15.167:55702) 2024-05-11T08:07:06.640 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.637+0000 7f3777fff640 1 -- 172.21.15.167:0/1584397201 learned_addr learned my addr 172.21.15.167:0/1584397201 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:07:06.640 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.637+0000 7f37777fe640 1 --2- 172.21.15.167:0/1584397201 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3778148900 0x7f377815c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:06.640 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.637+0000 7f377c865640 1 --2- 172.21.15.167:0/1584397201 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3778160470 0x7f37781e8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:06.640 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.638+0000 7f3777fff640 1 -- 172.21.15.167:0/1584397201 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3778160470 msgr2=0x7f37781e8b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:06.640 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.638+0000 7f3777fff640 1 --2- 172.21.15.167:0/1584397201 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3778160470 0x7f37781e8b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:06.640 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.638+0000 7f3777fff640 1 -- 172.21.15.167:0/1584397201 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3778148900 msgr2=0x7f377815c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:06.640 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.638+0000 7f3777fff640 1 --2- 172.21.15.167:0/1584397201 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3778148900 0x7f377815c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:06.640 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.638+0000 7f3777fff640 1 -- 172.21.15.167:0/1584397201 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3778142f50 con 0x7f3778071800 2024-05-11T08:07:06.640 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.638+0000 7f377c865640 1 --2- 172.21.15.167:0/1584397201 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3778160470 0x7f37781e8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T08:07:06.641 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.638+0000 7f3777fff640 1 --2- 172.21.15.167:0/1584397201 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3778071800 0x7f3778142970 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f376800c4c0 tx=0x7f376800c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:06.641 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.638+0000 7f3774ff9640 1 -- 172.21.15.167:0/1584397201 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37680052d0 con 0x7f3778071800 2024-05-11T08:07:06.641 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.638+0000 7f3774ff9640 1 -- 172.21.15.167:0/1584397201 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3768005490 con 0x7f3778071800 2024-05-11T08:07:06.641 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.638+0000 7f3774ff9640 1 -- 172.21.15.167:0/1584397201 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3768012870 con 0x7f3778071800 2024-05-11T08:07:06.641 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.638+0000 7f377e2c8640 1 -- 172.21.15.167:0/1584397201 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3778143230 con 0x7f3778071800 2024-05-11T08:07:06.641 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.638+0000 7f377e2c8640 1 -- 172.21.15.167:0/1584397201 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37781437f0 con 0x7f3778071800 2024-05-11T08:07:06.641 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.639+0000 7f377e2c8640 1 -- 172.21.15.167:0/1584397201 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3778069bc0 con 0x7f3778071800 2024-05-11T08:07:06.641 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.640+0000 7f3774ff9640 1 -- 172.21.15.167:0/1584397201 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f3768012a10 con 0x7f3778071800 2024-05-11T08:07:06.642 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.640+0000 7f3774ff9640 1 --2- 172.21.15.167:0/1584397201 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f37640762d0 0x7f3764078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:06.642 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.640+0000 7f3774ff9640 1 -- 172.21.15.167:0/1584397201 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 6496+0+0 (secure 0 0 0) 0x7f37680974b0 con 0x7f3778071800 2024-05-11T08:07:06.642 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.640+0000 7f37777fe640 1 --2- 172.21.15.167:0/1584397201 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f37640762d0 0x7f3764078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:06.642 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.641+0000 7f37777fe640 1 --2- 172.21.15.167:0/1584397201 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f37640762d0 0x7f3764078790 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f376c034000 tx=0x7f376c02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:06.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.645+0000 7f3774ff9640 1 -- 172.21.15.167:0/1584397201 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f3768060b00 con 0x7f3778071800 2024-05-11T08:07:06.866 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.866+0000 7f377e2c8640 1 -- 172.21.15.167:0/1584397201 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f37780b6cf0 con 0x7f3778071800 2024-05-11T08:07:06.867 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.867+0000 7f3774ff9640 1 -- 172.21.15.167:0/1584397201 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5192 (secure 0 0 0) 0x7f3768060400 con 0x7f3778071800 2024-05-11T08:07:06.867 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T08:07:06.867 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4604,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2674983673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":2674983673},{"type":"v1","addr":"172.21.15.167:6837","nonce":2674983673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4607,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6834/1018548066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1018548066},{"type":"v1","addr":"172.21.15.204:6834","nonce":1018548066}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4729,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3369722249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3369722249},{"type":"v1","addr":"172.21.15.167:6836","nonce":3369722249}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-11T08:06:47.943939+0000","modified":"2024-05-11T08:07:04.006894+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4750},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4750":{"gid":4750,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.204:6835/469864006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6833","nonce":469864006},{"type":"v1","addr":"172.21.15.204:6835","nonce":469864006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6,7],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4750,"qdb_cluster":[4750]},"id":2}]} 2024-05-11T08:07:06.867 INFO:teuthology.orchestra.run.smithi167.stderr:dumped fsmap epoch 17 2024-05-11T08:07:06.870 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.869+0000 7f377e2c8640 1 -- 172.21.15.167:0/1584397201 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f37640762d0 msgr2=0x7f3764078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:06.870 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.869+0000 7f377e2c8640 1 --2- 172.21.15.167:0/1584397201 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f37640762d0 0x7f3764078790 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f376c034000 tx=0x7f376c02e000 comp rx=0 tx=0).stop 2024-05-11T08:07:06.870 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.869+0000 7f377e2c8640 1 -- 172.21.15.167:0/1584397201 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3778071800 msgr2=0x7f3778142970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:06.870 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.869+0000 7f377e2c8640 1 --2- 172.21.15.167:0/1584397201 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3778071800 0x7f3778142970 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f376800c4c0 tx=0x7f376800c980 comp rx=0 tx=0).stop 2024-05-11T08:07:06.870 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.869+0000 7f377e2c8640 1 -- 172.21.15.167:0/1584397201 shutdown_connections 2024-05-11T08:07:06.870 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.869+0000 7f377e2c8640 1 --2- 172.21.15.167:0/1584397201 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f37640762d0 0x7f3764078790 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:06.870 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.869+0000 7f377e2c8640 1 --2- 172.21.15.167:0/1584397201 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3778160470 0x7f37781e8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:06.870 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.869+0000 7f377e2c8640 1 --2- 172.21.15.167:0/1584397201 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3778148900 0x7f377815c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:06.870 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.869+0000 7f377e2c8640 1 --2- 172.21.15.167:0/1584397201 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3778071800 0x7f3778142970 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:06.870 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.869+0000 7f377e2c8640 1 -- 172.21.15.167:0/1584397201 >> 172.21.15.167:0/1584397201 conn(0x7f37780d1e40 msgr2=0x7f377815e5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:07:06.870 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.869+0000 7f377e2c8640 1 -- 172.21.15.167:0/1584397201 shutdown_connections 2024-05-11T08:07:06.870 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:06.869+0000 7f377e2c8640 1 -- 172.21.15.167:0/1584397201 wait complete. 2024-05-11T08:07:06.880 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json flush journal 2024-05-11T08:07:07.014 INFO:teuthology.orchestra.run.smithi204.stdout:{"message":"","return_code":0} 2024-05-11T08:07:07.015 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-05-11T08:07:07.015 DEBUG:teuthology.orchestra.run.smithi167:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001fb.00000000 parent 2024-05-11T08:07:07.059 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.059+0000 7f52c7fb5f40 1 Processor -- start 2024-05-11T08:07:07.059 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.059+0000 7f52c7fb5f40 1 -- start start 2024-05-11T08:07:07.064 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.059+0000 7f52c7fb5f40 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ffb48c6fc0 0x55ffb499fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:07.064 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.059+0000 7f52c7fb5f40 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ffb48c9d30 0x55ffb49970e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:07.064 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.059+0000 7f52c7fb5f40 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55ffb499f760 0x55ffb490bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:07.064 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.059+0000 7f52c7fb5f40 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55ffb48c73f0 con 0x55ffb48c9d30 2024-05-11T08:07:07.064 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.059+0000 7f52c7fb5f40 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55ffb486ff50 con 0x55ffb48c6fc0 2024-05-11T08:07:07.064 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.059+0000 7f52c7fb5f40 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x55ffb49118e0 con 0x55ffb499f760 2024-05-11T08:07:07.064 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.059+0000 7f52bd2ae640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ffb48c9d30 0x55ffb49970e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:07.064 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.060+0000 7f52bd2ae640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ffb48c9d30 0x55ffb49970e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:55714/0 (socket says 172.21.15.167:55714) 2024-05-11T08:07:07.064 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.060+0000 7f52bd2ae640 1 -- 172.21.15.167:0/1075244406 learned_addr learned my addr 172.21.15.167:0/1075244406 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:07:07.064 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.060+0000 7f52c5aaf640 1 --2- 172.21.15.167:0/1075244406 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ffb48c6fc0 0x55ffb499fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:07.065 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.060+0000 7f52c62b0640 1 --2- 172.21.15.167:0/1075244406 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55ffb499f760 0x55ffb490bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:07.065 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.060+0000 7f52bd2ae640 1 -- 172.21.15.167:0/1075244406 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55ffb499f760 msgr2=0x55ffb490bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:07:07.065 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.060+0000 7f52bd2ae640 1 --2- 172.21.15.167:0/1075244406 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55ffb499f760 0x55ffb490bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:07.065 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.060+0000 7f52bd2ae640 1 -- 172.21.15.167:0/1075244406 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ffb48c6fc0 msgr2=0x55ffb499fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:07:07.065 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.060+0000 7f52bd2ae640 1 --2- 172.21.15.167:0/1075244406 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ffb48c6fc0 0x55ffb499fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:07.065 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.060+0000 7f52bd2ae640 1 -- 172.21.15.167:0/1075244406 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ffb48c9610 con 0x55ffb48c9d30 2024-05-11T08:07:07.065 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.060+0000 7f52bd2ae640 1 --2- 172.21.15.167:0/1075244406 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ffb48c9d30 0x55ffb49970e0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f52b0025d20 tx=0x7f52b0051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=858b83cf7ec99d02 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:07.065 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.060+0000 7f52c4aad640 1 -- 172.21.15.167:0/1075244406 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b0053880 con 0x55ffb48c9d30 2024-05-11T08:07:07.065 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.060+0000 7f52c4aad640 1 -- 172.21.15.167:0/1075244406 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52b0053a40 con 0x55ffb48c9d30 2024-05-11T08:07:07.065 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.060+0000 7f52c4aad640 1 -- 172.21.15.167:0/1075244406 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b0053e30 con 0x55ffb48c9d30 2024-05-11T08:07:07.065 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.061+0000 7f52c7fb5f40 1 -- 172.21.15.167:0/1075244406 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ffb48c9d30 msgr2=0x55ffb49970e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:07.065 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.061+0000 7f52c7fb5f40 1 --2- 172.21.15.167:0/1075244406 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ffb48c9d30 0x55ffb49970e0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f52b0025d20 tx=0x7f52b0051000 comp rx=0 tx=0).stop 2024-05-11T08:07:07.066 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.061+0000 7f52c7fb5f40 1 -- 172.21.15.167:0/1075244406 shutdown_connections 2024-05-11T08:07:07.066 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.061+0000 7f52c7fb5f40 1 --2- 172.21.15.167:0/1075244406 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55ffb499f760 0x55ffb490bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:07.066 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.061+0000 7f52c7fb5f40 1 --2- 172.21.15.167:0/1075244406 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ffb48c9d30 0x55ffb49970e0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:07.066 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.061+0000 7f52c7fb5f40 1 --2- 172.21.15.167:0/1075244406 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ffb48c6fc0 0x55ffb499fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:07.066 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.061+0000 7f52c7fb5f40 1 -- 172.21.15.167:0/1075244406 >> 172.21.15.167:0/1075244406 conn(0x55ffb48f4950 msgr2=0x55ffb48f4d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:07:07.066 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.061+0000 7f52c7fb5f40 1 -- 172.21.15.167:0/1075244406 shutdown_connections 2024-05-11T08:07:07.066 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.061+0000 7f52c7fb5f40 1 -- 172.21.15.167:0/1075244406 wait complete. 2024-05-11T08:07:07.066 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.061+0000 7f52c7fb5f40 1 Processor -- start 2024-05-11T08:07:07.066 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.061+0000 7f52c7fb5f40 1 -- start start 2024-05-11T08:07:07.066 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.062+0000 7f52c7fb5f40 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ffb48c6fc0 0x55ffb48f3860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:07.066 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.062+0000 7f52c7fb5f40 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55ffb48c9d30 0x55ffb4a39b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:07.066 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.062+0000 7f52c7fb5f40 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ffb499f760 0x55ffb4a3a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:07.066 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.062+0000 7f52c7fb5f40 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55ffb499e190 con 0x55ffb499f760 2024-05-11T08:07:07.066 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.062+0000 7f52c7fb5f40 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55ffb48f62a0 con 0x55ffb48c6fc0 2024-05-11T08:07:07.067 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.062+0000 7f52c7fb5f40 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x55ffb48ca1b0 con 0x55ffb48c9d30 2024-05-11T08:07:07.067 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.062+0000 7f52c62b0640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ffb499f760 0x55ffb4a3a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:07.067 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.062+0000 7f52c62b0640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ffb499f760 0x55ffb4a3a050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:55720/0 (socket says 172.21.15.167:55720) 2024-05-11T08:07:07.067 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.062+0000 7f52c62b0640 1 -- 172.21.15.167:0/3317432451 learned_addr learned my addr 172.21.15.167:0/3317432451 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:07:07.067 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.062+0000 7f52c5aaf640 1 --2- 172.21.15.167:0/3317432451 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ffb48c6fc0 0x55ffb48f3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:07.067 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.062+0000 7f52bd2ae640 1 --2- 172.21.15.167:0/3317432451 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55ffb48c9d30 0x55ffb4a39b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:07.067 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.062+0000 7f52c62b0640 1 -- 172.21.15.167:0/3317432451 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55ffb48c9d30 msgr2=0x55ffb4a39b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:07.067 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.062+0000 7f52c62b0640 1 --2- 172.21.15.167:0/3317432451 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55ffb48c9d30 0x55ffb4a39b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:07.067 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.062+0000 7f52c62b0640 1 -- 172.21.15.167:0/3317432451 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ffb48c6fc0 msgr2=0x55ffb48f3860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:07.067 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.062+0000 7f52c62b0640 1 --2- 172.21.15.167:0/3317432451 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ffb48c6fc0 0x55ffb48f3860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:07.067 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.062+0000 7f52c62b0640 1 -- 172.21.15.167:0/3317432451 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ffb49a0080 con 0x55ffb499f760 2024-05-11T08:07:07.067 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.062+0000 7f52bd2ae640 1 --2- 172.21.15.167:0/3317432451 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55ffb48c9d30 0x55ffb4a39b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T08:07:07.067 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.062+0000 7f52c5aaf640 1 --2- 172.21.15.167:0/3317432451 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ffb48c6fc0 0x55ffb48f3860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T08:07:07.068 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.062+0000 7f52c62b0640 1 --2- 172.21.15.167:0/3317432451 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ffb499f760 0x55ffb4a3a050 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f52c000d3d0 tx=0x7f52c000d890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:07.068 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.063+0000 7f52bf7fe640 1 -- 172.21.15.167:0/3317432451 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52c0006020 con 0x55ffb499f760 2024-05-11T08:07:07.068 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.063+0000 7f52bf7fe640 1 -- 172.21.15.167:0/3317432451 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52c00061c0 con 0x55ffb499f760 2024-05-11T08:07:07.068 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.063+0000 7f52c7fb5f40 1 -- 172.21.15.167:0/3317432451 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55ffb49911d0 con 0x55ffb499f760 2024-05-11T08:07:07.068 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.063+0000 7f52bf7fe640 1 -- 172.21.15.167:0/3317432451 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52c00125f0 con 0x55ffb499f760 2024-05-11T08:07:07.068 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.063+0000 7f52c7fb5f40 1 -- 172.21.15.167:0/3317432451 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ffb4a31570 con 0x55ffb499f760 2024-05-11T08:07:07.068 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.064+0000 7f52bf7fe640 1 -- 172.21.15.167:0/3317432451 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f52c001a030 con 0x55ffb499f760 2024-05-11T08:07:07.070 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.065+0000 7f52bf7fe640 1 --2- 172.21.15.167:0/3317432451 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f52a8074cb0 0x7f52a8077170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:07.070 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.065+0000 7f52bf7fe640 1 -- 172.21.15.167:0/3317432451 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 6496+0+0 (secure 0 0 0) 0x7f52c0095e00 con 0x55ffb499f760 2024-05-11T08:07:07.070 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.065+0000 7f52c5aaf640 1 --2- 172.21.15.167:0/3317432451 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f52a8074cb0 0x7f52a8077170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:07.070 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.065+0000 7f52c7fb5f40 1 --2- 172.21.15.167:0/3317432451 >> [v2:172.21.15.167:6824/453689293,v1:172.21.15.167:6825/453689293] conn(0x55ffb48f3da0 0x55ffb4994710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:07.070 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.065+0000 7f52c7fb5f40 1 -- 172.21.15.167:0/3317432451 --> [v2:172.21.15.167:6824/453689293,v1:172.21.15.167:6825/453689293] -- osd_op(unknown.0.0:1 6.13 6:ca8fdbba:::100000001fb.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44) -- 0x55ffb490d8c0 con 0x55ffb48f3da0 2024-05-11T08:07:07.070 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.065+0000 7f52bd2ae640 1 --2- 172.21.15.167:0/3317432451 >> [v2:172.21.15.167:6824/453689293,v1:172.21.15.167:6825/453689293] conn(0x55ffb48f3da0 0x55ffb4994710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:07.070 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.065+0000 7f52c5aaf640 1 --2- 172.21.15.167:0/3317432451 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f52a8074cb0 0x7f52a8077170 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f52b8009680 tx=0x7f52b800e040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:07.070 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.065+0000 7f52bd2ae640 1 --2- 172.21.15.167:0/3317432451 >> [v2:172.21.15.167:6824/453689293,v1:172.21.15.167:6825/453689293] conn(0x55ffb48f3da0 0x55ffb4994710 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:07.071 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.066+0000 7f52bd2ae640 1 -- 172.21.15.167:0/3317432451 <== osd.3 v2:172.21.15.167:6824/453689293 1 ==== osd_op_reply(1 100000001fb.00000000 [getxattr (103) out=103b] v0'0 uv3 ondisk = 0) ==== 164+0+103 (crc 0 0 0) 0x7f52b0051070 con 0x55ffb48f3da0 2024-05-11T08:07:07.071 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.066+0000 7f52c7fb5f40 1 -- 172.21.15.167:0/3317432451 >> [v2:172.21.15.167:6824/453689293,v1:172.21.15.167:6825/453689293] conn(0x55ffb48f3da0 msgr2=0x55ffb4994710 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:07.071 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.066+0000 7f52c7fb5f40 1 --2- 172.21.15.167:0/3317432451 >> [v2:172.21.15.167:6824/453689293,v1:172.21.15.167:6825/453689293] conn(0x55ffb48f3da0 0x55ffb4994710 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:07.071 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.066+0000 7f52c7fb5f40 1 -- 172.21.15.167:0/3317432451 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f52a8074cb0 msgr2=0x7f52a8077170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:07.071 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.066+0000 7f52c7fb5f40 1 --2- 172.21.15.167:0/3317432451 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f52a8074cb0 0x7f52a8077170 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f52b8009680 tx=0x7f52b800e040 comp rx=0 tx=0).stop 2024-05-11T08:07:07.071 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.066+0000 7f52c7fb5f40 1 -- 172.21.15.167:0/3317432451 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ffb499f760 msgr2=0x55ffb4a3a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:07.071 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.066+0000 7f52c7fb5f40 1 --2- 172.21.15.167:0/3317432451 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ffb499f760 0x55ffb4a3a050 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f52c000d3d0 tx=0x7f52c000d890 comp rx=0 tx=0).stop 2024-05-11T08:07:07.071 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.066+0000 7f52c62b0640 1 -- 172.21.15.167:0/3317432451 reap_dead start 2024-05-11T08:07:07.071 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.066+0000 7f52c7fb5f40 1 -- 172.21.15.167:0/3317432451 shutdown_connections 2024-05-11T08:07:07.071 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.066+0000 7f52c7fb5f40 1 -- 172.21.15.167:0/3317432451 >> 172.21.15.167:0/3317432451 conn(0x55ffb48f4950 msgr2=0x55ffb48f4d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:07:07.071 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.066+0000 7f52c7fb5f40 1 -- 172.21.15.167:0/3317432451 shutdown_connections 2024-05-11T08:07:07.071 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.066+0000 7f52c7fb5f40 1 -- 172.21.15.167:0/3317432451 wait complete. 2024-05-11T08:07:07.072 INFO:teuthology.orchestra.run.smithi167.stdout:a��alphaparent_b 2024-05-11T08:07:07.072 DEBUG:teuthology.orchestra.run.smithi167:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2024-05-11T08:07:07.119 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-05-11T08:07:07.119 INFO:teuthology.orchestra.run.smithi167.stdout: "ino": 1099511628283, 2024-05-11T08:07:07.119 INFO:teuthology.orchestra.run.smithi167.stdout: "ancestors": [ 2024-05-11T08:07:07.119 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T08:07:07.119 INFO:teuthology.orchestra.run.smithi167.stdout: "dirino": 1099511628284, 2024-05-11T08:07:07.119 INFO:teuthology.orchestra.run.smithi167.stdout: "dname": "alpha", 2024-05-11T08:07:07.119 INFO:teuthology.orchestra.run.smithi167.stdout: "version": 7 2024-05-11T08:07:07.119 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T08:07:07.120 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T08:07:07.120 INFO:teuthology.orchestra.run.smithi167.stdout: "dirino": 1, 2024-05-11T08:07:07.120 INFO:teuthology.orchestra.run.smithi167.stdout: "dname": "parent_b", 2024-05-11T08:07:07.120 INFO:teuthology.orchestra.run.smithi167.stdout: "version": 30 2024-05-11T08:07:07.120 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T08:07:07.120 INFO:teuthology.orchestra.run.smithi167.stdout: ], 2024-05-11T08:07:07.120 INFO:teuthology.orchestra.run.smithi167.stdout: "pool": 7, 2024-05-11T08:07:07.120 INFO:teuthology.orchestra.run.smithi167.stdout: "old_pools": [ 2024-05-11T08:07:07.120 INFO:teuthology.orchestra.run.smithi167.stdout: 6 2024-05-11T08:07:07.120 INFO:teuthology.orchestra.run.smithi167.stdout: ] 2024-05-11T08:07:07.120 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-05-11T08:07:07.121 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T08:07:07.122 DEBUG:teuthology.orchestra.run.smithi167:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool data_new getxattr 100000001fb.00000000 parent 2024-05-11T08:07:07.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.206+0000 7fee97716f40 1 Processor -- start 2024-05-11T08:07:07.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.206+0000 7fee97716f40 1 -- start start 2024-05-11T08:07:07.210 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.206+0000 7fee97716f40 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x558842e18480 0x558842eb3c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:07.210 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.206+0000 7fee97716f40 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558842e30130 0x558842de71f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:07.210 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.206+0000 7fee97716f40 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558842ebd510 0x558842de7730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:07.210 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.206+0000 7fee97716f40 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x558842d90f50 con 0x558842e30130 2024-05-11T08:07:07.210 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.206+0000 7fee97716f40 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x558842d94f90 con 0x558842ebd510 2024-05-11T08:07:07.210 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.206+0000 7fee97716f40 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x558842de7c70 con 0x558842e18480 2024-05-11T08:07:07.210 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.206+0000 7fee94a0f640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558842e30130 0x558842de71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:07.210 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.206+0000 7fee94a0f640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558842e30130 0x558842de71f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:55744/0 (socket says 172.21.15.167:55744) 2024-05-11T08:07:07.210 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.206+0000 7fee94a0f640 1 -- 172.21.15.167:0/3169217693 learned_addr learned my addr 172.21.15.167:0/3169217693 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:07:07.210 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.207+0000 7fee95a11640 1 --2- 172.21.15.167:0/3169217693 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558842ebd510 0x558842de7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:07.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.207+0000 7fee95210640 1 --2- 172.21.15.167:0/3169217693 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x558842e18480 0x558842eb3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:07.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.207+0000 7fee94a0f640 1 -- 172.21.15.167:0/3169217693 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x558842e18480 msgr2=0x558842eb3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:07:07.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.207+0000 7fee94a0f640 1 --2- 172.21.15.167:0/3169217693 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x558842e18480 0x558842eb3c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:07.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.207+0000 7fee94a0f640 1 -- 172.21.15.167:0/3169217693 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558842ebd510 msgr2=0x558842de7730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:07:07.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.207+0000 7fee94a0f640 1 --2- 172.21.15.167:0/3169217693 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558842ebd510 0x558842de7730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:07.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.207+0000 7fee94a0f640 1 -- 172.21.15.167:0/3169217693 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558842de7ec0 con 0x558842e30130 2024-05-11T08:07:07.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.207+0000 7fee95210640 1 --2- 172.21.15.167:0/3169217693 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x558842e18480 0x558842eb3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T08:07:07.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.207+0000 7fee94a0f640 1 --2- 172.21.15.167:0/3169217693 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558842e30130 0x558842de71f0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fee8c00a5c0 tx=0x7fee8c00aa80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=868ebe73f0babf19 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:07.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.207+0000 7fee877fe640 1 -- 172.21.15.167:0/3169217693 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee8c0089f0 con 0x558842e30130 2024-05-11T08:07:07.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.207+0000 7fee877fe640 1 -- 172.21.15.167:0/3169217693 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee8c00f070 con 0x558842e30130 2024-05-11T08:07:07.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.207+0000 7fee877fe640 1 -- 172.21.15.167:0/3169217693 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee8c008df0 con 0x558842e30130 2024-05-11T08:07:07.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.207+0000 7fee97716f40 1 -- 172.21.15.167:0/3169217693 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558842e30130 msgr2=0x558842de71f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:07.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.207+0000 7fee97716f40 1 --2- 172.21.15.167:0/3169217693 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558842e30130 0x558842de71f0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fee8c00a5c0 tx=0x7fee8c00aa80 comp rx=0 tx=0).stop 2024-05-11T08:07:07.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.208+0000 7fee97716f40 1 -- 172.21.15.167:0/3169217693 shutdown_connections 2024-05-11T08:07:07.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.208+0000 7fee97716f40 1 --2- 172.21.15.167:0/3169217693 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558842ebd510 0x558842de7730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:07.212 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.208+0000 7fee97716f40 1 --2- 172.21.15.167:0/3169217693 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558842e30130 0x558842de71f0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:07.212 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.208+0000 7fee97716f40 1 --2- 172.21.15.167:0/3169217693 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x558842e18480 0x558842eb3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:07.212 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.208+0000 7fee97716f40 1 -- 172.21.15.167:0/3169217693 >> 172.21.15.167:0/3169217693 conn(0x558842de84f0 msgr2=0x558842ebd8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:07:07.212 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.208+0000 7fee97716f40 1 -- 172.21.15.167:0/3169217693 shutdown_connections 2024-05-11T08:07:07.212 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.208+0000 7fee97716f40 1 -- 172.21.15.167:0/3169217693 wait complete. 2024-05-11T08:07:07.212 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.208+0000 7fee97716f40 1 Processor -- start 2024-05-11T08:07:07.212 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.208+0000 7fee97716f40 1 -- start start 2024-05-11T08:07:07.212 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.209+0000 7fee97716f40 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558842e18480 0x558842e2d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:07.212 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.209+0000 7fee97716f40 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x558842e30130 0x558842ebe460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:07.212 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.209+0000 7fee97716f40 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558842ebd510 0x558842de72c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:07.212 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.209+0000 7fee97716f40 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x558842ebf080 con 0x558842ebd510 2024-05-11T08:07:07.212 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.209+0000 7fee97716f40 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x558842ebeda0 con 0x558842e18480 2024-05-11T08:07:07.212 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.209+0000 7fee97716f40 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x558842eb41c0 con 0x558842e30130 2024-05-11T08:07:07.212 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.209+0000 7fee95a11640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558842ebd510 0x558842de72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:07.212 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.209+0000 7fee95a11640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558842ebd510 0x558842de72c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:55752/0 (socket says 172.21.15.167:55752) 2024-05-11T08:07:07.213 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.209+0000 7fee95a11640 1 -- 172.21.15.167:0/1484738136 learned_addr learned my addr 172.21.15.167:0/1484738136 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:07:07.213 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.209+0000 7fee95210640 1 --2- 172.21.15.167:0/1484738136 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558842e18480 0x558842e2d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:07.213 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.209+0000 7fee94a0f640 1 --2- 172.21.15.167:0/1484738136 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x558842e30130 0x558842ebe460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:07.213 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.209+0000 7fee95a11640 1 -- 172.21.15.167:0/1484738136 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x558842e30130 msgr2=0x558842ebe460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:07.213 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.209+0000 7fee95a11640 1 --2- 172.21.15.167:0/1484738136 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x558842e30130 0x558842ebe460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:07.213 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.209+0000 7fee95a11640 1 -- 172.21.15.167:0/1484738136 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558842e18480 msgr2=0x558842e2d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:07.213 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.209+0000 7fee95a11640 1 --2- 172.21.15.167:0/1484738136 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558842e18480 0x558842e2d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:07.213 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.209+0000 7fee95a11640 1 -- 172.21.15.167:0/1484738136 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558842eb0910 con 0x558842ebd510 2024-05-11T08:07:07.213 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.209+0000 7fee95a11640 1 --2- 172.21.15.167:0/1484738136 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558842ebd510 0x558842de72c0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fee90025ae0 tx=0x7fee90003330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:07.213 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.210+0000 7fee867fc640 1 -- 172.21.15.167:0/1484738136 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee90069070 con 0x558842ebd510 2024-05-11T08:07:07.213 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.210+0000 7fee867fc640 1 -- 172.21.15.167:0/1484738136 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee900643f0 con 0x558842ebd510 2024-05-11T08:07:07.213 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.210+0000 7fee97716f40 1 -- 172.21.15.167:0/1484738136 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x558842eb69b0 con 0x558842ebd510 2024-05-11T08:07:07.213 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.210+0000 7fee867fc640 1 -- 172.21.15.167:0/1484738136 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee90058600 con 0x558842ebd510 2024-05-11T08:07:07.213 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.210+0000 7fee97716f40 1 -- 172.21.15.167:0/1484738136 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558842eb03f0 con 0x558842ebd510 2024-05-11T08:07:07.215 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.211+0000 7fee867fc640 1 -- 172.21.15.167:0/1484738136 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fee9006c070 con 0x558842ebd510 2024-05-11T08:07:07.216 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.212+0000 7fee867fc640 1 --2- 172.21.15.167:0/1484738136 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fee78074cb0 0x7fee78077170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:07.216 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.212+0000 7fee867fc640 1 -- 172.21.15.167:0/1484738136 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 6496+0+0 (secure 0 0 0) 0x7fee900da040 con 0x558842ebd510 2024-05-11T08:07:07.216 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.212+0000 7fee95210640 1 --2- 172.21.15.167:0/1484738136 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fee78074cb0 0x7fee78077170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:07.216 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.212+0000 7fee97716f40 1 --2- 172.21.15.167:0/1484738136 >> [v2:172.21.15.167:6804/3863307356,v1:172.21.15.167:6806/3863307356] conn(0x558842de7800 0x558842f5a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:07.216 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.212+0000 7fee97716f40 1 -- 172.21.15.167:0/1484738136 --> [v2:172.21.15.167:6804/3863307356,v1:172.21.15.167:6806/3863307356] -- osd_op(unknown.0.0:1 7.13 7:ca8fdbba:::100000001fb.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44) -- 0x558842eb7980 con 0x558842de7800 2024-05-11T08:07:07.216 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.212+0000 7fee94a0f640 1 --2- 172.21.15.167:0/1484738136 >> [v2:172.21.15.167:6804/3863307356,v1:172.21.15.167:6806/3863307356] conn(0x558842de7800 0x558842f5a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:07.216 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.212+0000 7fee95210640 1 --2- 172.21.15.167:0/1484738136 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fee78074cb0 0x7fee78077170 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fee88000c00 tx=0x7fee880017d0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:07.216 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.212+0000 7fee94a0f640 1 --2- 172.21.15.167:0/1484738136 >> [v2:172.21.15.167:6804/3863307356,v1:172.21.15.167:6806/3863307356] conn(0x558842de7800 0x558842f5a960 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:07.216 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.213+0000 7fee94a0f640 1 -- 172.21.15.167:0/1484738136 <== osd.0 v2:172.21.15.167:6804/3863307356 1 ==== osd_op_reply(1 100000001fb.00000000 [getxattr (103) out=103b] v0'0 uv2 ondisk = 0) ==== 164+0+103 (crc 0 0 0) 0x7fee8c008870 con 0x558842de7800 2024-05-11T08:07:07.216 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.213+0000 7fee97716f40 1 -- 172.21.15.167:0/1484738136 >> [v2:172.21.15.167:6804/3863307356,v1:172.21.15.167:6806/3863307356] conn(0x558842de7800 msgr2=0x558842f5a960 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:07.216 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.213+0000 7fee97716f40 1 --2- 172.21.15.167:0/1484738136 >> [v2:172.21.15.167:6804/3863307356,v1:172.21.15.167:6806/3863307356] conn(0x558842de7800 0x558842f5a960 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:07.216 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.213+0000 7fee97716f40 1 -- 172.21.15.167:0/1484738136 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fee78074cb0 msgr2=0x7fee78077170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:07.216 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.213+0000 7fee97716f40 1 --2- 172.21.15.167:0/1484738136 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fee78074cb0 0x7fee78077170 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fee88000c00 tx=0x7fee880017d0 comp rx=0 tx=0).stop 2024-05-11T08:07:07.216 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.213+0000 7fee97716f40 1 -- 172.21.15.167:0/1484738136 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558842ebd510 msgr2=0x558842de72c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:07.217 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.213+0000 7fee97716f40 1 --2- 172.21.15.167:0/1484738136 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558842ebd510 0x558842de72c0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fee90025ae0 tx=0x7fee90003330 comp rx=0 tx=0).stop 2024-05-11T08:07:07.217 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.213+0000 7fee95a11640 1 -- 172.21.15.167:0/1484738136 reap_dead start 2024-05-11T08:07:07.217 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.213+0000 7fee97716f40 1 -- 172.21.15.167:0/1484738136 shutdown_connections 2024-05-11T08:07:07.217 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.213+0000 7fee97716f40 1 -- 172.21.15.167:0/1484738136 >> 172.21.15.167:0/1484738136 conn(0x558842de84f0 msgr2=0x558842ebd8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:07:07.217 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.213+0000 7fee97716f40 1 -- 172.21.15.167:0/1484738136 shutdown_connections 2024-05-11T08:07:07.217 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.213+0000 7fee97716f40 1 -- 172.21.15.167:0/1484738136 wait complete. 2024-05-11T08:07:07.218 INFO:teuthology.orchestra.run.smithi167.stdout:a��alphaparent_c% 2024-05-11T08:07:07.218 DEBUG:teuthology.orchestra.run.smithi167:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2024-05-11T08:07:07.265 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-05-11T08:07:07.265 INFO:teuthology.orchestra.run.smithi167.stdout: "ino": 1099511628283, 2024-05-11T08:07:07.265 INFO:teuthology.orchestra.run.smithi167.stdout: "ancestors": [ 2024-05-11T08:07:07.265 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T08:07:07.265 INFO:teuthology.orchestra.run.smithi167.stdout: "dirino": 1099511628285, 2024-05-11T08:07:07.265 INFO:teuthology.orchestra.run.smithi167.stdout: "dname": "alpha", 2024-05-11T08:07:07.265 INFO:teuthology.orchestra.run.smithi167.stdout: "version": 8 2024-05-11T08:07:07.266 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-05-11T08:07:07.266 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-05-11T08:07:07.266 INFO:teuthology.orchestra.run.smithi167.stdout: "dirino": 1, 2024-05-11T08:07:07.266 INFO:teuthology.orchestra.run.smithi167.stdout: "dname": "parent_c", 2024-05-11T08:07:07.266 INFO:teuthology.orchestra.run.smithi167.stdout: "version": 37 2024-05-11T08:07:07.266 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-05-11T08:07:07.266 INFO:teuthology.orchestra.run.smithi167.stdout: ], 2024-05-11T08:07:07.266 INFO:teuthology.orchestra.run.smithi167.stdout: "pool": 7, 2024-05-11T08:07:07.266 INFO:teuthology.orchestra.run.smithi167.stdout: "old_pools": [ 2024-05-11T08:07:07.266 INFO:teuthology.orchestra.run.smithi167.stdout: 6 2024-05-11T08:07:07.266 INFO:teuthology.orchestra.run.smithi167.stdout: ] 2024-05-11T08:07:07.266 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-05-11T08:07:07.266 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T08:07:07.267 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T08:07:07.267 DEBUG:teuthology.orchestra.run.smithi167:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.file.layout.object_size -v 8388608 ./parent_c/alpha) 2024-05-11T08:07:07.328 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-11T08:07:07.469 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.469+0000 7f672ba99640 1 Processor -- start 2024-05-11T08:07:07.469 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.469+0000 7f672ba99640 1 -- start start 2024-05-11T08:07:07.470 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.470+0000 7f672ba99640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f672413eeb0 0x7f672406f480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:07.471 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.470+0000 7f672ba99640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f67240b50a0 0x7f6724073cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:07.471 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.470+0000 7f672ba99640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f672415fb90 con 0x7f672413f290 2024-05-11T08:07:07.471 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.470+0000 7f672ba99640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f672415d5b0 con 0x7f67240b50a0 2024-05-11T08:07:07.471 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.470+0000 7f672ba99640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f672415fe70 con 0x7f672413eeb0 2024-05-11T08:07:07.471 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.470+0000 7f672a036640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f67240b50a0 0x7f6724073cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:07.471 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.470+0000 7f672a036640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f67240b50a0 0x7f6724073cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.167:50428/0 (socket says 172.21.15.167:50428) 2024-05-11T08:07:07.471 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.470+0000 7f6729835640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f672413eeb0 0x7f672406f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:07.471 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.470+0000 7f672a036640 1 -- 172.21.15.167:0/1700794599 learned_addr learned my addr 172.21.15.167:0/1700794599 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:07:07.471 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.470+0000 7f6729034640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f672413f290 0x7f67240735c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.167:41972/0 (socket says 172.21.15.167:41972) 2024-05-11T08:07:07.471 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.470+0000 7f6729835640 1 -- 172.21.15.167:0/1700794599 >> v2:172.21.15.204:3300/0 conn(0x7f67240b50a0 msgr2=0x7f6724073cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:07:07.471 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.470+0000 7f6729835640 1 --2- 172.21.15.167:0/1700794599 >> v2:172.21.15.204:3300/0 conn(0x7f67240b50a0 0x7f6724073cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:07.471 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.470+0000 7f6729835640 1 -- 172.21.15.167:0/1700794599 >> v1:172.21.15.204:6789/0 conn(0x7f672413f290 legacy=0x7f67240735c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:07.472 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.470+0000 7f6729835640 1 -- 172.21.15.167:0/1700794599 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6724160150 con 0x7f672413eeb0 2024-05-11T08:07:07.472 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.470+0000 7f672a036640 1 --2- 172.21.15.167:0/1700794599 >> v2:172.21.15.204:3300/0 conn(0x7f67240b50a0 0x7f6724073cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T08:07:07.473 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.471+0000 7f6729835640 1 --2- 172.21.15.167:0/1700794599 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f672413eeb0 0x7f672406f480 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f67200097a0 tx=0x7f672002a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d14bfe94e3413d3f server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:07.473 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.471+0000 7f6728833640 1 -- 172.21.15.167:0/1700794599 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6720037070 con 0x7f672413eeb0 2024-05-11T08:07:07.473 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.471+0000 7f6728833640 1 -- 172.21.15.167:0/1700794599 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f672002ae50 con 0x7f672413eeb0 2024-05-11T08:07:07.473 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.471+0000 7f6728833640 1 -- 172.21.15.167:0/1700794599 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6720032660 con 0x7f672413eeb0 2024-05-11T08:07:07.473 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.472+0000 7f672ba99640 1 -- 172.21.15.167:0/1700794599 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f672413eeb0 msgr2=0x7f672406f480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:07.474 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.472+0000 7f672ba99640 1 --2- 172.21.15.167:0/1700794599 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f672413eeb0 0x7f672406f480 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f67200097a0 tx=0x7f672002a100 comp rx=0 tx=0).stop 2024-05-11T08:07:07.474 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.472+0000 7f672ba99640 1 -- 172.21.15.167:0/1700794599 shutdown_connections 2024-05-11T08:07:07.474 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.472+0000 7f672ba99640 1 --2- 172.21.15.167:0/1700794599 >> v2:172.21.15.204:3300/0 conn(0x7f67240b50a0 0x7f6724073cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:07.474 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.472+0000 7f672ba99640 1 --2- 172.21.15.167:0/1700794599 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f672413eeb0 0x7f672406f480 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:07.474 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.472+0000 7f672ba99640 1 -- 172.21.15.167:0/1700794599 >> 172.21.15.167:0/1700794599 conn(0x7f67240cd8f0 msgr2=0x7f67240cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:07:07.474 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.472+0000 7f672ba99640 1 -- 172.21.15.167:0/1700794599 shutdown_connections 2024-05-11T08:07:07.474 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.472+0000 7f672ba99640 1 -- 172.21.15.167:0/1700794599 wait complete. 2024-05-11T08:07:07.474 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.473+0000 7f672ba99640 1 Processor -- start 2024-05-11T08:07:07.474 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.473+0000 7f672ba99640 1 -- start start 2024-05-11T08:07:07.477 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.474+0000 7f672ba99640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f67240b50a0 0x7f67241f8370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:07.477 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.474+0000 7f672ba99640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f672413eeb0 0x7f6724153ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:07.477 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.474+0000 7f672ba99640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f672413f290 0x7f672413d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:07.477 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.474+0000 7f672ba99640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f672406f9c0 con 0x7f672413f290 2024-05-11T08:07:07.477 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.474+0000 7f672ba99640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6724160710 con 0x7f67240b50a0 2024-05-11T08:07:07.477 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.474+0000 7f672ba99640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f672406fb40 con 0x7f672413eeb0 2024-05-11T08:07:07.477 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.474+0000 7f672a036640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f672413f290 0x7f672413d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:07.477 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.474+0000 7f672a036640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f672413f290 0x7f672413d5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:55758/0 (socket says 172.21.15.167:55758) 2024-05-11T08:07:07.477 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.474+0000 7f672a036640 1 -- 172.21.15.167:0/974435093 learned_addr learned my addr 172.21.15.167:0/974435093 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:07:07.477 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.474+0000 7f6729835640 1 --2- 172.21.15.167:0/974435093 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f67240b50a0 0x7f67241f8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:07.477 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.474+0000 7f6729034640 1 --2- 172.21.15.167:0/974435093 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f672413eeb0 0x7f6724153ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:07.477 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.474+0000 7f672a036640 1 -- 172.21.15.167:0/974435093 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f672413eeb0 msgr2=0x7f6724153ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:07.477 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.474+0000 7f672a036640 1 --2- 172.21.15.167:0/974435093 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f672413eeb0 0x7f6724153ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:07.477 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.474+0000 7f672a036640 1 -- 172.21.15.167:0/974435093 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f67240b50a0 msgr2=0x7f67241f8370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:07.478 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.474+0000 7f672a036640 1 --2- 172.21.15.167:0/974435093 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f67240b50a0 0x7f67241f8370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:07.478 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.474+0000 7f672a036640 1 -- 172.21.15.167:0/974435093 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f672414c0a0 con 0x7f672413f290 2024-05-11T08:07:07.478 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.474+0000 7f672a036640 1 --2- 172.21.15.167:0/974435093 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f672413f290 0x7f672413d5c0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f671c00e340 tx=0x7f671c00e800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:07.478 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.475+0000 7f671a7fc640 1 -- 172.21.15.167:0/974435093 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f671c01a070 con 0x7f672413f290 2024-05-11T08:07:07.478 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.475+0000 7f671a7fc640 1 -- 172.21.15.167:0/974435093 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f671c00b0d0 con 0x7f672413f290 2024-05-11T08:07:07.478 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.475+0000 7f672ba99640 1 -- 172.21.15.167:0/974435093 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67241f8950 con 0x7f672413f290 2024-05-11T08:07:07.478 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.475+0000 7f671a7fc640 1 -- 172.21.15.167:0/974435093 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f671c00b480 con 0x7f672413f290 2024-05-11T08:07:07.478 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.475+0000 7f672ba99640 1 -- 172.21.15.167:0/974435093 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f672413e340 con 0x7f672413f290 2024-05-11T08:07:07.478 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.476+0000 7f672ba99640 1 -- 172.21.15.167:0/974435093 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6724069590 con 0x7f672413f290 2024-05-11T08:07:07.478 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.476+0000 7f671a7fc640 1 -- 172.21.15.167:0/974435093 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f671c005d10 con 0x7f672413f290 2024-05-11T08:07:07.478 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.477+0000 7f671a7fc640 1 --2- 172.21.15.167:0/974435093 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f6708076300 0x7f67080787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:07.478 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.477+0000 7f671a7fc640 1 -- 172.21.15.167:0/974435093 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 6496+0+0 (secure 0 0 0) 0x7f671c097a10 con 0x7f672413f290 2024-05-11T08:07:07.478 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.477+0000 7f6729835640 1 --2- 172.21.15.167:0/974435093 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f6708076300 0x7f67080787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:07.479 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.477+0000 7f6729835640 1 --2- 172.21.15.167:0/974435093 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f6708076300 0x7f67080787c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f6720034000 tx=0x7f672002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:07.481 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.481+0000 7f671a7fc640 1 -- 172.21.15.167:0/974435093 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f671c060a20 con 0x7f672413f290 2024-05-11T08:07:07.698 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.698+0000 7f672ba99640 1 -- 172.21.15.167:0/974435093 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f67241f8ce0 con 0x7f672413f290 2024-05-11T08:07:07.698 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.698+0000 7f671a7fc640 1 -- 172.21.15.167:0/974435093 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5192 (secure 0 0 0) 0x7f671c064d20 con 0x7f672413f290 2024-05-11T08:07:07.699 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T08:07:07.699 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4604,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2674983673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":2674983673},{"type":"v1","addr":"172.21.15.167:6837","nonce":2674983673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4607,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6834/1018548066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1018548066},{"type":"v1","addr":"172.21.15.204:6834","nonce":1018548066}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4729,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3369722249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3369722249},{"type":"v1","addr":"172.21.15.167:6836","nonce":3369722249}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-11T08:06:47.943939+0000","modified":"2024-05-11T08:07:04.006894+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4750},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4750":{"gid":4750,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.204:6835/469864006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6833","nonce":469864006},{"type":"v1","addr":"172.21.15.204:6835","nonce":469864006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6,7],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4750,"qdb_cluster":[4750]},"id":2}]} 2024-05-11T08:07:07.699 INFO:teuthology.orchestra.run.smithi167.stderr:dumped fsmap epoch 17 2024-05-11T08:07:07.701 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.700+0000 7f672ba99640 1 -- 172.21.15.167:0/974435093 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f6708076300 msgr2=0x7f67080787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:07.701 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.700+0000 7f672ba99640 1 --2- 172.21.15.167:0/974435093 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f6708076300 0x7f67080787c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f6720034000 tx=0x7f672002e000 comp rx=0 tx=0).stop 2024-05-11T08:07:07.701 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.700+0000 7f672ba99640 1 -- 172.21.15.167:0/974435093 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f672413f290 msgr2=0x7f672413d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:07.701 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.700+0000 7f672ba99640 1 --2- 172.21.15.167:0/974435093 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f672413f290 0x7f672413d5c0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f671c00e340 tx=0x7f671c00e800 comp rx=0 tx=0).stop 2024-05-11T08:07:07.701 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.700+0000 7f672ba99640 1 -- 172.21.15.167:0/974435093 shutdown_connections 2024-05-11T08:07:07.702 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.700+0000 7f672ba99640 1 --2- 172.21.15.167:0/974435093 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f6708076300 0x7f67080787c0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:07.702 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.700+0000 7f672ba99640 1 --2- 172.21.15.167:0/974435093 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f672413f290 0x7f672413d5c0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:07.702 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.700+0000 7f672ba99640 1 --2- 172.21.15.167:0/974435093 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f672413eeb0 0x7f6724153ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:07.702 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.700+0000 7f672ba99640 1 --2- 172.21.15.167:0/974435093 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f67240b50a0 0x7f67241f8370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:07.702 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.700+0000 7f672ba99640 1 -- 172.21.15.167:0/974435093 >> 172.21.15.167:0/974435093 conn(0x7f67240cd8f0 msgr2=0x7f672415d7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:07:07.702 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.701+0000 7f672ba99640 1 -- 172.21.15.167:0/974435093 shutdown_connections 2024-05-11T08:07:07.702 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.701+0000 7f672ba99640 1 -- 172.21.15.167:0/974435093 wait complete. 2024-05-11T08:07:07.712 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json flush journal 2024-05-11T08:07:07.836 INFO:teuthology.orchestra.run.smithi204.stdout:{"message":"","return_code":0} 2024-05-11T08:07:07.836 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-05-11T08:07:07.836 DEBUG:teuthology.orchestra.run.smithi167:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool data_new getxattr 100000001fb.00000000 layout 2024-05-11T08:07:07.880 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.880+0000 7f1a233edf40 1 Processor -- start 2024-05-11T08:07:07.880 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.880+0000 7f1a233edf40 1 -- start start 2024-05-11T08:07:07.885 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.881+0000 7f1a233edf40 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fea1cdf720 0x55fea1cc2140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:07.885 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.881+0000 7f1a233edf40 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fea1d6b4d0 0x55fea1d4d330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:07.885 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.881+0000 7f1a233edf40 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55fea1d6b8b0 0x55fea1e02570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:07.885 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.881+0000 7f1a233edf40 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55fea1cdfd20 con 0x55fea1cdf720 2024-05-11T08:07:07.885 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.881+0000 7f1a233edf40 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55fea1c3df50 con 0x55fea1d6b4d0 2024-05-11T08:07:07.885 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.881+0000 7f1a233edf40 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x55fea1d6be60 con 0x55fea1d6b8b0 2024-05-11T08:07:07.885 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.881+0000 7f1a20ee9640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fea1cdf720 0x55fea1cc2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:07.885 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.881+0000 7f1a20ee9640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fea1cdf720 0x55fea1cc2140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:55766/0 (socket says 172.21.15.167:55766) 2024-05-11T08:07:07.885 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.881+0000 7f1a20ee9640 1 -- 172.21.15.167:0/4060946290 learned_addr learned my addr 172.21.15.167:0/4060946290 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:07:07.885 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.881+0000 7f1a1bfff640 1 --2- 172.21.15.167:0/4060946290 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fea1d6b4d0 0x55fea1d4d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:07.885 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.881+0000 7f1a216ea640 1 --2- 172.21.15.167:0/4060946290 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55fea1d6b8b0 0x55fea1e02570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:07.885 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.881+0000 7f1a20ee9640 1 -- 172.21.15.167:0/4060946290 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55fea1d6b8b0 msgr2=0x55fea1e02570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:07:07.885 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.881+0000 7f1a20ee9640 1 --2- 172.21.15.167:0/4060946290 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55fea1d6b8b0 0x55fea1e02570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:07.886 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.881+0000 7f1a20ee9640 1 -- 172.21.15.167:0/4060946290 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fea1d6b4d0 msgr2=0x55fea1d4d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:07:07.886 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.881+0000 7f1a20ee9640 1 --2- 172.21.15.167:0/4060946290 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fea1d6b4d0 0x55fea1d4d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:07.886 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.881+0000 7f1a20ee9640 1 -- 172.21.15.167:0/4060946290 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fea1cc5e20 con 0x55fea1cdf720 2024-05-11T08:07:07.886 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.881+0000 7f1a216ea640 1 --2- 172.21.15.167:0/4060946290 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55fea1d6b8b0 0x55fea1e02570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T08:07:07.886 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.881+0000 7f1a20ee9640 1 --2- 172.21.15.167:0/4060946290 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fea1cdf720 0x55fea1cc2140 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f1a1400a7a0 tx=0x7f1a1400a890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd6d046a2ac96b3 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:07.886 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.881+0000 7f1a1affd640 1 -- 172.21.15.167:0/4060946290 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a14018070 con 0x55fea1cdf720 2024-05-11T08:07:07.886 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.881+0000 7f1a1affd640 1 -- 172.21.15.167:0/4060946290 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a14007600 con 0x55fea1cdf720 2024-05-11T08:07:07.886 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.881+0000 7f1a1affd640 1 -- 172.21.15.167:0/4060946290 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a140079b0 con 0x55fea1cdf720 2024-05-11T08:07:07.886 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.882+0000 7f1a233edf40 1 -- 172.21.15.167:0/4060946290 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fea1cdf720 msgr2=0x55fea1cc2140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:07.886 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.882+0000 7f1a233edf40 1 --2- 172.21.15.167:0/4060946290 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fea1cdf720 0x55fea1cc2140 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f1a1400a7a0 tx=0x7f1a1400a890 comp rx=0 tx=0).stop 2024-05-11T08:07:07.886 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.882+0000 7f1a233edf40 1 -- 172.21.15.167:0/4060946290 shutdown_connections 2024-05-11T08:07:07.886 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.882+0000 7f1a233edf40 1 --2- 172.21.15.167:0/4060946290 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55fea1d6b8b0 0x55fea1e02570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:07.886 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.882+0000 7f1a233edf40 1 --2- 172.21.15.167:0/4060946290 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fea1d6b4d0 0x55fea1d4d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:07.886 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.882+0000 7f1a233edf40 1 --2- 172.21.15.167:0/4060946290 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fea1cdf720 0x55fea1cc2140 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:07.887 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.882+0000 7f1a233edf40 1 -- 172.21.15.167:0/4060946290 >> 172.21.15.167:0/4060946290 conn(0x55fea1cc2950 msgr2=0x55fea1cc45d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:07:07.887 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.882+0000 7f1a233edf40 1 -- 172.21.15.167:0/4060946290 shutdown_connections 2024-05-11T08:07:07.887 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.882+0000 7f1a233edf40 1 -- 172.21.15.167:0/4060946290 wait complete. 2024-05-11T08:07:07.887 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.883+0000 7f1a233edf40 1 Processor -- start 2024-05-11T08:07:07.887 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.883+0000 7f1a233edf40 1 -- start start 2024-05-11T08:07:07.887 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.883+0000 7f1a233edf40 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fea1cdf720 0x55fea1d6e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:07.887 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.883+0000 7f1a233edf40 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fea1d6b4d0 0x55fea1d63860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:07.887 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.883+0000 7f1a233edf40 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55fea1d6b8b0 0x55fea1d4ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:07.887 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.883+0000 7f1a233edf40 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55fea1d6ea70 con 0x55fea1cdf720 2024-05-11T08:07:07.887 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.883+0000 7f1a233edf40 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55fea1cc5b40 con 0x55fea1d6b4d0 2024-05-11T08:07:07.887 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.883+0000 7f1a233edf40 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x55fea1cc2680 con 0x55fea1d6b8b0 2024-05-11T08:07:07.887 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.883+0000 7f1a20ee9640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fea1cdf720 0x55fea1d6e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:07.887 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.883+0000 7f1a20ee9640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fea1cdf720 0x55fea1d6e200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:55776/0 (socket says 172.21.15.167:55776) 2024-05-11T08:07:07.887 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.883+0000 7f1a20ee9640 1 -- 172.21.15.167:0/3303033891 learned_addr learned my addr 172.21.15.167:0/3303033891 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:07:07.887 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.883+0000 7f1a1bfff640 1 --2- 172.21.15.167:0/3303033891 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fea1d6b4d0 0x55fea1d63860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:07.888 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.883+0000 7f1a216ea640 1 --2- 172.21.15.167:0/3303033891 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55fea1d6b8b0 0x55fea1d4ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:07.888 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.884+0000 7f1a20ee9640 1 -- 172.21.15.167:0/3303033891 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55fea1d6b8b0 msgr2=0x55fea1d4ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:07.888 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.884+0000 7f1a20ee9640 1 --2- 172.21.15.167:0/3303033891 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55fea1d6b8b0 0x55fea1d4ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:07.888 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.884+0000 7f1a20ee9640 1 -- 172.21.15.167:0/3303033891 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fea1d6b4d0 msgr2=0x55fea1d63860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:07.888 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.884+0000 7f1a20ee9640 1 --2- 172.21.15.167:0/3303033891 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fea1d6b4d0 0x55fea1d63860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:07.888 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.884+0000 7f1a20ee9640 1 -- 172.21.15.167:0/3303033891 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fea1dfa0e0 con 0x55fea1cdf720 2024-05-11T08:07:07.888 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.884+0000 7f1a20ee9640 1 --2- 172.21.15.167:0/3303033891 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fea1cdf720 0x55fea1d6e200 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f1a14000c00 tx=0x7f1a14003020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:07.888 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.884+0000 7f1a19ffb640 1 -- 172.21.15.167:0/3303033891 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a14018070 con 0x55fea1cdf720 2024-05-11T08:07:07.888 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.884+0000 7f1a19ffb640 1 -- 172.21.15.167:0/3303033891 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a14019040 con 0x55fea1cdf720 2024-05-11T08:07:07.888 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.884+0000 7f1a233edf40 1 -- 172.21.15.167:0/3303033891 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55fea1d4c440 con 0x55fea1cdf720 2024-05-11T08:07:07.888 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.884+0000 7f1a19ffb640 1 -- 172.21.15.167:0/3303033891 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a14007d00 con 0x55fea1cdf720 2024-05-11T08:07:07.888 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.884+0000 7f1a233edf40 1 -- 172.21.15.167:0/3303033891 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fea1dfc870 con 0x55fea1cdf720 2024-05-11T08:07:07.890 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.885+0000 7f1a19ffb640 1 -- 172.21.15.167:0/3303033891 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f1a14013410 con 0x55fea1cdf720 2024-05-11T08:07:07.890 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.886+0000 7f1a19ffb640 1 --2- 172.21.15.167:0/3303033891 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f1a04074cb0 0x7f1a04077170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:07.890 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.886+0000 7f1a1bfff640 1 --2- 172.21.15.167:0/3303033891 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f1a04074cb0 0x7f1a04077170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:07.891 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.886+0000 7f1a19ffb640 1 -- 172.21.15.167:0/3303033891 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 6496+0+0 (secure 0 0 0) 0x7f1a14096bb0 con 0x55fea1cdf720 2024-05-11T08:07:07.891 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.886+0000 7f1a233edf40 1 --2- 172.21.15.167:0/3303033891 >> [v2:172.21.15.167:6804/3863307356,v1:172.21.15.167:6806/3863307356] conn(0x55fea1cc4a70 0x55fea1dfa870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:07.891 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.886+0000 7f1a233edf40 1 -- 172.21.15.167:0/3303033891 --> [v2:172.21.15.167:6804/3863307356,v1:172.21.15.167:6806/3863307356] -- osd_op(unknown.0.0:1 7.13 7:ca8fdbba:::100000001fb.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44) -- 0x55fea1d4bfb0 con 0x55fea1cc4a70 2024-05-11T08:07:07.891 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.886+0000 7f1a216ea640 1 --2- 172.21.15.167:0/3303033891 >> [v2:172.21.15.167:6804/3863307356,v1:172.21.15.167:6806/3863307356] conn(0x55fea1cc4a70 0x55fea1dfa870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:07.891 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.886+0000 7f1a1bfff640 1 --2- 172.21.15.167:0/3303033891 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f1a04074cb0 0x7f1a04077170 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x55fea1cc24b0 tx=0x7f1a0c001a70 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:07.891 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.887+0000 7f1a216ea640 1 --2- 172.21.15.167:0/3303033891 >> [v2:172.21.15.167:6804/3863307356,v1:172.21.15.167:6806/3863307356] conn(0x55fea1cc4a70 0x55fea1dfa870 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:07.891 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.887+0000 7f1a216ea640 1 -- 172.21.15.167:0/3303033891 <== osd.0 v2:172.21.15.167:6804/3863307356 1 ==== osd_op_reply(1 100000001fb.00000000 [getxattr (30) out=30b] v0'0 uv3 ondisk = 0) ==== 164+0+30 (crc 0 0 0) 0x7f1a1c053870 con 0x55fea1cc4a70 2024-05-11T08:07:07.891 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.887+0000 7f1a233edf40 1 -- 172.21.15.167:0/3303033891 >> [v2:172.21.15.167:6804/3863307356,v1:172.21.15.167:6806/3863307356] conn(0x55fea1cc4a70 msgr2=0x55fea1dfa870 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:07.891 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.887+0000 7f1a233edf40 1 --2- 172.21.15.167:0/3303033891 >> [v2:172.21.15.167:6804/3863307356,v1:172.21.15.167:6806/3863307356] conn(0x55fea1cc4a70 0x55fea1dfa870 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:07.891 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.887+0000 7f1a233edf40 1 -- 172.21.15.167:0/3303033891 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f1a04074cb0 msgr2=0x7f1a04077170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:07.891 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.887+0000 7f1a233edf40 1 --2- 172.21.15.167:0/3303033891 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f1a04074cb0 0x7f1a04077170 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x55fea1cc24b0 tx=0x7f1a0c001a70 comp rx=0 tx=0).stop 2024-05-11T08:07:07.891 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.887+0000 7f1a233edf40 1 -- 172.21.15.167:0/3303033891 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fea1cdf720 msgr2=0x55fea1d6e200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:07.891 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.887+0000 7f1a233edf40 1 --2- 172.21.15.167:0/3303033891 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fea1cdf720 0x55fea1d6e200 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f1a14000c00 tx=0x7f1a14003020 comp rx=0 tx=0).stop 2024-05-11T08:07:07.891 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.887+0000 7f1a216ea640 1 -- 172.21.15.167:0/3303033891 reap_dead start 2024-05-11T08:07:07.891 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.887+0000 7f1a233edf40 1 -- 172.21.15.167:0/3303033891 shutdown_connections 2024-05-11T08:07:07.892 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.887+0000 7f1a233edf40 1 -- 172.21.15.167:0/3303033891 >> 172.21.15.167:0/3303033891 conn(0x55fea1cc2950 msgr2=0x55fea1d5eef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:07:07.892 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.887+0000 7f1a233edf40 1 -- 172.21.15.167:0/3303033891 shutdown_connections 2024-05-11T08:07:07.892 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:07.887+0000 7f1a233edf40 1 -- 172.21.15.167:0/3303033891 wait complete. 2024-05-11T08:07:07.892 INFO:teuthology.orchestra.run.smithi167.stdout:@� 2024-05-11T08:07:07.893 DEBUG:teuthology.orchestra.run.smithi167:> ceph-dencoder type file_layout_t import - decode dump_json 2024-05-11T08:07:07.940 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-05-11T08:07:07.940 INFO:teuthology.orchestra.run.smithi167.stdout: "stripe_unit": 4194304, 2024-05-11T08:07:07.940 INFO:teuthology.orchestra.run.smithi167.stdout: "stripe_count": 1, 2024-05-11T08:07:07.940 INFO:teuthology.orchestra.run.smithi167.stdout: "object_size": 8388608, 2024-05-11T08:07:07.940 INFO:teuthology.orchestra.run.smithi167.stdout: "pool_id": 7, 2024-05-11T08:07:07.940 INFO:teuthology.orchestra.run.smithi167.stdout: "pool_ns": "" 2024-05-11T08:07:07.940 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-05-11T08:07:07.940 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T08:07:07.943 DEBUG:teuthology.orchestra.run.smithi167:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001fb.00000000 layout 2024-05-11T08:07:08.027 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.027+0000 7f44f100bf40 1 Processor -- start 2024-05-11T08:07:08.027 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.027+0000 7f44f100bf40 1 -- start start 2024-05-11T08:07:08.032 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.027+0000 7f44f100bf40 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb0fdec090 0x55fb0febc860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:08.032 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.027+0000 7f44f100bf40 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fb0fe18d60 0x55fb0feb72e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:08.032 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.027+0000 7f44f100bf40 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55fb0febf100 0x55fb0febd100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:08.032 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.027+0000 7f44f100bf40 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55fb0fd91f50 con 0x55fb0fe18d60 2024-05-11T08:07:08.032 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.027+0000 7f44f100bf40 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55fb0fd95f90 con 0x55fb0fdec090 2024-05-11T08:07:08.032 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.027+0000 7f44f100bf40 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x55fb0febfbf0 con 0x55fb0febf100 2024-05-11T08:07:08.032 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.028+0000 7f44e7fff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fb0fe18d60 0x55fb0feb72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:08.032 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.028+0000 7f44e7fff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fb0fe18d60 0x55fb0feb72e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:55792/0 (socket says 172.21.15.167:55792) 2024-05-11T08:07:08.032 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.028+0000 7f44e7fff640 1 -- 172.21.15.167:0/1919457227 learned_addr learned my addr 172.21.15.167:0/1919457227 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:07:08.032 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.028+0000 7f44ee62a640 1 --2- 172.21.15.167:0/1919457227 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55fb0febf100 0x55fb0febd100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:08.032 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.028+0000 7f44ede29640 1 --2- 172.21.15.167:0/1919457227 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb0fdec090 0x55fb0febc860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:08.033 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.028+0000 7f44e7fff640 1 -- 172.21.15.167:0/1919457227 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55fb0febf100 msgr2=0x55fb0febd100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:07:08.033 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.028+0000 7f44e7fff640 1 --2- 172.21.15.167:0/1919457227 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55fb0febf100 0x55fb0febd100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:08.033 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.028+0000 7f44e7fff640 1 -- 172.21.15.167:0/1919457227 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb0fdec090 msgr2=0x55fb0febc860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:07:08.033 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.028+0000 7f44e7fff640 1 --2- 172.21.15.167:0/1919457227 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb0fdec090 0x55fb0febc860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:08.033 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.028+0000 7f44e7fff640 1 -- 172.21.15.167:0/1919457227 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb0fec14c0 con 0x55fb0fe18d60 2024-05-11T08:07:08.033 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.028+0000 7f44e7fff640 1 --2- 172.21.15.167:0/1919457227 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fb0fe18d60 0x55fb0feb72e0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f44d800a4f0 tx=0x7f44d8051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=524b2a8fc08358a3 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:08.033 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.028+0000 7f44ece27640 1 -- 172.21.15.167:0/1919457227 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44d8063070 con 0x55fb0fe18d60 2024-05-11T08:07:08.033 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.028+0000 7f44ece27640 1 -- 172.21.15.167:0/1919457227 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44d8055b20 con 0x55fb0fe18d60 2024-05-11T08:07:08.033 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.028+0000 7f44ece27640 1 -- 172.21.15.167:0/1919457227 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44d8068460 con 0x55fb0fe18d60 2024-05-11T08:07:08.033 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.029+0000 7f44f100bf40 1 -- 172.21.15.167:0/1919457227 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fb0fe18d60 msgr2=0x55fb0feb72e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:08.033 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.029+0000 7f44f100bf40 1 --2- 172.21.15.167:0/1919457227 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fb0fe18d60 0x55fb0feb72e0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f44d800a4f0 tx=0x7f44d8051000 comp rx=0 tx=0).stop 2024-05-11T08:07:08.033 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.029+0000 7f44f100bf40 1 -- 172.21.15.167:0/1919457227 shutdown_connections 2024-05-11T08:07:08.033 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.029+0000 7f44f100bf40 1 --2- 172.21.15.167:0/1919457227 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55fb0febf100 0x55fb0febd100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:08.033 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.029+0000 7f44f100bf40 1 --2- 172.21.15.167:0/1919457227 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fb0fe18d60 0x55fb0feb72e0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:08.033 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.029+0000 7f44f100bf40 1 --2- 172.21.15.167:0/1919457227 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb0fdec090 0x55fb0febc860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:08.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.029+0000 7f44f100bf40 1 -- 172.21.15.167:0/1919457227 >> 172.21.15.167:0/1919457227 conn(0x55fb0fe16950 msgr2=0x55fb0febf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:07:08.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.029+0000 7f44f100bf40 1 -- 172.21.15.167:0/1919457227 shutdown_connections 2024-05-11T08:07:08.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.029+0000 7f44f100bf40 1 -- 172.21.15.167:0/1919457227 wait complete. 2024-05-11T08:07:08.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.029+0000 7f44f100bf40 1 Processor -- start 2024-05-11T08:07:08.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.029+0000 7f44f100bf40 1 -- start start 2024-05-11T08:07:08.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.030+0000 7f44f100bf40 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55fb0fdec090 0x55fb0feb94f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:08.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.030+0000 7f44f100bf40 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fb0fe18d60 0x55fb0febb390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:08.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.030+0000 7f44f100bf40 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb0febf100 0x55fb0feb10a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:08.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.030+0000 7f44f100bf40 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55fb0fe16380 con 0x55fb0fe18d60 2024-05-11T08:07:08.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.030+0000 7f44f100bf40 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55fb0feb81a0 con 0x55fb0febf100 2024-05-11T08:07:08.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.030+0000 7f44f100bf40 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x55fb0feb7d30 con 0x55fb0fdec090 2024-05-11T08:07:08.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.030+0000 7f44e7fff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fb0fe18d60 0x55fb0febb390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:08.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.030+0000 7f44e7fff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fb0fe18d60 0x55fb0febb390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:55808/0 (socket says 172.21.15.167:55808) 2024-05-11T08:07:08.035 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.030+0000 7f44e7fff640 1 -- 172.21.15.167:0/2695144521 learned_addr learned my addr 172.21.15.167:0/2695144521 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:07:08.035 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.030+0000 7f44ee62a640 1 --2- 172.21.15.167:0/2695144521 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb0febf100 0x55fb0feb10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:08.035 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.030+0000 7f44ede29640 1 --2- 172.21.15.167:0/2695144521 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55fb0fdec090 0x55fb0feb94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:08.035 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.030+0000 7f44e7fff640 1 -- 172.21.15.167:0/2695144521 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55fb0fdec090 msgr2=0x55fb0feb94f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:08.035 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.030+0000 7f44e7fff640 1 --2- 172.21.15.167:0/2695144521 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55fb0fdec090 0x55fb0feb94f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:08.035 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.030+0000 7f44e7fff640 1 -- 172.21.15.167:0/2695144521 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb0febf100 msgr2=0x55fb0feb10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:08.035 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.030+0000 7f44e7fff640 1 --2- 172.21.15.167:0/2695144521 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb0febf100 0x55fb0feb10a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:08.035 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.030+0000 7f44e7fff640 1 -- 172.21.15.167:0/2695144521 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb0febe470 con 0x55fb0fe18d60 2024-05-11T08:07:08.035 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.031+0000 7f44e7fff640 1 --2- 172.21.15.167:0/2695144521 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fb0fe18d60 0x55fb0febb390 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f44d800a4f0 tx=0x7f44d8002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:08.035 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.031+0000 7f44e6ffd640 1 -- 172.21.15.167:0/2695144521 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44d8077070 con 0x55fb0fe18d60 2024-05-11T08:07:08.035 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.031+0000 7f44e6ffd640 1 -- 172.21.15.167:0/2695144521 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44d8055cd0 con 0x55fb0fe18d60 2024-05-11T08:07:08.036 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.031+0000 7f44f100bf40 1 -- 172.21.15.167:0/2695144521 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55fb0febe700 con 0x55fb0fe18d60 2024-05-11T08:07:08.036 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.031+0000 7f44e6ffd640 1 -- 172.21.15.167:0/2695144521 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44d8063050 con 0x55fb0fe18d60 2024-05-11T08:07:08.036 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.031+0000 7f44f100bf40 1 -- 172.21.15.167:0/2695144521 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb0feaf690 con 0x55fb0fe18d60 2024-05-11T08:07:08.038 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.032+0000 7f44e6ffd640 1 -- 172.21.15.167:0/2695144521 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f44d807a070 con 0x55fb0fe18d60 2024-05-11T08:07:08.038 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.033+0000 7f44e6ffd640 1 --2- 172.21.15.167:0/2695144521 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f44d0074cb0 0x7f44d0077170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:08.038 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.033+0000 7f44e6ffd640 1 -- 172.21.15.167:0/2695144521 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 6496+0+0 (secure 0 0 0) 0x7f44d80dbdc0 con 0x55fb0fe18d60 2024-05-11T08:07:08.038 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.033+0000 7f44ede29640 1 --2- 172.21.15.167:0/2695144521 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f44d0074cb0 0x7f44d0077170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:08.038 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.033+0000 7f44f100bf40 1 --2- 172.21.15.167:0/2695144521 >> [v2:172.21.15.167:6824/453689293,v1:172.21.15.167:6825/453689293] conn(0x55fb0feb9a30 0x55fb0feae290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:08.038 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.033+0000 7f44f100bf40 1 -- 172.21.15.167:0/2695144521 --> [v2:172.21.15.167:6824/453689293,v1:172.21.15.167:6825/453689293] -- osd_op(unknown.0.0:1 6.13 6:ca8fdbba:::100000001fb.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44) -- 0x55fb0fdead50 con 0x55fb0feb9a30 2024-05-11T08:07:08.038 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.033+0000 7f44ee62a640 1 --2- 172.21.15.167:0/2695144521 >> [v2:172.21.15.167:6824/453689293,v1:172.21.15.167:6825/453689293] conn(0x55fb0feb9a30 0x55fb0feae290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:08.038 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.033+0000 7f44ede29640 1 --2- 172.21.15.167:0/2695144521 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f44d0074cb0 0x7f44d0077170 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x55fb0fec1ec0 tx=0x7f44e0001720 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:08.038 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.033+0000 7f44ee62a640 1 --2- 172.21.15.167:0/2695144521 >> [v2:172.21.15.167:6824/453689293,v1:172.21.15.167:6825/453689293] conn(0x55fb0feb9a30 0x55fb0feae290 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:08.038 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.034+0000 7f44ee62a640 1 -- 172.21.15.167:0/2695144521 <== osd.3 v2:172.21.15.167:6824/453689293 1 ==== osd_op_reply(1 100000001fb.00000000 [getxattr (30) out=30b] v0'0 uv3 ondisk = 0) ==== 164+0+30 (crc 0 0 0) 0x7f44e8006030 con 0x55fb0feb9a30 2024-05-11T08:07:08.038 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.034+0000 7f44f100bf40 1 -- 172.21.15.167:0/2695144521 >> [v2:172.21.15.167:6824/453689293,v1:172.21.15.167:6825/453689293] conn(0x55fb0feb9a30 msgr2=0x55fb0feae290 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:08.039 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.034+0000 7f44f100bf40 1 --2- 172.21.15.167:0/2695144521 >> [v2:172.21.15.167:6824/453689293,v1:172.21.15.167:6825/453689293] conn(0x55fb0feb9a30 0x55fb0feae290 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:08.039 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.034+0000 7f44f100bf40 1 -- 172.21.15.167:0/2695144521 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f44d0074cb0 msgr2=0x7f44d0077170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:08.039 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.034+0000 7f44f100bf40 1 --2- 172.21.15.167:0/2695144521 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f44d0074cb0 0x7f44d0077170 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x55fb0fec1ec0 tx=0x7f44e0001720 comp rx=0 tx=0).stop 2024-05-11T08:07:08.039 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.034+0000 7f44f100bf40 1 -- 172.21.15.167:0/2695144521 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fb0fe18d60 msgr2=0x55fb0febb390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:08.039 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.034+0000 7f44f100bf40 1 --2- 172.21.15.167:0/2695144521 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55fb0fe18d60 0x55fb0febb390 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f44d800a4f0 tx=0x7f44d8002fb0 comp rx=0 tx=0).stop 2024-05-11T08:07:08.039 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.034+0000 7f44ee62a640 1 -- 172.21.15.167:0/2695144521 reap_dead start 2024-05-11T08:07:08.039 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.034+0000 7f44f100bf40 1 -- 172.21.15.167:0/2695144521 shutdown_connections 2024-05-11T08:07:08.039 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.034+0000 7f44f100bf40 1 -- 172.21.15.167:0/2695144521 >> 172.21.15.167:0/2695144521 conn(0x55fb0fe16950 msgr2=0x55fb0fdeb5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:07:08.039 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.034+0000 7f44f100bf40 1 -- 172.21.15.167:0/2695144521 shutdown_connections 2024-05-11T08:07:08.039 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.034+0000 7f44f100bf40 1 -- 172.21.15.167:0/2695144521 wait complete. 2024-05-11T08:07:08.040 INFO:teuthology.orchestra.run.smithi167.stdout:@@ 2024-05-11T08:07:08.040 DEBUG:teuthology.orchestra.run.smithi167:> ceph-dencoder type file_layout_t import - decode dump_json 2024-05-11T08:07:08.087 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-05-11T08:07:08.087 INFO:teuthology.orchestra.run.smithi167.stdout: "stripe_unit": 4194304, 2024-05-11T08:07:08.088 INFO:teuthology.orchestra.run.smithi167.stdout: "stripe_count": 1, 2024-05-11T08:07:08.088 INFO:teuthology.orchestra.run.smithi167.stdout: "object_size": 4194304, 2024-05-11T08:07:08.088 INFO:teuthology.orchestra.run.smithi167.stdout: "pool_id": 6, 2024-05-11T08:07:08.088 INFO:teuthology.orchestra.run.smithi167.stdout: "pool_ns": "" 2024-05-11T08:07:08.088 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-05-11T08:07:08.088 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T08:07:08.090 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-11T08:07:08.090 DEBUG:teuthology.orchestra.run.smithi167:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-11T08:07:08.168 DEBUG:teuthology.orchestra.run.smithi204:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-11T08:07:08.207 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-11T08:07:08.207 INFO:tasks.cephfs.mount:Terminating background process 2024-05-11T08:07:08.207 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-05-11T08:07:08.207 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T08:07:08.225 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-05-11T08:07:08.225 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T08:07:08.281 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-11T08:07:08.422 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.422+0000 7f00c2645640 1 Processor -- start 2024-05-11T08:07:08.422 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.422+0000 7f00c2645640 1 -- start start 2024-05-11T08:07:08.424 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.423+0000 7f00c2645640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f00bc1598d0 0x7f00bc0d2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:08.424 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.423+0000 7f00c2645640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f00bc0cb6c0 0x7f00bc0d0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:08.424 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.423+0000 7f00c2645640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00bc0d1190 con 0x7f00bc0d0640 2024-05-11T08:07:08.424 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.423+0000 7f00c2645640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f00bc06d500 con 0x7f00bc0cb6c0 2024-05-11T08:07:08.424 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.423+0000 7f00c2645640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f00bc06d7e0 con 0x7f00bc1598d0 2024-05-11T08:07:08.424 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.423+0000 7f00c0be2640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f00bc0cb6c0 0x7f00bc0d0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:08.424 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.423+0000 7f00c0be2640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f00bc0cb6c0 0x7f00bc0d0aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:55820/0 (socket says 172.21.15.167:55820) 2024-05-11T08:07:08.424 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.423+0000 7f00c0be2640 1 -- 172.21.15.167:0/2152780636 learned_addr learned my addr 172.21.15.167:0/2152780636 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:07:08.425 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.423+0000 7f00bbfff640 1 --2- 172.21.15.167:0/2152780636 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f00bc1598d0 0x7f00bc0d2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:08.425 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.423+0000 7f00c0be2640 1 -- 172.21.15.167:0/2152780636 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f00bc1598d0 msgr2=0x7f00bc0d2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:07:08.425 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.423+0000 7f00c0be2640 1 --2- 172.21.15.167:0/2152780636 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f00bc1598d0 0x7f00bc0d2cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:08.425 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.423+0000 7f00c0be2640 1 -- 172.21.15.167:0/2152780636 >> v1:172.21.15.204:6789/0 conn(0x7f00bc0d0640 legacy=0x7f00bc0cafb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:08.425 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.423+0000 7f00c0be2640 1 -- 172.21.15.167:0/2152780636 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00bc06dac0 con 0x7f00bc0cb6c0 2024-05-11T08:07:08.425 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.424+0000 7f00c0be2640 1 --2- 172.21.15.167:0/2152780636 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f00bc0cb6c0 0x7f00bc0d0aa0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f00b4009e50 tx=0x7f00b402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dfe9f3077113c88 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:08.425 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.424+0000 7f00baffd640 1 -- 172.21.15.167:0/2152780636 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00b4036070 con 0x7f00bc0cb6c0 2024-05-11T08:07:08.425 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.424+0000 7f00baffd640 1 -- 172.21.15.167:0/2152780636 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00b4031600 con 0x7f00bc0cb6c0 2024-05-11T08:07:08.425 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.424+0000 7f00baffd640 1 -- 172.21.15.167:0/2152780636 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00b40319d0 con 0x7f00bc0cb6c0 2024-05-11T08:07:08.426 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.425+0000 7f00c2645640 1 -- 172.21.15.167:0/2152780636 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f00bc0cb6c0 msgr2=0x7f00bc0d0aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:08.426 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.425+0000 7f00c2645640 1 --2- 172.21.15.167:0/2152780636 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f00bc0cb6c0 0x7f00bc0d0aa0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f00b4009e50 tx=0x7f00b402a710 comp rx=0 tx=0).stop 2024-05-11T08:07:08.426 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.425+0000 7f00c2645640 1 -- 172.21.15.167:0/2152780636 shutdown_connections 2024-05-11T08:07:08.427 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.425+0000 7f00c2645640 1 --2- 172.21.15.167:0/2152780636 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f00bc0cb6c0 0x7f00bc0d0aa0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:08.427 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.425+0000 7f00c2645640 1 --2- 172.21.15.167:0/2152780636 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f00bc1598d0 0x7f00bc0d2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:08.427 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.425+0000 7f00c2645640 1 -- 172.21.15.167:0/2152780636 >> 172.21.15.167:0/2152780636 conn(0x7f00bc0d1ba0 msgr2=0x7f00bc0d1fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:07:08.427 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.425+0000 7f00c2645640 1 -- 172.21.15.167:0/2152780636 shutdown_connections 2024-05-11T08:07:08.427 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.425+0000 7f00c2645640 1 -- 172.21.15.167:0/2152780636 wait complete. 2024-05-11T08:07:08.427 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.426+0000 7f00c2645640 1 Processor -- start 2024-05-11T08:07:08.427 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.426+0000 7f00c2645640 1 -- start start 2024-05-11T08:07:08.429 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.427+0000 7f00c2645640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f00bc0cb6c0 0x7f00bc14eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:08.429 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.427+0000 7f00c2645640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f00bc0d0640 0x7f00bc0cc1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:08.429 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.427+0000 7f00c2645640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f00bc1598d0 0x7f00bc1f9070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:08.429 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.427+0000 7f00c2645640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f00bc06d1b0 con 0x7f00bc0d0640 2024-05-11T08:07:08.429 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.427+0000 7f00c2645640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f00bc159cb0 con 0x7f00bc1598d0 2024-05-11T08:07:08.429 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.427+0000 7f00c2645640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f00bc06dfe0 con 0x7f00bc0cb6c0 2024-05-11T08:07:08.429 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.427+0000 7f00bb7fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f00bc0d0640 0x7f00bc0cc1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:08.429 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.427+0000 7f00bb7fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f00bc0d0640 0x7f00bc0cc1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:55834/0 (socket says 172.21.15.167:55834) 2024-05-11T08:07:08.429 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.427+0000 7f00bb7fe640 1 -- 172.21.15.167:0/1077036151 learned_addr learned my addr 172.21.15.167:0/1077036151 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:07:08.429 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.427+0000 7f00c0be2640 1 --2- 172.21.15.167:0/1077036151 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f00bc1598d0 0x7f00bc1f9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:08.430 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.427+0000 7f00bbfff640 1 --2- 172.21.15.167:0/1077036151 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f00bc0cb6c0 0x7f00bc14eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:08.430 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.427+0000 7f00bb7fe640 1 -- 172.21.15.167:0/1077036151 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f00bc0cb6c0 msgr2=0x7f00bc14eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:08.430 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.427+0000 7f00bb7fe640 1 --2- 172.21.15.167:0/1077036151 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f00bc0cb6c0 0x7f00bc14eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:08.430 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.427+0000 7f00bb7fe640 1 -- 172.21.15.167:0/1077036151 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f00bc1598d0 msgr2=0x7f00bc1f9070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:08.430 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.427+0000 7f00bb7fe640 1 --2- 172.21.15.167:0/1077036151 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f00bc1598d0 0x7f00bc1f9070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:08.430 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.427+0000 7f00bb7fe640 1 -- 172.21.15.167:0/1077036151 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00bc0cc940 con 0x7f00bc0d0640 2024-05-11T08:07:08.430 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.427+0000 7f00bb7fe640 1 --2- 172.21.15.167:0/1077036151 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f00bc0d0640 0x7f00bc0cc1f0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f00b000c450 tx=0x7f00b000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:08.430 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.428+0000 7f00b8ff9640 1 -- 172.21.15.167:0/1077036151 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00b00052d0 con 0x7f00bc0d0640 2024-05-11T08:07:08.430 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.428+0000 7f00b8ff9640 1 -- 172.21.15.167:0/1077036151 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00b0005470 con 0x7f00bc0d0640 2024-05-11T08:07:08.430 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.428+0000 7f00b8ff9640 1 -- 172.21.15.167:0/1077036151 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00b0012870 con 0x7f00bc0d0640 2024-05-11T08:07:08.430 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.428+0000 7f00c2645640 1 -- 172.21.15.167:0/1077036151 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00bc0ccc20 con 0x7f00bc0d0640 2024-05-11T08:07:08.430 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.428+0000 7f00c2645640 1 -- 172.21.15.167:0/1077036151 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00bc1f95b0 con 0x7f00bc0d0640 2024-05-11T08:07:08.430 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.429+0000 7f00c2645640 1 -- 172.21.15.167:0/1077036151 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00bc1f89a0 con 0x7f00bc0d0640 2024-05-11T08:07:08.431 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.429+0000 7f00b8ff9640 1 -- 172.21.15.167:0/1077036151 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f00b0012a10 con 0x7f00bc0d0640 2024-05-11T08:07:08.431 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.430+0000 7f00b8ff9640 1 --2- 172.21.15.167:0/1077036151 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f00a80762d0 0x7f00a8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:08.431 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.430+0000 7f00b8ff9640 1 -- 172.21.15.167:0/1077036151 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 6496+0+0 (secure 0 0 0) 0x7f00b0097330 con 0x7f00bc0d0640 2024-05-11T08:07:08.431 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.430+0000 7f00bbfff640 1 --2- 172.21.15.167:0/1077036151 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f00a80762d0 0x7f00a8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:08.432 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.430+0000 7f00bbfff640 1 --2- 172.21.15.167:0/1077036151 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f00a80762d0 0x7f00a8078790 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f00ac009aa0 tx=0x7f00ac009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:08.434 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.434+0000 7f00b8ff9640 1 -- 172.21.15.167:0/1077036151 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f00b00603c0 con 0x7f00bc0d0640 2024-05-11T08:07:08.625 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.624+0000 7f00c2645640 1 -- 172.21.15.167:0/1077036151 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f00bc0bb310 con 0x7f00bc0d0640 2024-05-11T08:07:08.625 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.625+0000 7f00b8ff9640 1 -- 172.21.15.167:0/1077036151 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v44) ==== 80+0+0 (secure 0 0 0) 0x7f00b00646c0 con 0x7f00bc0d0640 2024-05-11T08:07:08.625 INFO:teuthology.orchestra.run.smithi167.stderr:listed 0 entries 2024-05-11T08:07:08.627 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.627+0000 7f00c2645640 1 -- 172.21.15.167:0/1077036151 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f00a80762d0 msgr2=0x7f00a8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:08.627 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.627+0000 7f00c2645640 1 --2- 172.21.15.167:0/1077036151 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f00a80762d0 0x7f00a8078790 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f00ac009aa0 tx=0x7f00ac009190 comp rx=0 tx=0).stop 2024-05-11T08:07:08.628 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.627+0000 7f00c2645640 1 -- 172.21.15.167:0/1077036151 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f00bc0d0640 msgr2=0x7f00bc0cc1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:08.628 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.627+0000 7f00c2645640 1 --2- 172.21.15.167:0/1077036151 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f00bc0d0640 0x7f00bc0cc1f0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f00b000c450 tx=0x7f00b000c910 comp rx=0 tx=0).stop 2024-05-11T08:07:08.628 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.627+0000 7f00c2645640 1 -- 172.21.15.167:0/1077036151 shutdown_connections 2024-05-11T08:07:08.628 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.627+0000 7f00c2645640 1 --2- 172.21.15.167:0/1077036151 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f00a80762d0 0x7f00a8078790 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:08.628 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.627+0000 7f00c2645640 1 --2- 172.21.15.167:0/1077036151 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f00bc1598d0 0x7f00bc1f9070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:08.628 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.627+0000 7f00c2645640 1 --2- 172.21.15.167:0/1077036151 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f00bc0d0640 0x7f00bc0cc1f0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:08.628 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.627+0000 7f00c2645640 1 --2- 172.21.15.167:0/1077036151 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f00bc0cb6c0 0x7f00bc14eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:08.628 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.627+0000 7f00c2645640 1 -- 172.21.15.167:0/1077036151 >> 172.21.15.167:0/1077036151 conn(0x7f00bc0d1ba0 msgr2=0x7f00bc0cf4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:07:08.628 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.627+0000 7f00c2645640 1 -- 172.21.15.167:0/1077036151 shutdown_connections 2024-05-11T08:07:08.628 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.627+0000 7f00c2645640 1 -- 172.21.15.167:0/1077036151 wait complete. 2024-05-11T08:07:08.638 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-11T08:07:08.638 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T08:07:08.639 DEBUG:teuthology.orchestra.run.smithi167:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-05-11T08:07:08.685 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi167.front.sepia.ceph.com 2024-05-11T08:07:08.685 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T08:07:08.685 DEBUG:teuthology.orchestra.run.smithi167:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-11T08:07:08.710 INFO:tasks.cephfs.mount:Terminating background process 2024-05-11T08:07:08.710 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-11T08:07:08.710 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T08:07:08.727 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056314,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497640k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1336 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248820k,nr_inodes=812205,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497640k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-11T08:07:08.727 INFO:tasks.cephfs.mount:Terminating background process 2024-05-11T08:07:08.727 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-05-11T08:07:08.727 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T08:07:08.766 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056335,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20691 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-11T08:07:08.766 INFO:tasks.cephfs.mount:Terminating background process 2024-05-11T08:07:08.766 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-11T08:07:08.766 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T08:07:08.783 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056314,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497640k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1336 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248820k,nr_inodes=812205,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497640k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-11T08:07:08.783 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-11T08:07:08.906 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.906+0000 7fbd87358640 1 Processor -- start 2024-05-11T08:07:08.906 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.906+0000 7fbd87358640 1 -- start start 2024-05-11T08:07:08.907 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.907+0000 7fbd87358640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fbd801598d0 0x7fbd800d2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:08.908 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.907+0000 7fbd87358640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbd800d0640 0x7fbd800d0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:08.908 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.907+0000 7fbd87358640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fbd800cafb0 0x7fbd800cb400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:08.908 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.907+0000 7fbd87358640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7fbd800cb940 con 0x7fbd800cafb0 2024-05-11T08:07:08.908 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.907+0000 7fbd87358640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fbd800d1190 con 0x7fbd801598d0 2024-05-11T08:07:08.908 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.907+0000 7fbd87358640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fbd8006d500 con 0x7fbd800d0640 2024-05-11T08:07:08.908 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.907+0000 7fbd858f5640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fbd800cafb0 0x7fbd800cb400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:08.908 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.907+0000 7fbd858f5640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fbd800cafb0 0x7fbd800cb400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:55836/0 (socket says 172.21.15.167:55836) 2024-05-11T08:07:08.908 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.907+0000 7fbd858f5640 1 -- 172.21.15.167:0/184070999 learned_addr learned my addr 172.21.15.167:0/184070999 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:07:08.908 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.907+0000 7fbd850f4640 1 --2- 172.21.15.167:0/184070999 >> v2:172.21.15.204:3300/0 conn(0x7fbd801598d0 0x7fbd800d2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:08.908 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.907+0000 7fbd848f3640 1 --2- 172.21.15.167:0/184070999 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbd800d0640 0x7fbd800d0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:08.908 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.907+0000 7fbd858f5640 1 -- 172.21.15.167:0/184070999 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbd800d0640 msgr2=0x7fbd800d0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:07:08.908 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.907+0000 7fbd858f5640 1 --2- 172.21.15.167:0/184070999 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbd800d0640 0x7fbd800d0aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:08.909 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.907+0000 7fbd858f5640 1 -- 172.21.15.167:0/184070999 >> v2:172.21.15.204:3300/0 conn(0x7fbd801598d0 msgr2=0x7fbd800d2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:07:08.909 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.907+0000 7fbd858f5640 1 --2- 172.21.15.167:0/184070999 >> v2:172.21.15.204:3300/0 conn(0x7fbd801598d0 0x7fbd800d2cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:08.909 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.907+0000 7fbd858f5640 1 -- 172.21.15.167:0/184070999 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd8006d7e0 con 0x7fbd800cafb0 2024-05-11T08:07:08.909 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.907+0000 7fbd848f3640 1 --2- 172.21.15.167:0/184070999 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbd800d0640 0x7fbd800d0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T08:07:08.912 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.908+0000 7fbd858f5640 1 --2- 172.21.15.167:0/184070999 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbd800cafb0 0x7fbd800cb400 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fbd8008d610 tx=0x7fbd7802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=590eefac6321bd server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:08.912 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.908+0000 7fbd77fff640 1 -- 172.21.15.167:0/184070999 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd78036070 con 0x7fbd800cafb0 2024-05-11T08:07:08.912 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.908+0000 7fbd77fff640 1 -- 172.21.15.167:0/184070999 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd78031600 con 0x7fbd800cafb0 2024-05-11T08:07:08.912 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.908+0000 7fbd77fff640 1 -- 172.21.15.167:0/184070999 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd780319d0 con 0x7fbd800cafb0 2024-05-11T08:07:08.912 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.909+0000 7fbd87358640 1 -- 172.21.15.167:0/184070999 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbd800cafb0 msgr2=0x7fbd800cb400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:08.912 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.909+0000 7fbd87358640 1 --2- 172.21.15.167:0/184070999 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbd800cafb0 0x7fbd800cb400 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fbd8008d610 tx=0x7fbd7802a6a0 comp rx=0 tx=0).stop 2024-05-11T08:07:08.912 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.909+0000 7fbd87358640 1 -- 172.21.15.167:0/184070999 shutdown_connections 2024-05-11T08:07:08.912 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.909+0000 7fbd87358640 1 --2- 172.21.15.167:0/184070999 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbd800cafb0 0x7fbd800cb400 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:08.912 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.909+0000 7fbd87358640 1 --2- 172.21.15.167:0/184070999 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbd800d0640 0x7fbd800d0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:08.913 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.909+0000 7fbd87358640 1 --2- 172.21.15.167:0/184070999 >> v2:172.21.15.204:3300/0 conn(0x7fbd801598d0 0x7fbd800d2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:08.913 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.909+0000 7fbd87358640 1 -- 172.21.15.167:0/184070999 >> 172.21.15.167:0/184070999 conn(0x7fbd800d1ba0 msgr2=0x7fbd800d1fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:07:08.913 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.909+0000 7fbd87358640 1 -- 172.21.15.167:0/184070999 shutdown_connections 2024-05-11T08:07:08.913 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.909+0000 7fbd87358640 1 -- 172.21.15.167:0/184070999 wait complete. 2024-05-11T08:07:08.913 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.910+0000 7fbd87358640 1 Processor -- start 2024-05-11T08:07:08.913 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.910+0000 7fbd87358640 1 -- start start 2024-05-11T08:07:08.913 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.911+0000 7fbd87358640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbd800cafb0 0x7fbd8014db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:08.913 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.911+0000 7fbd87358640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbd800d0640 0x7fbd80162c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:08.913 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.911+0000 7fbd87358640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbd801598d0 0x7fbd801631d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:08.913 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.911+0000 7fbd87358640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fbd80159cb0 con 0x7fbd801598d0 2024-05-11T08:07:08.913 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.911+0000 7fbd87358640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fbd801607c0 con 0x7fbd800cafb0 2024-05-11T08:07:08.913 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.911+0000 7fbd87358640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fbd8006d1b0 con 0x7fbd800d0640 2024-05-11T08:07:08.913 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.911+0000 7fbd858f5640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbd801598d0 0x7fbd801631d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:08.913 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.911+0000 7fbd858f5640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbd801598d0 0x7fbd801631d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:55844/0 (socket says 172.21.15.167:55844) 2024-05-11T08:07:08.913 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.911+0000 7fbd858f5640 1 -- 172.21.15.167:0/1201288805 learned_addr learned my addr 172.21.15.167:0/1201288805 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:07:08.914 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.911+0000 7fbd848f3640 1 --2- 172.21.15.167:0/1201288805 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbd800d0640 0x7fbd80162c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:08.914 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.911+0000 7fbd850f4640 1 --2- 172.21.15.167:0/1201288805 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbd800cafb0 0x7fbd8014db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:08.914 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.911+0000 7fbd858f5640 1 -- 172.21.15.167:0/1201288805 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbd800d0640 msgr2=0x7fbd80162c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:08.914 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.911+0000 7fbd858f5640 1 --2- 172.21.15.167:0/1201288805 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbd800d0640 0x7fbd80162c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:08.914 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.911+0000 7fbd858f5640 1 -- 172.21.15.167:0/1201288805 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbd800cafb0 msgr2=0x7fbd8014db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:08.914 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.911+0000 7fbd858f5640 1 --2- 172.21.15.167:0/1201288805 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbd800cafb0 0x7fbd8014db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:08.914 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.911+0000 7fbd858f5640 1 -- 172.21.15.167:0/1201288805 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd800cf9d0 con 0x7fbd801598d0 2024-05-11T08:07:08.914 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.911+0000 7fbd858f5640 1 --2- 172.21.15.167:0/1201288805 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbd801598d0 0x7fbd801631d0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fbd78009f40 tx=0x7fbd78004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:08.914 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.911+0000 7fbd75ffb640 1 -- 172.21.15.167:0/1201288805 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd78036050 con 0x7fbd801598d0 2024-05-11T08:07:08.914 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.911+0000 7fbd75ffb640 1 -- 172.21.15.167:0/1201288805 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd78042070 con 0x7fbd801598d0 2024-05-11T08:07:08.914 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.911+0000 7fbd75ffb640 1 -- 172.21.15.167:0/1201288805 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd78009510 con 0x7fbd801598d0 2024-05-11T08:07:08.914 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.912+0000 7fbd87358640 1 -- 172.21.15.167:0/1201288805 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd800ce930 con 0x7fbd801598d0 2024-05-11T08:07:08.914 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.912+0000 7fbd87358640 1 -- 172.21.15.167:0/1201288805 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd800cf1d0 con 0x7fbd801598d0 2024-05-11T08:07:08.915 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.913+0000 7fbd87358640 1 -- 172.21.15.167:0/1201288805 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd800ced20 con 0x7fbd801598d0 2024-05-11T08:07:08.915 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.913+0000 7fbd75ffb640 1 -- 172.21.15.167:0/1201288805 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fbd78007bd0 con 0x7fbd801598d0 2024-05-11T08:07:08.915 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.914+0000 7fbd75ffb640 1 --2- 172.21.15.167:0/1201288805 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fbd640762d0 0x7fbd64078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:08.916 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.914+0000 7fbd75ffb640 1 -- 172.21.15.167:0/1201288805 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 6496+0+0 (secure 0 0 0) 0x7fbd780b8640 con 0x7fbd801598d0 2024-05-11T08:07:08.916 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.914+0000 7fbd850f4640 1 --2- 172.21.15.167:0/1201288805 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fbd640762d0 0x7fbd64078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:08.916 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.914+0000 7fbd850f4640 1 --2- 172.21.15.167:0/1201288805 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fbd640762d0 0x7fbd64078790 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fbd7c006fd0 tx=0x7fbd7c008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:08.918 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:08.918+0000 7fbd75ffb640 1 -- 172.21.15.167:0/1201288805 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fbd78081640 con 0x7fbd801598d0 2024-05-11T08:07:09.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.136+0000 7fbd87358640 1 -- 172.21.15.167:0/1201288805 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbd800d1780 con 0x7fbd801598d0 2024-05-11T08:07:09.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.137+0000 7fbd75ffb640 1 -- 172.21.15.167:0/1201288805 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5192 (secure 0 0 0) 0x7fbd800d1780 con 0x7fbd801598d0 2024-05-11T08:07:09.137 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T08:07:09.137 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4604,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2674983673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":2674983673},{"type":"v1","addr":"172.21.15.167:6837","nonce":2674983673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4607,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6834/1018548066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1018548066},{"type":"v1","addr":"172.21.15.204:6834","nonce":1018548066}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4729,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3369722249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3369722249},{"type":"v1","addr":"172.21.15.167:6836","nonce":3369722249}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-11T08:06:47.943939+0000","modified":"2024-05-11T08:07:04.006894+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4750},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4750":{"gid":4750,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.204:6835/469864006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6833","nonce":469864006},{"type":"v1","addr":"172.21.15.204:6835","nonce":469864006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6,7],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4750,"qdb_cluster":[4750]},"id":2}]} 2024-05-11T08:07:09.138 INFO:teuthology.orchestra.run.smithi167.stderr:dumped fsmap epoch 17 2024-05-11T08:07:09.140 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.139+0000 7fbd87358640 1 -- 172.21.15.167:0/1201288805 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fbd640762d0 msgr2=0x7fbd64078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:09.140 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.139+0000 7fbd87358640 1 --2- 172.21.15.167:0/1201288805 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fbd640762d0 0x7fbd64078790 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fbd7c006fd0 tx=0x7fbd7c008040 comp rx=0 tx=0).stop 2024-05-11T08:07:09.140 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.139+0000 7fbd87358640 1 -- 172.21.15.167:0/1201288805 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbd801598d0 msgr2=0x7fbd801631d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:09.140 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.139+0000 7fbd87358640 1 --2- 172.21.15.167:0/1201288805 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbd801598d0 0x7fbd801631d0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fbd78009f40 tx=0x7fbd78004a40 comp rx=0 tx=0).stop 2024-05-11T08:07:09.140 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.139+0000 7fbd87358640 1 -- 172.21.15.167:0/1201288805 shutdown_connections 2024-05-11T08:07:09.140 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.139+0000 7fbd87358640 1 --2- 172.21.15.167:0/1201288805 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fbd640762d0 0x7fbd64078790 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:09.140 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.139+0000 7fbd87358640 1 --2- 172.21.15.167:0/1201288805 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbd801598d0 0x7fbd801631d0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:09.140 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.139+0000 7fbd87358640 1 --2- 172.21.15.167:0/1201288805 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbd800d0640 0x7fbd80162c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:09.140 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.139+0000 7fbd87358640 1 --2- 172.21.15.167:0/1201288805 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbd800cafb0 0x7fbd8014db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:09.140 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.139+0000 7fbd87358640 1 -- 172.21.15.167:0/1201288805 >> 172.21.15.167:0/1201288805 conn(0x7fbd800d1ba0 msgr2=0x7fbd800d32c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:07:09.140 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.140+0000 7fbd87358640 1 -- 172.21.15.167:0/1201288805 shutdown_connections 2024-05-11T08:07:09.140 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.140+0000 7fbd87358640 1 -- 172.21.15.167:0/1201288805 wait complete. 2024-05-11T08:07:09.151 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-11T08:07:09.250 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.250+0000 7f5380220640 1 Processor -- start 2024-05-11T08:07:09.250 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.250+0000 7f5380220640 1 -- start start 2024-05-11T08:07:09.253 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.251+0000 7f5380220640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f537813f290 0x7f53780735c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:09.254 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.251+0000 7f5380220640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f53780b50a0 0x7f5378073b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:09.254 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.251+0000 7f5380220640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f537815fb90 con 0x7f537813eeb0 2024-05-11T08:07:09.254 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.251+0000 7f5380220640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f537815d5b0 con 0x7f537813f290 2024-05-11T08:07:09.254 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.251+0000 7f5380220640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f537815fe70 con 0x7f53780b50a0 2024-05-11T08:07:09.254 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.251+0000 7f537dfbc640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f537813eeb0 0x7f537806f480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:47420/0 (socket says 172.21.15.167:47420) 2024-05-11T08:07:09.254 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.251+0000 7f537dfbc640 1 -- 172.21.15.167:0/3775665230 learned_addr learned my addr 172.21.15.167:0/3775665230 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T08:07:09.254 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.251+0000 7f537e7bd640 1 --2- 172.21.15.167:0/3775665230 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f53780b50a0 0x7f5378073b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:09.254 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.251+0000 7f537d7bb640 1 --2- 172.21.15.167:0/3775665230 >> v2:172.21.15.204:3300/0 conn(0x7f537813f290 0x7f53780735c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:09.254 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.251+0000 7f537cfba640 1 -- 172.21.15.167:0/3775665230 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3989525373 0 0) 0x7f537815fb90 con 0x7f537813eeb0 2024-05-11T08:07:09.254 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.251+0000 7f537cfba640 1 -- 172.21.15.167:0/3775665230 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5364003520 con 0x7f537813eeb0 2024-05-11T08:07:09.254 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.251+0000 7f537cfba640 1 -- 172.21.15.167:0/3775665230 <== mon.0 v1:172.21.15.167:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3848861971 0 0) 0x7f5364003520 con 0x7f537813eeb0 2024-05-11T08:07:09.254 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.251+0000 7f537cfba640 1 -- 172.21.15.167:0/3775665230 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f53780b50a0 msgr2=0x7f5378073b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:07:09.254 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.251+0000 7f537cfba640 1 --2- 172.21.15.167:0/3775665230 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f53780b50a0 0x7f5378073b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:09.255 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.251+0000 7f537cfba640 1 -- 172.21.15.167:0/3775665230 >> v2:172.21.15.204:3300/0 conn(0x7f537813f290 msgr2=0x7f53780735c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:07:09.255 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.251+0000 7f537cfba640 1 --2- 172.21.15.167:0/3775665230 >> v2:172.21.15.204:3300/0 conn(0x7f537813f290 0x7f53780735c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:09.255 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.251+0000 7f537cfba640 1 -- 172.21.15.167:0/3775665230 --> v1:172.21.15.167:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5378160150 con 0x7f537813eeb0 2024-05-11T08:07:09.255 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.251+0000 7f537cfba640 1 -- 172.21.15.167:0/3775665230 <== mon.0 v1:172.21.15.167:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3602277846 0 0) 0x7f53740016f0 con 0x7f537813eeb0 2024-05-11T08:07:09.255 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.251+0000 7f537cfba640 1 -- 172.21.15.167:0/3775665230 >> v1:172.21.15.167:6789/0 conn(0x7f537813eeb0 legacy=0x7f537806f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:09.255 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.251+0000 7f537cfba640 1 --2- 172.21.15.167:0/3775665230 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f53780b50a0 0x7f5364003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:09.255 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.251+0000 7f537cfba640 1 --2- 172.21.15.167:0/3775665230 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5364003fd0 0x7f5364004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:09.255 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.251+0000 7f537cfba640 1 --2- 172.21.15.167:0/3775665230 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5364004d40 0x7f53640051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:09.255 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.252+0000 7f537cfba640 1 -- 172.21.15.167:0/3775665230 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f537815fe70 con 0x7f5364003fd0 2024-05-11T08:07:09.255 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.252+0000 7f537cfba640 1 -- 172.21.15.167:0/3775665230 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f536400ba20 con 0x7f5364004d40 2024-05-11T08:07:09.255 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.252+0000 7f537cfba640 1 -- 172.21.15.167:0/3775665230 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f536400bc20 con 0x7f53780b50a0 2024-05-11T08:07:09.255 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.252+0000 7f537d7bb640 1 --2- 172.21.15.167:0/3775665230 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5364003fd0 0x7f5364004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:09.255 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.252+0000 7f537dfbc640 1 --2- 172.21.15.167:0/3775665230 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f53780b50a0 0x7f5364003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:09.255 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.252+0000 7f537e7bd640 1 --2- 172.21.15.167:0/3775665230 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5364004d40 0x7f53640051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:09.255 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.252+0000 7f537d7bb640 1 -- 172.21.15.167:0/3775665230 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f53780b50a0 msgr2=0x7f5364003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:07:09.256 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.252+0000 7f537d7bb640 1 --2- 172.21.15.167:0/3775665230 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f53780b50a0 0x7f5364003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:09.256 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.252+0000 7f537d7bb640 1 -- 172.21.15.167:0/3775665230 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5364004d40 msgr2=0x7f53640051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:07:09.256 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.252+0000 7f537d7bb640 1 --2- 172.21.15.167:0/3775665230 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5364004d40 0x7f53640051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:09.256 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.252+0000 7f537d7bb640 1 -- 172.21.15.167:0/3775665230 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5364003520 con 0x7f5364003fd0 2024-05-11T08:07:09.256 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.253+0000 7f537d7bb640 1 --2- 172.21.15.167:0/3775665230 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5364003fd0 0x7f5364004800 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f5368009ac0 tx=0x7f5364005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c8f3d8eade867b2 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:09.256 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.253+0000 7f537cfba640 1 -- 172.21.15.167:0/3775665230 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5368036070 con 0x7f5364003fd0 2024-05-11T08:07:09.256 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.253+0000 7f537cfba640 1 -- 172.21.15.167:0/3775665230 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5368031400 con 0x7f5364003fd0 2024-05-11T08:07:09.256 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.253+0000 7f537cfba640 1 -- 172.21.15.167:0/3775665230 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5368009690 con 0x7f5364003fd0 2024-05-11T08:07:09.256 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.253+0000 7f5380220640 1 -- 172.21.15.167:0/3775665230 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5364003fd0 msgr2=0x7f5364004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:09.256 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.253+0000 7f5380220640 1 --2- 172.21.15.167:0/3775665230 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5364003fd0 0x7f5364004800 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f5368009ac0 tx=0x7f5364005830 comp rx=0 tx=0).stop 2024-05-11T08:07:09.256 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.253+0000 7f537e7bd640 1 -- 172.21.15.167:0/3775665230 reap_dead start 2024-05-11T08:07:09.256 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.253+0000 7f5380220640 1 -- 172.21.15.167:0/3775665230 shutdown_connections 2024-05-11T08:07:09.256 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.253+0000 7f5380220640 1 -- 172.21.15.167:0/3775665230 >> 172.21.15.167:0/3775665230 conn(0x7f53780cd8f0 msgr2=0x7f53780cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:07:09.256 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.253+0000 7f5380220640 1 -- 172.21.15.167:0/3775665230 shutdown_connections 2024-05-11T08:07:09.257 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.253+0000 7f5380220640 1 -- 172.21.15.167:0/3775665230 wait complete. 2024-05-11T08:07:09.259 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.254+0000 7f5380220640 1 Processor -- start 2024-05-11T08:07:09.259 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.255+0000 7f5380220640 1 -- start start 2024-05-11T08:07:09.259 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.255+0000 7f5380220640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5364003fd0 0x7f5378152fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:09.259 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.255+0000 7f5380220640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f537815dc90 0x7f537815c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:09.259 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.255+0000 7f5380220640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f537813b980 0x7f537814dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:09.259 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.255+0000 7f5380220640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f537806fb90 con 0x7f537815dc90 2024-05-11T08:07:09.259 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.255+0000 7f5380220640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5378160710 con 0x7f537813b980 2024-05-11T08:07:09.260 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.255+0000 7f5380220640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f537806fd10 con 0x7f5364003fd0 2024-05-11T08:07:09.260 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.255+0000 7f537d7bb640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f537815dc90 0x7f537815c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:09.260 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.255+0000 7f537d7bb640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f537815dc90 0x7f537815c430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:55862/0 (socket says 172.21.15.167:55862) 2024-05-11T08:07:09.260 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.255+0000 7f537d7bb640 1 -- 172.21.15.167:0/2404844312 learned_addr learned my addr 172.21.15.167:0/2404844312 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:07:09.260 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.256+0000 7f537d7bb640 1 -- 172.21.15.167:0/2404844312 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5364003fd0 msgr2=0x7f5378152fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:09.260 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.256+0000 7f537d7bb640 1 --2- 172.21.15.167:0/2404844312 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5364003fd0 0x7f5378152fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:09.260 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.256+0000 7f537e7bd640 1 --2- 172.21.15.167:0/2404844312 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f537813b980 0x7f537814dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:09.260 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.256+0000 7f537d7bb640 1 -- 172.21.15.167:0/2404844312 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f537813b980 msgr2=0x7f537814dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:09.260 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.256+0000 7f537d7bb640 1 --2- 172.21.15.167:0/2404844312 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f537813b980 0x7f537814dc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:09.260 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.256+0000 7f537d7bb640 1 -- 172.21.15.167:0/2404844312 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f537815c970 con 0x7f537815dc90 2024-05-11T08:07:09.260 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.256+0000 7f537d7bb640 1 --2- 172.21.15.167:0/2404844312 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f537815dc90 0x7f537815c430 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f5368000c00 tx=0x7f5368004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:09.260 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.256+0000 7f536e7fc640 1 -- 172.21.15.167:0/2404844312 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5368040020 con 0x7f537815dc90 2024-05-11T08:07:09.260 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.256+0000 7f536e7fc640 1 -- 172.21.15.167:0/2404844312 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5368003c60 con 0x7f537815dc90 2024-05-11T08:07:09.260 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.256+0000 7f5380220640 1 -- 172.21.15.167:0/2404844312 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f537815cbf0 con 0x7f537815dc90 2024-05-11T08:07:09.261 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.256+0000 7f536e7fc640 1 -- 172.21.15.167:0/2404844312 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5368036040 con 0x7f537815dc90 2024-05-11T08:07:09.261 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.256+0000 7f5380220640 1 -- 172.21.15.167:0/2404844312 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f537815d1b0 con 0x7f537815dc90 2024-05-11T08:07:09.261 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.257+0000 7f5380220640 1 -- 172.21.15.167:0/2404844312 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5378069590 con 0x7f537815dc90 2024-05-11T08:07:09.261 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.258+0000 7f536e7fc640 1 -- 172.21.15.167:0/2404844312 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f5368043020 con 0x7f537815dc90 2024-05-11T08:07:09.261 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.258+0000 7f536e7fc640 1 --2- 172.21.15.167:0/2404844312 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f535c0762a0 0x7f535c078760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:09.261 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.258+0000 7f536e7fc640 1 -- 172.21.15.167:0/2404844312 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 6496+0+0 (secure 0 0 0) 0x7f53680b7ac0 con 0x7f537815dc90 2024-05-11T08:07:09.261 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.258+0000 7f537dfbc640 1 --2- 172.21.15.167:0/2404844312 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f535c0762a0 0x7f535c078760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:09.261 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.259+0000 7f537dfbc640 1 --2- 172.21.15.167:0/2404844312 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f535c0762a0 0x7f535c078760 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f5374007cd0 tx=0x7f53740014b0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:09.262 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.262+0000 7f536e7fc640 1 -- 172.21.15.167:0/2404844312 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f536804a070 con 0x7f537815dc90 2024-05-11T08:07:09.479 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.479+0000 7f5380220640 1 -- 172.21.15.167:0/2404844312 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f53780729b0 con 0x7f537815dc90 2024-05-11T08:07:09.479 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.479+0000 7f536e7fc640 1 -- 172.21.15.167:0/2404844312 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5192 (secure 0 0 0) 0x7f537815c970 con 0x7f537815dc90 2024-05-11T08:07:09.479 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T08:07:09.480 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4604,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2674983673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":2674983673},{"type":"v1","addr":"172.21.15.167:6837","nonce":2674983673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4607,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6834/1018548066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1018548066},{"type":"v1","addr":"172.21.15.204:6834","nonce":1018548066}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4729,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3369722249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3369722249},{"type":"v1","addr":"172.21.15.167:6836","nonce":3369722249}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-11T08:06:47.943939+0000","modified":"2024-05-11T08:07:04.006894+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4750},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4750":{"gid":4750,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.204:6835/469864006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6833","nonce":469864006},{"type":"v1","addr":"172.21.15.204:6835","nonce":469864006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6,7],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4750,"qdb_cluster":[4750]},"id":2}]} 2024-05-11T08:07:09.480 INFO:teuthology.orchestra.run.smithi167.stderr:dumped fsmap epoch 17 2024-05-11T08:07:09.482 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.481+0000 7f5380220640 1 -- 172.21.15.167:0/2404844312 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f535c0762a0 msgr2=0x7f535c078760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:09.482 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.481+0000 7f5380220640 1 --2- 172.21.15.167:0/2404844312 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f535c0762a0 0x7f535c078760 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f5374007cd0 tx=0x7f53740014b0 comp rx=0 tx=0).stop 2024-05-11T08:07:09.482 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.481+0000 7f5380220640 1 -- 172.21.15.167:0/2404844312 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f537815dc90 msgr2=0x7f537815c430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:09.482 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.481+0000 7f5380220640 1 --2- 172.21.15.167:0/2404844312 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f537815dc90 0x7f537815c430 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f5368000c00 tx=0x7f5368004730 comp rx=0 tx=0).stop 2024-05-11T08:07:09.482 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.481+0000 7f5380220640 1 -- 172.21.15.167:0/2404844312 shutdown_connections 2024-05-11T08:07:09.482 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.481+0000 7f5380220640 1 --2- 172.21.15.167:0/2404844312 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f535c0762a0 0x7f535c078760 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:09.482 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.481+0000 7f5380220640 1 --2- 172.21.15.167:0/2404844312 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f537813b980 0x7f537814dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:09.482 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.481+0000 7f5380220640 1 --2- 172.21.15.167:0/2404844312 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f537815dc90 0x7f537815c430 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:09.482 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.481+0000 7f5380220640 1 --2- 172.21.15.167:0/2404844312 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5364003fd0 0x7f5378152fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:09.483 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.481+0000 7f5380220640 1 -- 172.21.15.167:0/2404844312 >> 172.21.15.167:0/2404844312 conn(0x7f53780cd8f0 msgr2=0x7f53780cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:07:09.483 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.482+0000 7f5380220640 1 -- 172.21.15.167:0/2404844312 shutdown_connections 2024-05-11T08:07:09.483 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.482+0000 7f5380220640 1 -- 172.21.15.167:0/2404844312 wait complete. 2024-05-11T08:07:09.493 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-11T08:07:09.593 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.593+0000 7efe9f66d640 1 Processor -- start 2024-05-11T08:07:09.594 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.594+0000 7efe9f66d640 1 -- start start 2024-05-11T08:07:09.595 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.594+0000 7efe9f66d640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7efe980d2cf0 0x7efe980cafb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:09.595 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.594+0000 7efe9f66d640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe980d1190 con 0x7efe981598d0 2024-05-11T08:07:09.595 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.594+0000 7efe9f66d640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe9806d500 con 0x7efe980cb4f0 2024-05-11T08:07:09.595 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.594+0000 7efe9f66d640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7efe980cb920 con 0x7efe980d2cf0 2024-05-11T08:07:09.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.595+0000 7efe9d409640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7efe981598d0 0x7efe980d0640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:47430/0 (socket says 172.21.15.167:47430) 2024-05-11T08:07:09.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.595+0000 7efe9d409640 1 -- 172.21.15.167:0/2646801047 learned_addr learned my addr 172.21.15.167:0/2646801047 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T08:07:09.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.595+0000 7efe9cc08640 1 --2- 172.21.15.167:0/2646801047 >> v2:172.21.15.204:3300/0 conn(0x7efe980d2cf0 0x7efe980cafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:09.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.595+0000 7efe97fff640 1 -- 172.21.15.167:0/2646801047 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2131417682 0 0) 0x7efe980d1190 con 0x7efe981598d0 2024-05-11T08:07:09.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.595+0000 7efe97fff640 1 -- 172.21.15.167:0/2646801047 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efe80003520 con 0x7efe981598d0 2024-05-11T08:07:09.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.595+0000 7efe97fff640 1 -- 172.21.15.167:0/2646801047 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2081637092 0 0) 0x7efe9806d500 con 0x7efe980cb4f0 2024-05-11T08:07:09.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.595+0000 7efe97fff640 1 -- 172.21.15.167:0/2646801047 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efe980d1190 con 0x7efe980cb4f0 2024-05-11T08:07:09.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.595+0000 7efe97fff640 1 -- 172.21.15.167:0/2646801047 <== mon.0 v1:172.21.15.167:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3204696935 0 0) 0x7efe80003520 con 0x7efe981598d0 2024-05-11T08:07:09.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.595+0000 7efe9cc08640 1 -- 172.21.15.167:0/2646801047 >> v1:172.21.15.204:6789/0 conn(0x7efe980cb4f0 legacy=0x7efe9806dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:09.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.595+0000 7efe9cc08640 1 -- 172.21.15.167:0/2646801047 >> v1:172.21.15.167:6789/0 conn(0x7efe981598d0 legacy=0x7efe980d0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:09.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.595+0000 7efe9cc08640 1 -- 172.21.15.167:0/2646801047 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe9806d7e0 con 0x7efe980d2cf0 2024-05-11T08:07:09.598 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.596+0000 7efe9cc08640 1 --2- 172.21.15.167:0/2646801047 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efe980d2cf0 0x7efe980cafb0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7efe8c0097a0 tx=0x7efe8c02a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e03fd968feb909dd server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:09.598 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.596+0000 7efe97fff640 1 -- 172.21.15.167:0/2646801047 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe8c037070 con 0x7efe980d2cf0 2024-05-11T08:07:09.598 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.596+0000 7efe97fff640 1 -- 172.21.15.167:0/2646801047 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe8c02ac50 con 0x7efe980d2cf0 2024-05-11T08:07:09.598 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.596+0000 7efe97fff640 1 -- 172.21.15.167:0/2646801047 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe88001a70 con 0x7efe980d2cf0 2024-05-11T08:07:09.598 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.596+0000 7efe9f66d640 1 -- 172.21.15.167:0/2646801047 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efe980d2cf0 msgr2=0x7efe980cafb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:09.598 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.596+0000 7efe9f66d640 1 --2- 172.21.15.167:0/2646801047 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efe980d2cf0 0x7efe980cafb0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7efe8c0097a0 tx=0x7efe8c02a1c0 comp rx=0 tx=0).stop 2024-05-11T08:07:09.598 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.597+0000 7efe9f66d640 1 -- 172.21.15.167:0/2646801047 shutdown_connections 2024-05-11T08:07:09.598 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.597+0000 7efe9f66d640 1 --2- 172.21.15.167:0/2646801047 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efe980d2cf0 0x7efe980cafb0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:09.598 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.597+0000 7efe9f66d640 1 -- 172.21.15.167:0/2646801047 >> 172.21.15.167:0/2646801047 conn(0x7efe980d1ba0 msgr2=0x7efe980d1fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:07:09.598 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.597+0000 7efe9f66d640 1 -- 172.21.15.167:0/2646801047 shutdown_connections 2024-05-11T08:07:09.599 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.597+0000 7efe9f66d640 1 -- 172.21.15.167:0/2646801047 wait complete. 2024-05-11T08:07:09.599 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.598+0000 7efe9f66d640 1 Processor -- start 2024-05-11T08:07:09.599 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.598+0000 7efe9f66d640 1 -- start start 2024-05-11T08:07:09.601 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.598+0000 7efe9f66d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efe980cb4f0 0x7efe980ba820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:09.601 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.598+0000 7efe9f66d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efe980d2cf0 0x7efe980cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:09.601 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.598+0000 7efe9f66d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efe981598d0 0x7efe9814aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:09.601 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.598+0000 7efe9f66d640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7efe98159cb0 con 0x7efe980cb4f0 2024-05-11T08:07:09.601 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.598+0000 7efe9f66d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7efe981607c0 con 0x7efe980d2cf0 2024-05-11T08:07:09.601 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.598+0000 7efe9f66d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7efe9806d1b0 con 0x7efe981598d0 2024-05-11T08:07:09.601 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.599+0000 7efe9d409640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efe980cb4f0 0x7efe980ba820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:09.601 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.599+0000 7efe9d409640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efe980cb4f0 0x7efe980ba820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:55876/0 (socket says 172.21.15.167:55876) 2024-05-11T08:07:09.601 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.599+0000 7efe9d409640 1 -- 172.21.15.167:0/3280687501 learned_addr learned my addr 172.21.15.167:0/3280687501 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:07:09.601 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.599+0000 7efe9cc08640 1 --2- 172.21.15.167:0/3280687501 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efe980d2cf0 0x7efe980cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:09.602 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.599+0000 7efe9dc0a640 1 --2- 172.21.15.167:0/3280687501 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efe981598d0 0x7efe9814aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:09.602 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.599+0000 7efe9d409640 1 -- 172.21.15.167:0/3280687501 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efe981598d0 msgr2=0x7efe9814aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:09.602 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.599+0000 7efe9d409640 1 --2- 172.21.15.167:0/3280687501 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efe981598d0 0x7efe9814aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:09.602 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.599+0000 7efe9d409640 1 -- 172.21.15.167:0/3280687501 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efe980d2cf0 msgr2=0x7efe980cfa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:09.602 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.599+0000 7efe9d409640 1 --2- 172.21.15.167:0/3280687501 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efe980d2cf0 0x7efe980cfa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:09.602 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.599+0000 7efe9d409640 1 -- 172.21.15.167:0/3280687501 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe981f5210 con 0x7efe980cb4f0 2024-05-11T08:07:09.602 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.599+0000 7efe9d409640 1 --2- 172.21.15.167:0/3280687501 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efe980cb4f0 0x7efe980ba820 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7efe88001ed0 tx=0x7efe8800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:09.602 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.599+0000 7efe95ffb640 1 -- 172.21.15.167:0/3280687501 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe88017070 con 0x7efe980cb4f0 2024-05-11T08:07:09.602 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.599+0000 7efe95ffb640 1 -- 172.21.15.167:0/3280687501 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe88012400 con 0x7efe980cb4f0 2024-05-11T08:07:09.602 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.599+0000 7efe95ffb640 1 -- 172.21.15.167:0/3280687501 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe880127d0 con 0x7efe980cb4f0 2024-05-11T08:07:09.602 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.599+0000 7efe9f66d640 1 -- 172.21.15.167:0/3280687501 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe981f5490 con 0x7efe980cb4f0 2024-05-11T08:07:09.602 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.600+0000 7efe9f66d640 1 -- 172.21.15.167:0/3280687501 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe981f5a50 con 0x7efe980cb4f0 2024-05-11T08:07:09.602 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.600+0000 7efe9f66d640 1 -- 172.21.15.167:0/3280687501 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe98069590 con 0x7efe980cb4f0 2024-05-11T08:07:09.602 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.601+0000 7efe95ffb640 1 -- 172.21.15.167:0/3280687501 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7efe8801a070 con 0x7efe980cb4f0 2024-05-11T08:07:09.603 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.601+0000 7efe95ffb640 1 --2- 172.21.15.167:0/3280687501 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7efe840762d0 0x7efe84078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:09.603 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.601+0000 7efe95ffb640 1 -- 172.21.15.167:0/3280687501 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 6496+0+0 (secure 0 0 0) 0x7efe88097850 con 0x7efe980cb4f0 2024-05-11T08:07:09.603 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.601+0000 7efe9cc08640 1 --2- 172.21.15.167:0/3280687501 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7efe840762d0 0x7efe84078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:09.603 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.602+0000 7efe9cc08640 1 --2- 172.21.15.167:0/3280687501 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7efe840762d0 0x7efe84078790 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7efe8c034000 tx=0x7efe8c02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:09.605 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.606+0000 7efe95ffb640 1 -- 172.21.15.167:0/3280687501 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7efe981f5210 con 0x7efe980cb4f0 2024-05-11T08:07:09.790 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.790+0000 7efe9f66d640 1 -- 172.21.15.167:0/3280687501 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efe980d0a60 con 0x7efe980cb4f0 2024-05-11T08:07:09.791 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.791+0000 7efe95ffb640 1 -- 172.21.15.167:0/3280687501 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) ==== 74+0+18802 (secure 0 0 0) 0x7efe88060ea0 con 0x7efe980cb4f0 2024-05-11T08:07:09.791 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T08:07:09.792 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":44,"fsid":"2034778d-8ff1-4823-aa44-ded32b518241","created":"2024-05-11T08:05:31.612421+0000","modified":"2024-05-11T08:07:05.656974+0000","last_up_change":"2024-05-11T08:05:42.557382+0000","last_in_change":"2024-05-11T08:05:33.757311+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-11T08:05:43.931263+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-05-11T08:06:45.857156+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-05-11T08:06:46.898591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"data_new","create_time":"2024-05-11T08:07:03.541639+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"587eaced-5ded-406d-884b-30c1caa8686b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":3863307356},{"type":"v1","addr":"172.21.15.167:6806","nonce":3863307356}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":3863307356},{"type":"v1","addr":"172.21.15.167:6810","nonce":3863307356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":3863307356},{"type":"v1","addr":"172.21.15.167:6815","nonce":3863307356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":3863307356},{"type":"v1","addr":"172.21.15.167:6813","nonce":3863307356}]},"public_addr":"172.21.15.167:6806/3863307356","cluster_addr":"172.21.15.167:6810/3863307356","heartbeat_back_addr":"172.21.15.167:6815/3863307356","heartbeat_front_addr":"172.21.15.167:6813/3863307356","state":["exists","up"]},{"osd":1,"uuid":"37c59104-2dd0-4cf9-85ea-56636f6b02a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":1585024048},{"type":"v1","addr":"172.21.15.167:6817","nonce":1585024048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1585024048},{"type":"v1","addr":"172.21.15.167:6819","nonce":1585024048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1585024048},{"type":"v1","addr":"172.21.15.167:6823","nonce":1585024048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":1585024048},{"type":"v1","addr":"172.21.15.167:6821","nonce":1585024048}]},"public_addr":"172.21.15.167:6817/1585024048","cluster_addr":"172.21.15.167:6819/1585024048","heartbeat_back_addr":"172.21.15.167:6823/1585024048","heartbeat_front_addr":"172.21.15.167:6821/1585024048","state":["exists","up"]},{"osd":2,"uuid":"2bb4f7f5-c698-451b-b4bb-78e716921291","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":3242166900},{"type":"v1","addr":"172.21.15.167:6801","nonce":3242166900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":3242166900},{"type":"v1","addr":"172.21.15.167:6803","nonce":3242166900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6809","nonce":3242166900},{"type":"v1","addr":"172.21.15.167:6811","nonce":3242166900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6805","nonce":3242166900},{"type":"v1","addr":"172.21.15.167:6807","nonce":3242166900}]},"public_addr":"172.21.15.167:6801/3242166900","cluster_addr":"172.21.15.167:6803/3242166900","heartbeat_back_addr":"172.21.15.167:6811/3242166900","heartbeat_front_addr":"172.21.15.167:6807/3242166900","state":["exists","up"]},{"osd":3,"uuid":"9dd26759-a338-49d2-8103-45d068b9c000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":453689293},{"type":"v1","addr":"172.21.15.167:6825","nonce":453689293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":453689293},{"type":"v1","addr":"172.21.15.167:6827","nonce":453689293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":453689293},{"type":"v1","addr":"172.21.15.167:6831","nonce":453689293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":453689293},{"type":"v1","addr":"172.21.15.167:6829","nonce":453689293}]},"public_addr":"172.21.15.167:6825/453689293","cluster_addr":"172.21.15.167:6827/453689293","heartbeat_back_addr":"172.21.15.167:6831/453689293","heartbeat_front_addr":"172.21.15.167:6829/453689293","state":["exists","up"]},{"osd":4,"uuid":"cb27250d-2a80-4c7b-8cef-3a7af05c8a16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1311831500},{"type":"v1","addr":"172.21.15.204:6801","nonce":1311831500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1311831500},{"type":"v1","addr":"172.21.15.204:6803","nonce":1311831500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1311831500},{"type":"v1","addr":"172.21.15.204:6807","nonce":1311831500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1311831500},{"type":"v1","addr":"172.21.15.204:6805","nonce":1311831500}]},"public_addr":"172.21.15.204:6801/1311831500","cluster_addr":"172.21.15.204:6803/1311831500","heartbeat_back_addr":"172.21.15.204:6807/1311831500","heartbeat_front_addr":"172.21.15.204:6805/1311831500","state":["exists","up"]},{"osd":5,"uuid":"92998541-ffec-430b-ab76-8c3cb7c2752d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2921847521},{"type":"v1","addr":"172.21.15.204:6825","nonce":2921847521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2921847521},{"type":"v1","addr":"172.21.15.204:6827","nonce":2921847521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2921847521},{"type":"v1","addr":"172.21.15.204:6831","nonce":2921847521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2921847521},{"type":"v1","addr":"172.21.15.204:6829","nonce":2921847521}]},"public_addr":"172.21.15.204:6825/2921847521","cluster_addr":"172.21.15.204:6827/2921847521","heartbeat_back_addr":"172.21.15.204:6831/2921847521","heartbeat_front_addr":"172.21.15.204:6829/2921847521","state":["exists","up"]},{"osd":6,"uuid":"9543479c-2ec5-42a8-8933-707bfed2e048","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":2889851077},{"type":"v1","addr":"172.21.15.204:6817","nonce":2889851077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":2889851077},{"type":"v1","addr":"172.21.15.204:6819","nonce":2889851077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":2889851077},{"type":"v1","addr":"172.21.15.204:6823","nonce":2889851077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":2889851077},{"type":"v1","addr":"172.21.15.204:6821","nonce":2889851077}]},"public_addr":"172.21.15.204:6817/2889851077","cluster_addr":"172.21.15.204:6819/2889851077","heartbeat_back_addr":"172.21.15.204:6823/2889851077","heartbeat_front_addr":"172.21.15.204:6821/2889851077","state":["exists","up"]},{"osd":7,"uuid":"c45a5647-8358-409e-af91-c05dda65f0b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":1949534184},{"type":"v1","addr":"172.21.15.204:6809","nonce":1949534184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":1949534184},{"type":"v1","addr":"172.21.15.204:6811","nonce":1949534184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1949534184},{"type":"v1","addr":"172.21.15.204:6815","nonce":1949534184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1949534184},{"type":"v1","addr":"172.21.15.204:6813","nonce":1949534184}]},"public_addr":"172.21.15.204:6809/1949534184","cluster_addr":"172.21.15.204:6811/1949534184","heartbeat_back_addr":"172.21.15.204:6815/1949534184","heartbeat_front_addr":"172.21.15.204:6813/1949534184","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.952253+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.940689+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.860682+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.929350+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.901627+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.957451+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.987249+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.951197+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-11T08:07:09.794 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.793+0000 7efe9f66d640 1 -- 172.21.15.167:0/3280687501 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7efe840762d0 msgr2=0x7efe84078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:09.794 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.793+0000 7efe9f66d640 1 --2- 172.21.15.167:0/3280687501 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7efe840762d0 0x7efe84078790 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7efe8c034000 tx=0x7efe8c02e000 comp rx=0 tx=0).stop 2024-05-11T08:07:09.794 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.793+0000 7efe9f66d640 1 -- 172.21.15.167:0/3280687501 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efe980cb4f0 msgr2=0x7efe980ba820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:09.794 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.793+0000 7efe9f66d640 1 --2- 172.21.15.167:0/3280687501 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efe980cb4f0 0x7efe980ba820 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7efe88001ed0 tx=0x7efe8800a430 comp rx=0 tx=0).stop 2024-05-11T08:07:09.794 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.793+0000 7efe9f66d640 1 -- 172.21.15.167:0/3280687501 shutdown_connections 2024-05-11T08:07:09.794 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.793+0000 7efe9f66d640 1 --2- 172.21.15.167:0/3280687501 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7efe840762d0 0x7efe84078790 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:09.794 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.793+0000 7efe9f66d640 1 --2- 172.21.15.167:0/3280687501 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efe981598d0 0x7efe9814aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:09.794 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.793+0000 7efe9f66d640 1 --2- 172.21.15.167:0/3280687501 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efe980d2cf0 0x7efe980cfa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:09.794 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.793+0000 7efe9f66d640 1 --2- 172.21.15.167:0/3280687501 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efe980cb4f0 0x7efe980ba820 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:09.794 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.793+0000 7efe9f66d640 1 -- 172.21.15.167:0/3280687501 >> 172.21.15.167:0/3280687501 conn(0x7efe980d1ba0 msgr2=0x7efe980b5cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:07:09.794 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.793+0000 7efe9f66d640 1 -- 172.21.15.167:0/3280687501 shutdown_connections 2024-05-11T08:07:09.794 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.793+0000 7efe9f66d640 1 -- 172.21.15.167:0/3280687501 wait complete. 2024-05-11T08:07:09.805 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-11T08:07:09.805 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-11T08:07:09.905 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.905+0000 7f950c8bd640 1 Processor -- start 2024-05-11T08:07:09.905 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.905+0000 7f950c8bd640 1 -- start start 2024-05-11T08:07:09.907 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.906+0000 7f950c8bd640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f950813eeb0 0x7f950806f480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:09.907 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.906+0000 7f950c8bd640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f950813f290 0x7f950806f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:09.907 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.906+0000 7f950c8bd640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f95080b50a0 0x7f95080735c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:09.907 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.906+0000 7f950c8bd640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f950815fb90 con 0x7f950813f290 2024-05-11T08:07:09.907 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.906+0000 7f950c8bd640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f950815fe70 con 0x7f950813eeb0 2024-05-11T08:07:09.907 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.906+0000 7f950c8bd640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f9508160150 con 0x7f95080b50a0 2024-05-11T08:07:09.907 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.906+0000 7f9505d9b640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f950813f290 0x7f950806f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:09.907 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.906+0000 7f9505d9b640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f950813f290 0x7f950806f9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:55882/0 (socket says 172.21.15.167:55882) 2024-05-11T08:07:09.908 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.906+0000 7f9505d9b640 1 -- 172.21.15.167:0/2363342179 learned_addr learned my addr 172.21.15.167:0/2363342179 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:07:09.908 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.906+0000 7f950659c640 1 --2- 172.21.15.167:0/2363342179 >> v2:172.21.15.204:3300/0 conn(0x7f950813eeb0 0x7f950806f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:09.908 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.906+0000 7f9506d9d640 1 --2- 172.21.15.167:0/2363342179 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f95080b50a0 0x7f95080735c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:09.908 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.906+0000 7f9505d9b640 1 -- 172.21.15.167:0/2363342179 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f95080b50a0 msgr2=0x7f95080735c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:07:09.908 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.906+0000 7f9505d9b640 1 --2- 172.21.15.167:0/2363342179 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f95080b50a0 0x7f95080735c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:09.908 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.906+0000 7f9505d9b640 1 -- 172.21.15.167:0/2363342179 >> v2:172.21.15.204:3300/0 conn(0x7f950813eeb0 msgr2=0x7f950806f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:07:09.908 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.906+0000 7f9505d9b640 1 --2- 172.21.15.167:0/2363342179 >> v2:172.21.15.204:3300/0 conn(0x7f950813eeb0 0x7f950806f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:09.908 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.906+0000 7f9505d9b640 1 -- 172.21.15.167:0/2363342179 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9508160430 con 0x7f950813f290 2024-05-11T08:07:09.908 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.906+0000 7f950659c640 1 --2- 172.21.15.167:0/2363342179 >> v2:172.21.15.204:3300/0 conn(0x7f950813eeb0 0x7f950806f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T08:07:09.908 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.906+0000 7f9506d9d640 1 --2- 172.21.15.167:0/2363342179 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f95080b50a0 0x7f95080735c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T08:07:09.908 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.907+0000 7f9505d9b640 1 --2- 172.21.15.167:0/2363342179 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f950813f290 0x7f950806f9c0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f95080b6590 tx=0x7f94fc02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1cf3f8c137059c2 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:09.908 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.907+0000 7f950559a640 1 -- 172.21.15.167:0/2363342179 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94fc037070 con 0x7f950813f290 2024-05-11T08:07:09.908 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.907+0000 7f950559a640 1 -- 172.21.15.167:0/2363342179 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94fc032440 con 0x7f950813f290 2024-05-11T08:07:09.908 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.907+0000 7f950559a640 1 -- 172.21.15.167:0/2363342179 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94fc032810 con 0x7f950813f290 2024-05-11T08:07:09.911 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.908+0000 7f950c8bd640 1 -- 172.21.15.167:0/2363342179 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f950813f290 msgr2=0x7f950806f9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:09.911 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.908+0000 7f950c8bd640 1 --2- 172.21.15.167:0/2363342179 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f950813f290 0x7f950806f9c0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f95080b6590 tx=0x7f94fc02a140 comp rx=0 tx=0).stop 2024-05-11T08:07:09.911 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.908+0000 7f950c8bd640 1 -- 172.21.15.167:0/2363342179 shutdown_connections 2024-05-11T08:07:09.911 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.908+0000 7f950c8bd640 1 --2- 172.21.15.167:0/2363342179 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f95080b50a0 0x7f95080735c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:09.912 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.908+0000 7f950c8bd640 1 --2- 172.21.15.167:0/2363342179 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f950813f290 0x7f950806f9c0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:09.912 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.908+0000 7f950c8bd640 1 --2- 172.21.15.167:0/2363342179 >> v2:172.21.15.204:3300/0 conn(0x7f950813eeb0 0x7f950806f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:09.912 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.908+0000 7f950c8bd640 1 -- 172.21.15.167:0/2363342179 >> 172.21.15.167:0/2363342179 conn(0x7f95080cd8f0 msgr2=0x7f95080cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:07:09.912 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.908+0000 7f950c8bd640 1 -- 172.21.15.167:0/2363342179 shutdown_connections 2024-05-11T08:07:09.912 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.908+0000 7f950c8bd640 1 -- 172.21.15.167:0/2363342179 wait complete. 2024-05-11T08:07:09.912 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.909+0000 7f950c8bd640 1 Processor -- start 2024-05-11T08:07:09.912 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.909+0000 7f950c8bd640 1 -- start start 2024-05-11T08:07:09.912 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.910+0000 7f950c8bd640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f95080b50a0 0x7f950815d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:09.912 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.910+0000 7f950c8bd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f950813eeb0 0x7f950815edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:09.912 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.910+0000 7f950c8bd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f950813f290 0x7f95080721e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:09.912 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.910+0000 7f950c8bd640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9508073c80 con 0x7f950813eeb0 2024-05-11T08:07:09.912 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.910+0000 7f950c8bd640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f9508073b00 con 0x7f950813f290 2024-05-11T08:07:09.912 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.910+0000 7f950c8bd640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f9508073e00 con 0x7f95080b50a0 2024-05-11T08:07:09.912 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.910+0000 7f9505d9b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f950813eeb0 0x7f950815edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:09.912 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.910+0000 7f9505d9b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f950813eeb0 0x7f950815edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:55898/0 (socket says 172.21.15.167:55898) 2024-05-11T08:07:09.913 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.910+0000 7f9505d9b640 1 -- 172.21.15.167:0/2381980805 learned_addr learned my addr 172.21.15.167:0/2381980805 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:07:09.913 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.910+0000 7f9506d9d640 1 --2- 172.21.15.167:0/2381980805 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f950813f290 0x7f95080721e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:09.913 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.910+0000 7f9505d9b640 1 -- 172.21.15.167:0/2381980805 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f95080b50a0 msgr2=0x7f950815d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:09.913 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.910+0000 7f950659c640 1 --2- 172.21.15.167:0/2381980805 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f95080b50a0 0x7f950815d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:09.913 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.910+0000 7f9505d9b640 1 --2- 172.21.15.167:0/2381980805 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f95080b50a0 0x7f950815d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:09.913 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.910+0000 7f9505d9b640 1 -- 172.21.15.167:0/2381980805 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f950813f290 msgr2=0x7f95080721e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:09.913 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.910+0000 7f9505d9b640 1 --2- 172.21.15.167:0/2381980805 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f950813f290 0x7f95080721e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:09.913 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.910+0000 7f9505d9b640 1 -- 172.21.15.167:0/2381980805 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95081f6670 con 0x7f950813eeb0 2024-05-11T08:07:09.913 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.910+0000 7f9505d9b640 1 --2- 172.21.15.167:0/2381980805 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f950813eeb0 0x7f950815edf0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f94fc02a180 tx=0x7f94fc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:09.913 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.911+0000 7f94f6ffd640 1 -- 172.21.15.167:0/2381980805 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94fc041020 con 0x7f950813eeb0 2024-05-11T08:07:09.913 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.911+0000 7f94f6ffd640 1 -- 172.21.15.167:0/2381980805 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94fc0306c0 con 0x7f950813eeb0 2024-05-11T08:07:09.913 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.911+0000 7f950c8bd640 1 -- 172.21.15.167:0/2381980805 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95081f6950 con 0x7f950813eeb0 2024-05-11T08:07:09.913 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.911+0000 7f94f6ffd640 1 -- 172.21.15.167:0/2381980805 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94fc037040 con 0x7f950813eeb0 2024-05-11T08:07:09.913 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.911+0000 7f950c8bd640 1 -- 172.21.15.167:0/2381980805 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95081f6f10 con 0x7f950813eeb0 2024-05-11T08:07:09.913 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.912+0000 7f950c8bd640 1 -- 172.21.15.167:0/2381980805 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9508069590 con 0x7f950813eeb0 2024-05-11T08:07:09.914 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.912+0000 7f94f6ffd640 1 -- 172.21.15.167:0/2381980805 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f94fc044020 con 0x7f950813eeb0 2024-05-11T08:07:09.914 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.913+0000 7f94f6ffd640 1 --2- 172.21.15.167:0/2381980805 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f94e80762d0 0x7f94e8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:09.914 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.913+0000 7f950659c640 1 --2- 172.21.15.167:0/2381980805 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f94e80762d0 0x7f94e8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:09.914 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.913+0000 7f94f6ffd640 1 -- 172.21.15.167:0/2381980805 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 6496+0+0 (secure 0 0 0) 0x7f94fc0b8060 con 0x7f950813eeb0 2024-05-11T08:07:09.915 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.913+0000 7f950659c640 1 --2- 172.21.15.167:0/2381980805 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f94e80762d0 0x7f94e8078790 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f94f8006fd0 tx=0x7f94f8008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:09.917 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:09.917+0000 7f94f6ffd640 1 -- 172.21.15.167:0/2381980805 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f94fc04b030 con 0x7f950813eeb0 2024-05-11T08:07:10.133 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.133+0000 7f950c8bd640 1 -- 172.21.15.167:0/2381980805 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f950806c6a0 con 0x7f950813eeb0 2024-05-11T08:07:10.134 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.134+0000 7f94f6ffd640 1 -- 172.21.15.167:0/2381980805 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5192 (secure 0 0 0) 0x7f94fc081740 con 0x7f950813eeb0 2024-05-11T08:07:10.134 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T08:07:10.135 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4604,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2674983673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":2674983673},{"type":"v1","addr":"172.21.15.167:6837","nonce":2674983673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4607,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6834/1018548066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1018548066},{"type":"v1","addr":"172.21.15.204:6834","nonce":1018548066}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4729,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3369722249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3369722249},{"type":"v1","addr":"172.21.15.167:6836","nonce":3369722249}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-11T08:06:47.943939+0000","modified":"2024-05-11T08:07:04.006894+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4750},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4750":{"gid":4750,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.204:6835/469864006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6833","nonce":469864006},{"type":"v1","addr":"172.21.15.204:6835","nonce":469864006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6,7],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4750,"qdb_cluster":[4750]},"id":2}]} 2024-05-11T08:07:10.135 INFO:teuthology.orchestra.run.smithi167.stderr:dumped fsmap epoch 17 2024-05-11T08:07:10.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.136+0000 7f950c8bd640 1 -- 172.21.15.167:0/2381980805 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f94e80762d0 msgr2=0x7f94e8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:10.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.136+0000 7f950c8bd640 1 --2- 172.21.15.167:0/2381980805 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f94e80762d0 0x7f94e8078790 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f94f8006fd0 tx=0x7f94f8008040 comp rx=0 tx=0).stop 2024-05-11T08:07:10.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.136+0000 7f950c8bd640 1 -- 172.21.15.167:0/2381980805 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f950813eeb0 msgr2=0x7f950815edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:10.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.136+0000 7f950c8bd640 1 --2- 172.21.15.167:0/2381980805 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f950813eeb0 0x7f950815edf0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f94fc02a180 tx=0x7f94fc0046c0 comp rx=0 tx=0).stop 2024-05-11T08:07:10.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.136+0000 7f950c8bd640 1 -- 172.21.15.167:0/2381980805 shutdown_connections 2024-05-11T08:07:10.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.136+0000 7f950c8bd640 1 --2- 172.21.15.167:0/2381980805 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f94e80762d0 0x7f94e8078790 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:10.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.136+0000 7f950c8bd640 1 --2- 172.21.15.167:0/2381980805 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f950813f290 0x7f95080721e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:10.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.136+0000 7f950c8bd640 1 --2- 172.21.15.167:0/2381980805 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f950813eeb0 0x7f950815edf0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:10.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.136+0000 7f950c8bd640 1 --2- 172.21.15.167:0/2381980805 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f95080b50a0 0x7f950815d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:10.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.136+0000 7f950c8bd640 1 -- 172.21.15.167:0/2381980805 >> 172.21.15.167:0/2381980805 conn(0x7f95080cd8f0 msgr2=0x7f95080b22d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:07:10.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.136+0000 7f950c8bd640 1 -- 172.21.15.167:0/2381980805 shutdown_connections 2024-05-11T08:07:10.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.136+0000 7f950c8bd640 1 -- 172.21.15.167:0/2381980805 wait complete. 2024-05-11T08:07:10.147 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-11T08:07:10.248 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.248+0000 7f191a0da640 1 Processor -- start 2024-05-11T08:07:10.248 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.248+0000 7f191a0da640 1 -- start start 2024-05-11T08:07:10.249 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.249+0000 7f191a0da640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f19141598d0 0x7f19140d2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:10.249 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.249+0000 7f191a0da640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19140d1190 con 0x7f19140cb6c0 2024-05-11T08:07:10.249 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.249+0000 7f191a0da640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f191406d500 con 0x7f19140d0640 2024-05-11T08:07:10.249 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.249+0000 7f191a0da640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f191406d7e0 con 0x7f19141598d0 2024-05-11T08:07:10.249 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.249+0000 7f19137fe640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f19141598d0 0x7f19140d2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:10.250 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.249+0000 7f1913fff640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f19140cb6c0 0x7f191406dfe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:47444/0 (socket says 172.21.15.167:47444) 2024-05-11T08:07:10.250 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.249+0000 7f1913fff640 1 -- 172.21.15.167:0/1929365022 learned_addr learned my addr 172.21.15.167:0/1929365022 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T08:07:10.250 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.249+0000 7f19137fe640 1 -- 172.21.15.167:0/1929365022 >> v1:172.21.15.204:6789/0 conn(0x7f19140d0640 legacy=0x7f19140cafb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:07:10.250 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.249+0000 7f19137fe640 1 -- 172.21.15.167:0/1929365022 >> v1:172.21.15.167:6789/0 conn(0x7f19140cb6c0 legacy=0x7f191406dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:10.250 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.249+0000 7f19137fe640 1 -- 172.21.15.167:0/1929365022 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f191406dac0 con 0x7f19141598d0 2024-05-11T08:07:10.250 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.249+0000 7f19127fc640 1 -- 172.21.15.167:0/1929365022 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1082603563 0 0) 0x7f19140d1190 con 0x7f19140cb6c0 2024-05-11T08:07:10.251 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.250+0000 7f19137fe640 1 --2- 172.21.15.167:0/1929365022 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f19141598d0 0x7f19140d2cf0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f19040097a0 tx=0x7f190402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9475c3512d20e5 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:10.251 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.250+0000 7f19127fc640 1 -- 172.21.15.167:0/1929365022 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1904037070 con 0x7f19141598d0 2024-05-11T08:07:10.251 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.250+0000 7f19127fc640 1 -- 172.21.15.167:0/1929365022 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1904032440 con 0x7f19141598d0 2024-05-11T08:07:10.251 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.250+0000 7f19127fc640 1 -- 172.21.15.167:0/1929365022 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1904032810 con 0x7f19141598d0 2024-05-11T08:07:10.251 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.250+0000 7f191a0da640 1 -- 172.21.15.167:0/1929365022 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f19141598d0 msgr2=0x7f19140d2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:10.252 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.250+0000 7f191a0da640 1 --2- 172.21.15.167:0/1929365022 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f19141598d0 0x7f19140d2cf0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f19040097a0 tx=0x7f190402a180 comp rx=0 tx=0).stop 2024-05-11T08:07:10.252 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.250+0000 7f191a0da640 1 -- 172.21.15.167:0/1929365022 shutdown_connections 2024-05-11T08:07:10.252 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.250+0000 7f191a0da640 1 --2- 172.21.15.167:0/1929365022 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f19141598d0 0x7f19140d2cf0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:10.252 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.251+0000 7f191a0da640 1 -- 172.21.15.167:0/1929365022 >> 172.21.15.167:0/1929365022 conn(0x7f19140d1ba0 msgr2=0x7f19140d1fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:07:10.252 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.251+0000 7f191a0da640 1 -- 172.21.15.167:0/1929365022 shutdown_connections 2024-05-11T08:07:10.252 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.251+0000 7f191a0da640 1 -- 172.21.15.167:0/1929365022 wait complete. 2024-05-11T08:07:10.254 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.252+0000 7f191a0da640 1 Processor -- start 2024-05-11T08:07:10.254 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.252+0000 7f191a0da640 1 -- start start 2024-05-11T08:07:10.254 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.253+0000 7f191a0da640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f19140cb6c0 0x7f191414e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:10.254 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.253+0000 7f191a0da640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f19140d0640 0x7f19141f5340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:10.254 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.253+0000 7f191a0da640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f19141598d0 0x7f19141f5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:10.255 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.253+0000 7f191a0da640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f191406d1b0 con 0x7f19140d0640 2024-05-11T08:07:10.255 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.253+0000 7f191a0da640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1914159cb0 con 0x7f19141598d0 2024-05-11T08:07:10.255 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.253+0000 7f191a0da640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f19140d0ad0 con 0x7f19140cb6c0 2024-05-11T08:07:10.255 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.253+0000 7f1912ffd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f19140d0640 0x7f19141f5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:10.255 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.253+0000 7f1912ffd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f19140d0640 0x7f19141f5340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:55906/0 (socket says 172.21.15.167:55906) 2024-05-11T08:07:10.255 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.253+0000 7f1912ffd640 1 -- 172.21.15.167:0/3845928727 learned_addr learned my addr 172.21.15.167:0/3845928727 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:07:10.255 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.253+0000 7f1913fff640 1 --2- 172.21.15.167:0/3845928727 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f19141598d0 0x7f19141f5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:10.255 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.253+0000 7f19137fe640 1 --2- 172.21.15.167:0/3845928727 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f19140cb6c0 0x7f191414e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:10.255 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.253+0000 7f1912ffd640 1 -- 172.21.15.167:0/3845928727 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f19140cb6c0 msgr2=0x7f191414e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:10.255 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.253+0000 7f1912ffd640 1 --2- 172.21.15.167:0/3845928727 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f19140cb6c0 0x7f191414e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:10.255 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.253+0000 7f1912ffd640 1 -- 172.21.15.167:0/3845928727 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f19141598d0 msgr2=0x7f19141f5880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:10.255 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.253+0000 7f1912ffd640 1 --2- 172.21.15.167:0/3845928727 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f19141598d0 0x7f19141f5880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:10.255 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.253+0000 7f1912ffd640 1 -- 172.21.15.167:0/3845928727 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19141f6150 con 0x7f19140d0640 2024-05-11T08:07:10.255 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.253+0000 7f1913fff640 1 --2- 172.21.15.167:0/3845928727 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f19141598d0 0x7f19141f5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T08:07:10.256 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.253+0000 7f1912ffd640 1 --2- 172.21.15.167:0/3845928727 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f19140d0640 0x7f19141f5340 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f190800c410 tx=0x7f190800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:10.256 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.253+0000 7f18fbfff640 1 -- 172.21.15.167:0/3845928727 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1908017070 con 0x7f19140d0640 2024-05-11T08:07:10.256 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.253+0000 7f18fbfff640 1 -- 172.21.15.167:0/3845928727 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19080035f0 con 0x7f19140d0640 2024-05-11T08:07:10.256 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.253+0000 7f18fbfff640 1 -- 172.21.15.167:0/3845928727 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1908003a20 con 0x7f19140d0640 2024-05-11T08:07:10.256 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.253+0000 7f191a0da640 1 -- 172.21.15.167:0/3845928727 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19141f6430 con 0x7f19140d0640 2024-05-11T08:07:10.256 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.253+0000 7f191a0da640 1 -- 172.21.15.167:0/3845928727 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19141f69f0 con 0x7f19140d0640 2024-05-11T08:07:10.256 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.255+0000 7f191a0da640 1 -- 172.21.15.167:0/3845928727 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19140ccca0 con 0x7f19140d0640 2024-05-11T08:07:10.257 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.255+0000 7f18fbfff640 1 -- 172.21.15.167:0/3845928727 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f190801a070 con 0x7f19140d0640 2024-05-11T08:07:10.257 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.255+0000 7f18fbfff640 1 --2- 172.21.15.167:0/3845928727 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f19000762d0 0x7f1900078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:10.257 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.256+0000 7f19137fe640 1 --2- 172.21.15.167:0/3845928727 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f19000762d0 0x7f1900078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:10.257 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.256+0000 7f18fbfff640 1 -- 172.21.15.167:0/3845928727 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 6496+0+0 (secure 0 0 0) 0x7f1908096d40 con 0x7f19140d0640 2024-05-11T08:07:10.257 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.256+0000 7f19137fe640 1 --2- 172.21.15.167:0/3845928727 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f19000762d0 0x7f1900078790 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f1904034000 tx=0x7f190402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:10.260 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.260+0000 7f18fbfff640 1 -- 172.21.15.167:0/3845928727 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f1908060420 con 0x7f19140d0640 2024-05-11T08:07:10.479 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.480+0000 7f191a0da640 1 -- 172.21.15.167:0/3845928727 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1914069590 con 0x7f19140d0640 2024-05-11T08:07:10.480 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.480+0000 7f18fbfff640 1 -- 172.21.15.167:0/3845928727 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5192 (secure 0 0 0) 0x7f190805fd20 con 0x7f19140d0640 2024-05-11T08:07:10.480 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T08:07:10.480 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4604,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2674983673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":2674983673},{"type":"v1","addr":"172.21.15.167:6837","nonce":2674983673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4607,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6834/1018548066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1018548066},{"type":"v1","addr":"172.21.15.204:6834","nonce":1018548066}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4729,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3369722249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3369722249},{"type":"v1","addr":"172.21.15.167:6836","nonce":3369722249}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-11T08:06:47.943939+0000","modified":"2024-05-11T08:07:04.006894+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4750},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4750":{"gid":4750,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.204:6835/469864006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6833","nonce":469864006},{"type":"v1","addr":"172.21.15.204:6835","nonce":469864006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6,7],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4750,"qdb_cluster":[4750]},"id":2}]} 2024-05-11T08:07:10.481 INFO:teuthology.orchestra.run.smithi167.stderr:dumped fsmap epoch 17 2024-05-11T08:07:10.482 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.482+0000 7f191a0da640 1 -- 172.21.15.167:0/3845928727 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f19000762d0 msgr2=0x7f1900078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:10.482 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.482+0000 7f191a0da640 1 --2- 172.21.15.167:0/3845928727 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f19000762d0 0x7f1900078790 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f1904034000 tx=0x7f190402e000 comp rx=0 tx=0).stop 2024-05-11T08:07:10.483 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.482+0000 7f191a0da640 1 -- 172.21.15.167:0/3845928727 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f19140d0640 msgr2=0x7f19141f5340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:10.483 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.482+0000 7f191a0da640 1 --2- 172.21.15.167:0/3845928727 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f19140d0640 0x7f19141f5340 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f190800c410 tx=0x7f190800c8d0 comp rx=0 tx=0).stop 2024-05-11T08:07:10.483 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.482+0000 7f191a0da640 1 -- 172.21.15.167:0/3845928727 shutdown_connections 2024-05-11T08:07:10.483 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.482+0000 7f191a0da640 1 --2- 172.21.15.167:0/3845928727 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f19000762d0 0x7f1900078790 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:10.483 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.482+0000 7f191a0da640 1 --2- 172.21.15.167:0/3845928727 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f19141598d0 0x7f19141f5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:10.483 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.482+0000 7f191a0da640 1 --2- 172.21.15.167:0/3845928727 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f19140d0640 0x7f19141f5340 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:10.483 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.482+0000 7f191a0da640 1 --2- 172.21.15.167:0/3845928727 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f19140cb6c0 0x7f191414e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:10.483 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.482+0000 7f191a0da640 1 -- 172.21.15.167:0/3845928727 >> 172.21.15.167:0/3845928727 conn(0x7f19140d1ba0 msgr2=0x7f19140c9fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:07:10.483 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.482+0000 7f191a0da640 1 -- 172.21.15.167:0/3845928727 shutdown_connections 2024-05-11T08:07:10.483 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.482+0000 7f191a0da640 1 -- 172.21.15.167:0/3845928727 wait complete. 2024-05-11T08:07:10.494 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-11T08:07:10.593 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.593+0000 7ff128d33640 1 Processor -- start 2024-05-11T08:07:10.593 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.593+0000 7ff128d33640 1 -- start start 2024-05-11T08:07:10.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.594+0000 7ff128d33640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7ff12415f330 0x7ff1240b6860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:10.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.594+0000 7ff128d33640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff1240737a0 0x7ff1240d0080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:10.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.594+0000 7ff128d33640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff12406c930 con 0x7ff12406c450 2024-05-11T08:07:10.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.594+0000 7ff128d33640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7ff12406cdd0 con 0x7ff12415f330 2024-05-11T08:07:10.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.594+0000 7ff128d33640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7ff12406d0b0 con 0x7ff1240737a0 2024-05-11T08:07:10.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.594+0000 7ff12259c640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7ff12415f330 0x7ff1240b6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:10.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.594+0000 7ff12259c640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7ff12415f330 0x7ff1240b6860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:55916/0 (socket says 172.21.15.167:55916) 2024-05-11T08:07:10.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.594+0000 7ff11bfff640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7ff12406c450 0x7ff124073090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:47456/0 (socket says 172.21.15.167:47456) 2024-05-11T08:07:10.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.594+0000 7ff12259c640 1 -- 172.21.15.167:0/1431142179 learned_addr learned my addr 172.21.15.167:0/1431142179 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:07:10.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.594+0000 7ff122d9d640 1 --2- 172.21.15.167:0/1431142179 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff1240737a0 0x7ff1240d0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:10.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.594+0000 7ff12259c640 1 -- 172.21.15.167:0/1431142179 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff1240737a0 msgr2=0x7ff1240d0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:07:10.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.594+0000 7ff12259c640 1 --2- 172.21.15.167:0/1431142179 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff1240737a0 0x7ff1240d0080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:10.597 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.594+0000 7ff12259c640 1 -- 172.21.15.167:0/1431142179 >> v1:172.21.15.167:6789/0 conn(0x7ff12406c450 legacy=0x7ff124073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:10.597 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.594+0000 7ff12259c640 1 -- 172.21.15.167:0/1431142179 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff12406d390 con 0x7ff12415f330 2024-05-11T08:07:10.597 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.594+0000 7ff121d9b640 1 -- 172.21.15.167:0/1431142179 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1687266480 0 0) 0x7ff12406c930 con 0x7ff12406c450 2024-05-11T08:07:10.597 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.595+0000 7ff12259c640 1 --2- 172.21.15.167:0/1431142179 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff12415f330 0x7ff1240b6860 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7ff114009870 tx=0x7ff11402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e291a55405bbdce server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:10.597 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.595+0000 7ff121d9b640 1 -- 172.21.15.167:0/1431142179 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff114037070 con 0x7ff12415f330 2024-05-11T08:07:10.597 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.595+0000 7ff121d9b640 1 -- 172.21.15.167:0/1431142179 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff114032440 con 0x7ff12415f330 2024-05-11T08:07:10.597 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.595+0000 7ff121d9b640 1 -- 172.21.15.167:0/1431142179 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff114032810 con 0x7ff12415f330 2024-05-11T08:07:10.597 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.596+0000 7ff128d33640 1 -- 172.21.15.167:0/1431142179 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff12415f330 msgr2=0x7ff1240b6860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:10.597 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.596+0000 7ff128d33640 1 --2- 172.21.15.167:0/1431142179 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff12415f330 0x7ff1240b6860 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7ff114009870 tx=0x7ff11402a270 comp rx=0 tx=0).stop 2024-05-11T08:07:10.597 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.596+0000 7ff128d33640 1 -- 172.21.15.167:0/1431142179 shutdown_connections 2024-05-11T08:07:10.597 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.596+0000 7ff128d33640 1 --2- 172.21.15.167:0/1431142179 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff1240737a0 0x7ff1240d0080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:10.597 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.596+0000 7ff128d33640 1 --2- 172.21.15.167:0/1431142179 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff12415f330 0x7ff1240b6860 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:10.597 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.596+0000 7ff128d33640 1 -- 172.21.15.167:0/1431142179 >> 172.21.15.167:0/1431142179 conn(0x7ff124070290 msgr2=0x7ff124070670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:07:10.597 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.596+0000 7ff128d33640 1 -- 172.21.15.167:0/1431142179 shutdown_connections 2024-05-11T08:07:10.597 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.596+0000 7ff128d33640 1 -- 172.21.15.167:0/1431142179 wait complete. 2024-05-11T08:07:10.600 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.597+0000 7ff128d33640 1 Processor -- start 2024-05-11T08:07:10.600 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.597+0000 7ff128d33640 1 -- start start 2024-05-11T08:07:10.600 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.598+0000 7ff128d33640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff12406c450 0x7ff1240b0290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:10.600 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.598+0000 7ff128d33640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff1240737a0 0x7ff12414ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:10.600 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.598+0000 7ff128d33640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff12415f330 0x7ff12414b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:10.600 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.598+0000 7ff128d33640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff124073f00 con 0x7ff1240737a0 2024-05-11T08:07:10.600 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.598+0000 7ff128d33640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff124073d80 con 0x7ff12406c450 2024-05-11T08:07:10.600 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.598+0000 7ff128d33640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7ff124074740 con 0x7ff12415f330 2024-05-11T08:07:10.600 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.598+0000 7ff11bfff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff1240737a0 0x7ff12414ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:10.600 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.598+0000 7ff11bfff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff1240737a0 0x7ff12414ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:55930/0 (socket says 172.21.15.167:55930) 2024-05-11T08:07:10.600 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.598+0000 7ff11bfff640 1 -- 172.21.15.167:0/3979842878 learned_addr learned my addr 172.21.15.167:0/3979842878 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:07:10.600 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.598+0000 7ff12259c640 1 --2- 172.21.15.167:0/3979842878 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff12406c450 0x7ff1240b0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:10.601 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.598+0000 7ff122d9d640 1 --2- 172.21.15.167:0/3979842878 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff12415f330 0x7ff12414b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:10.601 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.598+0000 7ff11bfff640 1 -- 172.21.15.167:0/3979842878 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff12415f330 msgr2=0x7ff12414b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:10.601 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.598+0000 7ff11bfff640 1 --2- 172.21.15.167:0/3979842878 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff12415f330 0x7ff12414b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:10.601 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.598+0000 7ff11bfff640 1 -- 172.21.15.167:0/3979842878 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff12406c450 msgr2=0x7ff1240b0290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:10.601 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.598+0000 7ff11bfff640 1 --2- 172.21.15.167:0/3979842878 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff12406c450 0x7ff1240b0290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:10.601 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.598+0000 7ff11bfff640 1 -- 172.21.15.167:0/3979842878 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff124158050 con 0x7ff1240737a0 2024-05-11T08:07:10.601 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.598+0000 7ff11bfff640 1 --2- 172.21.15.167:0/3979842878 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff1240737a0 0x7ff12414ac50 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7ff10c00c450 tx=0x7ff10c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:10.601 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.599+0000 7ff11affd640 1 -- 172.21.15.167:0/3979842878 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff10c018070 con 0x7ff1240737a0 2024-05-11T08:07:10.601 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.599+0000 7ff11affd640 1 -- 172.21.15.167:0/3979842878 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff10c007480 con 0x7ff1240737a0 2024-05-11T08:07:10.601 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.599+0000 7ff11affd640 1 -- 172.21.15.167:0/3979842878 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff10c0047e0 con 0x7ff1240737a0 2024-05-11T08:07:10.601 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.599+0000 7ff128d33640 1 -- 172.21.15.167:0/3979842878 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff12415f9c0 con 0x7ff1240737a0 2024-05-11T08:07:10.601 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.599+0000 7ff128d33640 1 -- 172.21.15.167:0/3979842878 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff12415ff80 con 0x7ff1240737a0 2024-05-11T08:07:10.601 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.600+0000 7ff128d33640 1 -- 172.21.15.167:0/3979842878 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff124069590 con 0x7ff1240737a0 2024-05-11T08:07:10.602 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.600+0000 7ff11affd640 1 -- 172.21.15.167:0/3979842878 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7ff10c01b070 con 0x7ff1240737a0 2024-05-11T08:07:10.602 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.601+0000 7ff11affd640 1 --2- 172.21.15.167:0/3979842878 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7ff1040762d0 0x7ff104078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:10.602 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.601+0000 7ff11affd640 1 -- 172.21.15.167:0/3979842878 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 6496+0+0 (secure 0 0 0) 0x7ff10c096ae0 con 0x7ff1240737a0 2024-05-11T08:07:10.602 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.601+0000 7ff12259c640 1 --2- 172.21.15.167:0/3979842878 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7ff1040762d0 0x7ff104078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:10.602 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.601+0000 7ff12259c640 1 --2- 172.21.15.167:0/3979842878 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7ff1040762d0 0x7ff104078790 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ff114034000 tx=0x7ff11402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:10.605 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.605+0000 7ff11affd640 1 -- 172.21.15.167:0/3979842878 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7ff10c05fae0 con 0x7ff1240737a0 2024-05-11T08:07:10.788 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.788+0000 7ff128d33640 1 -- 172.21.15.167:0/3979842878 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff1240d0d10 con 0x7ff1240737a0 2024-05-11T08:07:10.789 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.790+0000 7ff11affd640 1 -- 172.21.15.167:0/3979842878 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) ==== 74+0+18802 (secure 0 0 0) 0x7ff10c068120 con 0x7ff1240737a0 2024-05-11T08:07:10.790 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T08:07:10.790 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":44,"fsid":"2034778d-8ff1-4823-aa44-ded32b518241","created":"2024-05-11T08:05:31.612421+0000","modified":"2024-05-11T08:07:05.656974+0000","last_up_change":"2024-05-11T08:05:42.557382+0000","last_in_change":"2024-05-11T08:05:33.757311+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-11T08:05:43.931263+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-05-11T08:06:45.857156+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-05-11T08:06:46.898591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"data_new","create_time":"2024-05-11T08:07:03.541639+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"587eaced-5ded-406d-884b-30c1caa8686b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":3863307356},{"type":"v1","addr":"172.21.15.167:6806","nonce":3863307356}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":3863307356},{"type":"v1","addr":"172.21.15.167:6810","nonce":3863307356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":3863307356},{"type":"v1","addr":"172.21.15.167:6815","nonce":3863307356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":3863307356},{"type":"v1","addr":"172.21.15.167:6813","nonce":3863307356}]},"public_addr":"172.21.15.167:6806/3863307356","cluster_addr":"172.21.15.167:6810/3863307356","heartbeat_back_addr":"172.21.15.167:6815/3863307356","heartbeat_front_addr":"172.21.15.167:6813/3863307356","state":["exists","up"]},{"osd":1,"uuid":"37c59104-2dd0-4cf9-85ea-56636f6b02a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":1585024048},{"type":"v1","addr":"172.21.15.167:6817","nonce":1585024048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1585024048},{"type":"v1","addr":"172.21.15.167:6819","nonce":1585024048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1585024048},{"type":"v1","addr":"172.21.15.167:6823","nonce":1585024048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":1585024048},{"type":"v1","addr":"172.21.15.167:6821","nonce":1585024048}]},"public_addr":"172.21.15.167:6817/1585024048","cluster_addr":"172.21.15.167:6819/1585024048","heartbeat_back_addr":"172.21.15.167:6823/1585024048","heartbeat_front_addr":"172.21.15.167:6821/1585024048","state":["exists","up"]},{"osd":2,"uuid":"2bb4f7f5-c698-451b-b4bb-78e716921291","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":3242166900},{"type":"v1","addr":"172.21.15.167:6801","nonce":3242166900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":3242166900},{"type":"v1","addr":"172.21.15.167:6803","nonce":3242166900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6809","nonce":3242166900},{"type":"v1","addr":"172.21.15.167:6811","nonce":3242166900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6805","nonce":3242166900},{"type":"v1","addr":"172.21.15.167:6807","nonce":3242166900}]},"public_addr":"172.21.15.167:6801/3242166900","cluster_addr":"172.21.15.167:6803/3242166900","heartbeat_back_addr":"172.21.15.167:6811/3242166900","heartbeat_front_addr":"172.21.15.167:6807/3242166900","state":["exists","up"]},{"osd":3,"uuid":"9dd26759-a338-49d2-8103-45d068b9c000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":453689293},{"type":"v1","addr":"172.21.15.167:6825","nonce":453689293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":453689293},{"type":"v1","addr":"172.21.15.167:6827","nonce":453689293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":453689293},{"type":"v1","addr":"172.21.15.167:6831","nonce":453689293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":453689293},{"type":"v1","addr":"172.21.15.167:6829","nonce":453689293}]},"public_addr":"172.21.15.167:6825/453689293","cluster_addr":"172.21.15.167:6827/453689293","heartbeat_back_addr":"172.21.15.167:6831/453689293","heartbeat_front_addr":"172.21.15.167:6829/453689293","state":["exists","up"]},{"osd":4,"uuid":"cb27250d-2a80-4c7b-8cef-3a7af05c8a16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1311831500},{"type":"v1","addr":"172.21.15.204:6801","nonce":1311831500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1311831500},{"type":"v1","addr":"172.21.15.204:6803","nonce":1311831500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1311831500},{"type":"v1","addr":"172.21.15.204:6807","nonce":1311831500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1311831500},{"type":"v1","addr":"172.21.15.204:6805","nonce":1311831500}]},"public_addr":"172.21.15.204:6801/1311831500","cluster_addr":"172.21.15.204:6803/1311831500","heartbeat_back_addr":"172.21.15.204:6807/1311831500","heartbeat_front_addr":"172.21.15.204:6805/1311831500","state":["exists","up"]},{"osd":5,"uuid":"92998541-ffec-430b-ab76-8c3cb7c2752d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2921847521},{"type":"v1","addr":"172.21.15.204:6825","nonce":2921847521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2921847521},{"type":"v1","addr":"172.21.15.204:6827","nonce":2921847521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2921847521},{"type":"v1","addr":"172.21.15.204:6831","nonce":2921847521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2921847521},{"type":"v1","addr":"172.21.15.204:6829","nonce":2921847521}]},"public_addr":"172.21.15.204:6825/2921847521","cluster_addr":"172.21.15.204:6827/2921847521","heartbeat_back_addr":"172.21.15.204:6831/2921847521","heartbeat_front_addr":"172.21.15.204:6829/2921847521","state":["exists","up"]},{"osd":6,"uuid":"9543479c-2ec5-42a8-8933-707bfed2e048","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":2889851077},{"type":"v1","addr":"172.21.15.204:6817","nonce":2889851077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":2889851077},{"type":"v1","addr":"172.21.15.204:6819","nonce":2889851077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":2889851077},{"type":"v1","addr":"172.21.15.204:6823","nonce":2889851077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":2889851077},{"type":"v1","addr":"172.21.15.204:6821","nonce":2889851077}]},"public_addr":"172.21.15.204:6817/2889851077","cluster_addr":"172.21.15.204:6819/2889851077","heartbeat_back_addr":"172.21.15.204:6823/2889851077","heartbeat_front_addr":"172.21.15.204:6821/2889851077","state":["exists","up"]},{"osd":7,"uuid":"c45a5647-8358-409e-af91-c05dda65f0b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":1949534184},{"type":"v1","addr":"172.21.15.204:6809","nonce":1949534184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":1949534184},{"type":"v1","addr":"172.21.15.204:6811","nonce":1949534184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1949534184},{"type":"v1","addr":"172.21.15.204:6815","nonce":1949534184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1949534184},{"type":"v1","addr":"172.21.15.204:6813","nonce":1949534184}]},"public_addr":"172.21.15.204:6809/1949534184","cluster_addr":"172.21.15.204:6811/1949534184","heartbeat_back_addr":"172.21.15.204:6815/1949534184","heartbeat_front_addr":"172.21.15.204:6813/1949534184","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.952253+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.940689+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.860682+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.929350+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.901627+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.957451+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.987249+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.951197+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-11T08:07:10.792 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.792+0000 7ff128d33640 1 -- 172.21.15.167:0/3979842878 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7ff1040762d0 msgr2=0x7ff104078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:10.792 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.792+0000 7ff128d33640 1 --2- 172.21.15.167:0/3979842878 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7ff1040762d0 0x7ff104078790 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ff114034000 tx=0x7ff11402e000 comp rx=0 tx=0).stop 2024-05-11T08:07:10.792 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.792+0000 7ff128d33640 1 -- 172.21.15.167:0/3979842878 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff1240737a0 msgr2=0x7ff12414ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:10.792 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.792+0000 7ff128d33640 1 --2- 172.21.15.167:0/3979842878 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff1240737a0 0x7ff12414ac50 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7ff10c00c450 tx=0x7ff10c00c910 comp rx=0 tx=0).stop 2024-05-11T08:07:10.792 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.792+0000 7ff128d33640 1 -- 172.21.15.167:0/3979842878 shutdown_connections 2024-05-11T08:07:10.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.792+0000 7ff128d33640 1 --2- 172.21.15.167:0/3979842878 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7ff1040762d0 0x7ff104078790 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:10.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.792+0000 7ff128d33640 1 --2- 172.21.15.167:0/3979842878 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff12415f330 0x7ff12414b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:10.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.792+0000 7ff128d33640 1 --2- 172.21.15.167:0/3979842878 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff1240737a0 0x7ff12414ac50 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:10.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.792+0000 7ff128d33640 1 --2- 172.21.15.167:0/3979842878 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff12406c450 0x7ff1240b0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:10.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.792+0000 7ff128d33640 1 -- 172.21.15.167:0/3979842878 >> 172.21.15.167:0/3979842878 conn(0x7ff124070290 msgr2=0x7ff1240b2c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:07:10.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.792+0000 7ff128d33640 1 -- 172.21.15.167:0/3979842878 shutdown_connections 2024-05-11T08:07:10.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.792+0000 7ff128d33640 1 -- 172.21.15.167:0/3979842878 wait complete. 2024-05-11T08:07:10.803 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-11T08:07:10.904 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.904+0000 7fb1f5738640 1 Processor -- start 2024-05-11T08:07:10.905 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.904+0000 7fb1f5738640 1 -- start start 2024-05-11T08:07:10.906 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.905+0000 7fb1f5738640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb1f00cd610 0x7fb1f0163be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:10.906 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.905+0000 7fb1f5738640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fb1f00cd9f0 0x7fb1f0160230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:10.906 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.905+0000 7fb1f5738640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb1f00b6300 con 0x7fb1f0160770 2024-05-11T08:07:10.906 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.905+0000 7fb1f5738640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7fb1f00b65e0 con 0x7fb1f00cd9f0 2024-05-11T08:07:10.907 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.905+0000 7fb1f5738640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fb1f0163100 con 0x7fb1f00cd610 2024-05-11T08:07:10.907 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.905+0000 7fb1e67fc640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fb1f00cd9f0 0x7fb1f0160230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:10.907 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.905+0000 7fb1ef7fe640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7fb1f0160770 0x7fb1f00d0ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:47460/0 (socket says 172.21.15.167:47460) 2024-05-11T08:07:10.907 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.905+0000 7fb1ef7fe640 1 -- 172.21.15.167:0/3696809559 learned_addr learned my addr 172.21.15.167:0/3696809559 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T08:07:10.907 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.905+0000 7fb1eeffd640 1 --2- 172.21.15.167:0/3696809559 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb1f00cd610 0x7fb1f0163be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:10.907 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.906+0000 7fb1e67fc640 1 -- 172.21.15.167:0/3696809559 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb1f00cd610 msgr2=0x7fb1f0163be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:07:10.907 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.906+0000 7fb1e67fc640 1 --2- 172.21.15.167:0/3696809559 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb1f00cd610 0x7fb1f0163be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:10.907 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.906+0000 7fb1e67fc640 1 -- 172.21.15.167:0/3696809559 >> v1:172.21.15.167:6789/0 conn(0x7fb1f0160770 legacy=0x7fb1f00d0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:10.907 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.906+0000 7fb1e67fc640 1 -- 172.21.15.167:0/3696809559 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1f01633e0 con 0x7fb1f00cd9f0 2024-05-11T08:07:10.907 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.906+0000 7fb1ee7fc640 1 -- 172.21.15.167:0/3696809559 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1419195409 0 0) 0x7fb1f00b6300 con 0x7fb1f0160770 2024-05-11T08:07:10.907 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.906+0000 7fb1e67fc640 1 --2- 172.21.15.167:0/3696809559 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb1f00cd9f0 0x7fb1f0160230 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fb1d80097a0 tx=0x7fb1d802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b981938aedf8160 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:10.907 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.906+0000 7fb1ee7fc640 1 -- 172.21.15.167:0/3696809559 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1d8037070 con 0x7fb1f00cd9f0 2024-05-11T08:07:10.907 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.906+0000 7fb1ee7fc640 1 -- 172.21.15.167:0/3696809559 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1d8032440 con 0x7fb1f00cd9f0 2024-05-11T08:07:10.907 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.906+0000 7fb1ee7fc640 1 -- 172.21.15.167:0/3696809559 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1d8032810 con 0x7fb1f00cd9f0 2024-05-11T08:07:10.909 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.907+0000 7fb1f5738640 1 -- 172.21.15.167:0/3696809559 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb1f00cd9f0 msgr2=0x7fb1f0160230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:10.909 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.907+0000 7fb1f5738640 1 --2- 172.21.15.167:0/3696809559 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb1f00cd9f0 0x7fb1f0160230 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fb1d80097a0 tx=0x7fb1d802a180 comp rx=0 tx=0).stop 2024-05-11T08:07:10.909 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.907+0000 7fb1f5738640 1 -- 172.21.15.167:0/3696809559 shutdown_connections 2024-05-11T08:07:10.909 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.907+0000 7fb1f5738640 1 --2- 172.21.15.167:0/3696809559 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb1f00cd9f0 0x7fb1f0160230 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:10.909 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.907+0000 7fb1f5738640 1 --2- 172.21.15.167:0/3696809559 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb1f00cd610 0x7fb1f0163be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:10.909 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.907+0000 7fb1f5738640 1 -- 172.21.15.167:0/3696809559 >> 172.21.15.167:0/3696809559 conn(0x7fb1f0072730 msgr2=0x7fb1f00d2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:07:10.909 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.907+0000 7fb1f5738640 1 -- 172.21.15.167:0/3696809559 shutdown_connections 2024-05-11T08:07:10.909 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.907+0000 7fb1f5738640 1 -- 172.21.15.167:0/3696809559 wait complete. 2024-05-11T08:07:10.909 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.908+0000 7fb1f5738640 1 Processor -- start 2024-05-11T08:07:10.909 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.908+0000 7fb1f5738640 1 -- start start 2024-05-11T08:07:10.911 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.909+0000 7fb1f5738640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb1f00cd610 0x7fb1f0157370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:10.911 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.909+0000 7fb1f5738640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb1f00cd9f0 0x7fb1f01f9a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:10.912 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.909+0000 7fb1f5738640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb1f0160770 0x7fb1f0161b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:10.912 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.909+0000 7fb1f5738640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb1f00cde50 con 0x7fb1f00cd9f0 2024-05-11T08:07:10.912 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.909+0000 7fb1f5738640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb1f00d2a80 con 0x7fb1f00cd610 2024-05-11T08:07:10.912 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.909+0000 7fb1f5738640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fb1f00b5e90 con 0x7fb1f0160770 2024-05-11T08:07:10.912 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.909+0000 7fb1e67fc640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb1f00cd9f0 0x7fb1f01f9a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:10.912 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.909+0000 7fb1e67fc640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb1f00cd9f0 0x7fb1f01f9a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:55938/0 (socket says 172.21.15.167:55938) 2024-05-11T08:07:10.912 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.909+0000 7fb1e67fc640 1 -- 172.21.15.167:0/1931559344 learned_addr learned my addr 172.21.15.167:0/1931559344 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:07:10.912 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.909+0000 7fb1e67fc640 1 -- 172.21.15.167:0/1931559344 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb1f0160770 msgr2=0x7fb1f0161b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:10.912 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.909+0000 7fb1e67fc640 1 --2- 172.21.15.167:0/1931559344 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb1f0160770 0x7fb1f0161b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:10.912 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.909+0000 7fb1e67fc640 1 -- 172.21.15.167:0/1931559344 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb1f00cd610 msgr2=0x7fb1f0157370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:10.912 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.909+0000 7fb1e67fc640 1 --2- 172.21.15.167:0/1931559344 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb1f00cd610 0x7fb1f0157370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:10.912 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.909+0000 7fb1e67fc640 1 -- 172.21.15.167:0/1931559344 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1f00ca130 con 0x7fb1f00cd9f0 2024-05-11T08:07:10.912 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.910+0000 7fb1e67fc640 1 --2- 172.21.15.167:0/1931559344 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb1f00cd9f0 0x7fb1f01f9a40 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fb1d8035040 tx=0x7fb1d80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:10.912 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.910+0000 7fb1e7fff640 1 -- 172.21.15.167:0/1931559344 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1d8037040 con 0x7fb1f00cd9f0 2024-05-11T08:07:10.913 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.910+0000 7fb1e7fff640 1 -- 172.21.15.167:0/1931559344 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1d8030770 con 0x7fb1f00cd9f0 2024-05-11T08:07:10.913 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.910+0000 7fb1e7fff640 1 -- 172.21.15.167:0/1931559344 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1d8030b40 con 0x7fb1f00cd9f0 2024-05-11T08:07:10.913 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.910+0000 7fb1f5738640 1 -- 172.21.15.167:0/1931559344 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1f00ca3d0 con 0x7fb1f00cd9f0 2024-05-11T08:07:10.913 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.910+0000 7fb1f5738640 1 -- 172.21.15.167:0/1931559344 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1f00ca990 con 0x7fb1f00cd9f0 2024-05-11T08:07:10.913 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.911+0000 7fb1f5738640 1 -- 172.21.15.167:0/1931559344 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1f0069590 con 0x7fb1f00cd9f0 2024-05-11T08:07:10.913 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.912+0000 7fb1e7fff640 1 -- 172.21.15.167:0/1931559344 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fb1d8044020 con 0x7fb1f00cd9f0 2024-05-11T08:07:10.914 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.912+0000 7fb1e7fff640 1 --2- 172.21.15.167:0/1931559344 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fb1d00762d0 0x7fb1d0078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:10.914 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.912+0000 7fb1e7fff640 1 -- 172.21.15.167:0/1931559344 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 6496+0+0 (secure 0 0 0) 0x7fb1d80b8850 con 0x7fb1f00cd9f0 2024-05-11T08:07:10.914 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.912+0000 7fb1eeffd640 1 --2- 172.21.15.167:0/1931559344 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fb1d00762d0 0x7fb1d0078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:10.914 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.913+0000 7fb1eeffd640 1 --2- 172.21.15.167:0/1931559344 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fb1d00762d0 0x7fb1d0078790 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb1f01f8f20 tx=0x7fb1e0008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:10.916 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:10.916+0000 7fb1e7fff640 1 -- 172.21.15.167:0/1931559344 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fb1d8032e20 con 0x7fb1f00cd9f0 2024-05-11T08:07:11.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:11.137+0000 7fb1f5738640 1 -- 172.21.15.167:0/1931559344 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fb1f0162d50 con 0x7fb1f00cd9f0 2024-05-11T08:07:11.138 INFO:tasks.ceph.mon.a.smithi167.stderr:2024-05-11T08:07:11.138+0000 7feeb5228640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-11T08:07:11.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:11.139+0000 7fb1e7fff640 1 -- 172.21.15.167:0/1931559344 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v18) ==== 161+0+0 (secure 0 0 0) 0x7fb1d8082030 con 0x7fb1f00cd9f0 2024-05-11T08:07:11.139 INFO:teuthology.orchestra.run.smithi167.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-11T08:07:11.142 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:11.142+0000 7fb1f5738640 1 -- 172.21.15.167:0/1931559344 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fb1d00762d0 msgr2=0x7fb1d0078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:11.142 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:11.142+0000 7fb1f5738640 1 --2- 172.21.15.167:0/1931559344 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fb1d00762d0 0x7fb1d0078790 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb1f01f8f20 tx=0x7fb1e0008040 comp rx=0 tx=0).stop 2024-05-11T08:07:11.142 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:11.142+0000 7fb1f5738640 1 -- 172.21.15.167:0/1931559344 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb1f00cd9f0 msgr2=0x7fb1f01f9a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:11.142 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:11.142+0000 7fb1f5738640 1 --2- 172.21.15.167:0/1931559344 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb1f00cd9f0 0x7fb1f01f9a40 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fb1d8035040 tx=0x7fb1d80046c0 comp rx=0 tx=0).stop 2024-05-11T08:07:11.143 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:11.142+0000 7fb1f5738640 1 -- 172.21.15.167:0/1931559344 shutdown_connections 2024-05-11T08:07:11.143 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:11.142+0000 7fb1f5738640 1 --2- 172.21.15.167:0/1931559344 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fb1d00762d0 0x7fb1d0078790 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:11.143 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:11.142+0000 7fb1f5738640 1 --2- 172.21.15.167:0/1931559344 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb1f0160770 0x7fb1f0161b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:11.143 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:11.142+0000 7fb1f5738640 1 --2- 172.21.15.167:0/1931559344 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb1f00cd9f0 0x7fb1f01f9a40 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:11.143 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:11.142+0000 7fb1f5738640 1 --2- 172.21.15.167:0/1931559344 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb1f00cd610 0x7fb1f0157370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:11.143 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:11.142+0000 7fb1f5738640 1 -- 172.21.15.167:0/1931559344 >> 172.21.15.167:0/1931559344 conn(0x7fb1f0072730 msgr2=0x7fb1f00b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:07:11.143 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:11.142+0000 7fb1f5738640 1 -- 172.21.15.167:0/1931559344 shutdown_connections 2024-05-11T08:07:11.143 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:11.142+0000 7fb1f5738640 1 -- 172.21.15.167:0/1931559344 wait complete. 2024-05-11T08:07:11.154 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-11T08:07:11.182 INFO:tasks.ceph.mds.b.smithi204.stdout:starting mds.b at 2024-05-11T08:07:11.256 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:11.256+0000 7febaa447640 1 Processor -- start 2024-05-11T08:07:11.256 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:11.256+0000 7febaa447640 1 -- start start 2024-05-11T08:07:11.259 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:11.257+0000 7febaa447640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7feba4147ff0 0x7feba40d24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:11.259 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:11.257+0000 7febaa447640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feba4148cf0 0x7feba40b5de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:11.259 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:11.257+0000 7febaa447640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feba40d1430 con 0x7feba40d2e80 2024-05-11T08:07:11.259 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:11.257+0000 7febaa447640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7feba40d2040 con 0x7feba4147ff0 2024-05-11T08:07:11.259 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:11.257+0000 7febaa447640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7feba40d1710 con 0x7feba4148cf0 2024-05-11T08:07:11.259 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:11.257+0000 7feba3fff640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7feba4147ff0 0x7feba40d24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:11.259 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:11.257+0000 7feba3fff640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7feba4147ff0 0x7feba40d24d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:55940/0 (socket says 172.21.15.167:55940) 2024-05-11T08:07:11.259 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:11.257+0000 7feba3fff640 1 -- 172.21.15.167:0/1383464963 learned_addr learned my addr 172.21.15.167:0/1383464963 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:07:11.259 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:11.257+0000 7feba37fe640 1 --2- 172.21.15.167:0/1383464963 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feba4148cf0 0x7feba40b5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:11.260 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:11.257+0000 7feba3fff640 1 -- 172.21.15.167:0/1383464963 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feba4148cf0 msgr2=0x7feba40b5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:07:11.260 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:11.257+0000 7feba3fff640 1 --2- 172.21.15.167:0/1383464963 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feba4148cf0 0x7feba40b5de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:11.260 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:11.257+0000 7feba3fff640 1 -- 172.21.15.167:0/1383464963 >> v1:172.21.15.204:6789/0 conn(0x7feba40d2e80 legacy=0x7feba415f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:11.260 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:11.257+0000 7feba3fff640 1 -- 172.21.15.167:0/1383464963 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feba40d19f0 con 0x7feba4147ff0 2024-05-11T08:07:11.260 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:11.258+0000 7feba3fff640 1 --2- 172.21.15.167:0/1383464963 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feba4147ff0 0x7feba40d24d0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7feb94009870 tx=0x7feb9402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6b32481bb7a804c server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:11.260 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:11.258+0000 7feba2ffd640 1 -- 172.21.15.167:0/1383464963 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb94037070 con 0x7feba4147ff0 2024-05-11T08:07:11.260 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:11.258+0000 7feba2ffd640 1 -- 172.21.15.167:0/1383464963 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb94032440 con 0x7feba4147ff0 2024-05-11T08:07:11.260 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:11.258+0000 7feba2ffd640 1 -- 172.21.15.167:0/1383464963 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb94032810 con 0x7feba4147ff0 2024-05-11T08:07:11.260 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:11.259+0000 7febaa447640 1 -- 172.21.15.167:0/1383464963 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feba4147ff0 msgr2=0x7feba40d24d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:11.260 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:11.259+0000 7febaa447640 1 --2- 172.21.15.167:0/1383464963 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feba4147ff0 0x7feba40d24d0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7feb94009870 tx=0x7feb9402a270 comp rx=0 tx=0).stop 2024-05-11T08:07:11.260 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:11.259+0000 7febaa447640 1 -- 172.21.15.167:0/1383464963 shutdown_connections 2024-05-11T08:07:11.260 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:11.259+0000 7febaa447640 1 --2- 172.21.15.167:0/1383464963 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feba4148cf0 0x7feba40b5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:11.260 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:11.259+0000 7febaa447640 1 --2- 172.21.15.167:0/1383464963 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feba4147ff0 0x7feba40d24d0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:11.260 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:11.259+0000 7febaa447640 1 -- 172.21.15.167:0/1383464963 >> 172.21.15.167:0/1383464963 conn(0x7feba40b8300 msgr2=0x7feba40b86e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:07:11.261 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:11.259+0000 7febaa447640 1 -- 172.21.15.167:0/1383464963 shutdown_connections 2024-05-11T08:07:11.261 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:11.259+0000 7febaa447640 1 -- 172.21.15.167:0/1383464963 wait complete. 2024-05-11T08:07:11.263 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:11.260+0000 7febaa447640 1 Processor -- start 2024-05-11T08:07:11.263 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:11.260+0000 7febaa447640 1 -- start start 2024-05-11T08:07:11.264 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:11.261+0000 7febaa447640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feba40d2e80 0x7feba415d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:11.264 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:11.261+0000 7febaa447640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feba4147ff0 0x7feba406ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:11.264 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:11.261+0000 7febaa447640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feba4148cf0 0x7feba40704e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:11.264 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:11.261+0000 7febaa447640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7feba40d2a10 con 0x7feba4147ff0 2024-05-11T08:07:11.264 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:11.261+0000 7febaa447640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7feba41483d0 con 0x7feba40d2e80 2024-05-11T08:07:11.264 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:11.261+0000 7febaa447640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7feba40b6320 con 0x7feba4148cf0 2024-05-11T08:07:11.264 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:11.261+0000 7feba37fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feba4147ff0 0x7feba406ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:11.264 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:11.261+0000 7feba37fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feba4147ff0 0x7feba406ffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:55956/0 (socket says 172.21.15.167:55956) 2024-05-11T08:07:11.264 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:11.261+0000 7feba37fe640 1 -- 172.21.15.167:0/1196366376 learned_addr learned my addr 172.21.15.167:0/1196366376 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:07:11.264 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:11.261+0000 7feba3fff640 1 --2- 172.21.15.167:0/1196366376 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feba40d2e80 0x7feba415d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:11.264 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:11.261+0000 7feba89e4640 1 --2- 172.21.15.167:0/1196366376 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feba4148cf0 0x7feba40704e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:11.264 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:11.261+0000 7feba37fe640 1 -- 172.21.15.167:0/1196366376 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feba4148cf0 msgr2=0x7feba40704e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:11.264 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:11.261+0000 7feba37fe640 1 --2- 172.21.15.167:0/1196366376 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feba4148cf0 0x7feba40704e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:11.264 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:11.261+0000 7feba37fe640 1 -- 172.21.15.167:0/1196366376 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feba40d2e80 msgr2=0x7feba415d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:11.264 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:11.261+0000 7feba37fe640 1 --2- 172.21.15.167:0/1196366376 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feba40d2e80 0x7feba415d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:11.265 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:11.261+0000 7feba37fe640 1 -- 172.21.15.167:0/1196366376 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feba40cfa00 con 0x7feba4147ff0 2024-05-11T08:07:11.265 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:11.261+0000 7feba37fe640 1 --2- 172.21.15.167:0/1196366376 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feba4147ff0 0x7feba406ffa0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7feb9800d5f0 tx=0x7feb9800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:11.265 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:11.261+0000 7feba0ff9640 1 -- 172.21.15.167:0/1196366376 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb980125f0 con 0x7feba4147ff0 2024-05-11T08:07:11.265 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:11.261+0000 7feba0ff9640 1 -- 172.21.15.167:0/1196366376 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb98019070 con 0x7feba4147ff0 2024-05-11T08:07:11.265 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:11.261+0000 7febaa447640 1 -- 172.21.15.167:0/1196366376 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feba40cfce0 con 0x7feba4147ff0 2024-05-11T08:07:11.265 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:11.261+0000 7feba0ff9640 1 -- 172.21.15.167:0/1196366376 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb980129d0 con 0x7feba4147ff0 2024-05-11T08:07:11.265 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:11.262+0000 7febaa447640 1 -- 172.21.15.167:0/1196366376 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feba415ddf0 con 0x7feba4147ff0 2024-05-11T08:07:11.265 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:11.262+0000 7febaa447640 1 -- 172.21.15.167:0/1196366376 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feba4069520 con 0x7feba4147ff0 2024-05-11T08:07:11.265 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:11.263+0000 7feba0ff9640 1 -- 172.21.15.167:0/1196366376 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7feb98012b70 con 0x7feba4147ff0 2024-05-11T08:07:11.265 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:11.264+0000 7feba0ff9640 1 --2- 172.21.15.167:0/1196366376 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7feb900762d0 0x7feb90078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:11.265 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:11.264+0000 7feba0ff9640 1 -- 172.21.15.167:0/1196366376 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 6582+0+0 (secure 0 0 0) 0x7feb98097bf0 con 0x7feba4147ff0 2024-05-11T08:07:11.265 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:11.264+0000 7feba3fff640 1 --2- 172.21.15.167:0/1196366376 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7feb900762d0 0x7feb90078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:11.266 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:11.264+0000 7feba3fff640 1 --2- 172.21.15.167:0/1196366376 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7feb900762d0 0x7feb90078790 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7feb94034000 tx=0x7feb9402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:11.268 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:11.268+0000 7feba0ff9640 1 -- 172.21.15.167:0/1196366376 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7feb98060c20 con 0x7feba4147ff0 2024-05-11T08:07:11.486 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:11.486+0000 7febaa447640 1 -- 172.21.15.167:0/1196366376 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7feba415e080 con 0x7feba4147ff0 2024-05-11T08:07:12.141 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:12.141+0000 7feba0ff9640 1 -- 172.21.15.167:0/1196366376 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v19) ==== 104+0+0 (secure 0 0 0) 0x7feb98064f20 con 0x7feba4147ff0 2024-05-11T08:07:12.143 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:12.143+0000 7febaa447640 1 -- 172.21.15.167:0/1196366376 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7feb900762d0 msgr2=0x7feb90078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:12.143 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:12.143+0000 7febaa447640 1 --2- 172.21.15.167:0/1196366376 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7feb900762d0 0x7feb90078790 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7feb94034000 tx=0x7feb9402e000 comp rx=0 tx=0).stop 2024-05-11T08:07:12.143 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:12.143+0000 7febaa447640 1 -- 172.21.15.167:0/1196366376 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feba4147ff0 msgr2=0x7feba406ffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:12.144 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:12.143+0000 7febaa447640 1 --2- 172.21.15.167:0/1196366376 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feba4147ff0 0x7feba406ffa0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7feb9800d5f0 tx=0x7feb9800dab0 comp rx=0 tx=0).stop 2024-05-11T08:07:12.144 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:12.143+0000 7febaa447640 1 -- 172.21.15.167:0/1196366376 shutdown_connections 2024-05-11T08:07:12.144 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:12.143+0000 7febaa447640 1 --2- 172.21.15.167:0/1196366376 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7feb900762d0 0x7feb90078790 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:12.144 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:12.143+0000 7febaa447640 1 --2- 172.21.15.167:0/1196366376 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feba4148cf0 0x7feba40704e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:12.144 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:12.143+0000 7febaa447640 1 --2- 172.21.15.167:0/1196366376 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feba4147ff0 0x7feba406ffa0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:12.144 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:12.143+0000 7febaa447640 1 --2- 172.21.15.167:0/1196366376 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feba40d2e80 0x7feba415d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:12.144 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:12.143+0000 7febaa447640 1 -- 172.21.15.167:0/1196366376 >> 172.21.15.167:0/1196366376 conn(0x7feba40b8300 msgr2=0x7feba40b86e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:07:12.144 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:12.143+0000 7febaa447640 1 -- 172.21.15.167:0/1196366376 shutdown_connections 2024-05-11T08:07:12.144 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:12.143+0000 7febaa447640 1 -- 172.21.15.167:0/1196366376 wait complete. 2024-05-11T08:07:12.156 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-11T08:07:12.262 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:12.262+0000 7f641a01d640 1 Processor -- start 2024-05-11T08:07:12.263 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:12.263+0000 7f641a01d640 1 -- start start 2024-05-11T08:07:12.264 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:12.264+0000 7f641a01d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6414146960 0x7f6414161560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:12.264 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:12.264+0000 7f641a01d640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f64140c9a30 0x7f6414070000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:12.264 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:12.264+0000 7f641a01d640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f6414070540 0x7f64140cd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:12.264 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:12.264+0000 7f641a01d640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f6414070980 con 0x7f6414070540 2024-05-11T08:07:12.264 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:12.264+0000 7f641a01d640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f64140d11c0 con 0x7f64140c9a30 2024-05-11T08:07:12.264 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:12.264+0000 7f641a01d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f64140d2ad0 con 0x7f6414146960 2024-05-11T08:07:12.264 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:12.264+0000 7f6413fff640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f6414070540 0x7f64140cd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:12.264 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:12.264+0000 7f6413fff640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f6414070540 0x7f64140cd500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:55960/0 (socket says 172.21.15.167:55960) 2024-05-11T08:07:12.264 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:12.264+0000 7f6413fff640 1 -- 172.21.15.167:0/1972600977 learned_addr learned my addr 172.21.15.167:0/1972600977 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:07:12.265 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:12.264+0000 7f64137fe640 1 --2- 172.21.15.167:0/1972600977 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6414146960 0x7f6414161560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:12.265 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:12.264+0000 7f6412ffd640 1 --2- 172.21.15.167:0/1972600977 >> v2:172.21.15.204:3300/0 conn(0x7f64140c9a30 0x7f6414070000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:12.265 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:12.264+0000 7f6413fff640 1 -- 172.21.15.167:0/1972600977 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6414146960 msgr2=0x7f6414161560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:07:12.265 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:12.264+0000 7f6413fff640 1 --2- 172.21.15.167:0/1972600977 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6414146960 0x7f6414161560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:12.265 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:12.264+0000 7f6413fff640 1 -- 172.21.15.167:0/1972600977 >> v2:172.21.15.204:3300/0 conn(0x7f64140c9a30 msgr2=0x7f6414070000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:07:12.265 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:12.264+0000 7f6413fff640 1 --2- 172.21.15.167:0/1972600977 >> v2:172.21.15.204:3300/0 conn(0x7f64140c9a30 0x7f6414070000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:12.265 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:12.264+0000 7f6413fff640 1 -- 172.21.15.167:0/1972600977 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64140d0c00 con 0x7f6414070540 2024-05-11T08:07:12.266 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:12.265+0000 7f6413fff640 1 --2- 172.21.15.167:0/1972600977 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6414070540 0x7f64140cd500 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f64140d2ce0 tx=0x7f640c02a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf7128d3be9d1ad8 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:12.266 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:12.265+0000 7f64127fc640 1 -- 172.21.15.167:0/1972600977 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f640c037070 con 0x7f6414070540 2024-05-11T08:07:12.266 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:12.265+0000 7f64127fc640 1 -- 172.21.15.167:0/1972600977 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f640c032440 con 0x7f6414070540 2024-05-11T08:07:12.266 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:12.265+0000 7f64127fc640 1 -- 172.21.15.167:0/1972600977 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f640c032810 con 0x7f6414070540 2024-05-11T08:07:12.266 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:12.265+0000 7f641a01d640 1 -- 172.21.15.167:0/1972600977 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6414070540 msgr2=0x7f64140cd500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:12.266 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:12.265+0000 7f641a01d640 1 --2- 172.21.15.167:0/1972600977 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6414070540 0x7f64140cd500 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f64140d2ce0 tx=0x7f640c02a130 comp rx=0 tx=0).stop 2024-05-11T08:07:12.266 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:12.265+0000 7f641a01d640 1 -- 172.21.15.167:0/1972600977 shutdown_connections 2024-05-11T08:07:12.266 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:12.265+0000 7f641a01d640 1 --2- 172.21.15.167:0/1972600977 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6414070540 0x7f64140cd500 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:12.266 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:12.265+0000 7f641a01d640 1 --2- 172.21.15.167:0/1972600977 >> v2:172.21.15.204:3300/0 conn(0x7f64140c9a30 0x7f6414070000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:12.266 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:12.265+0000 7f641a01d640 1 --2- 172.21.15.167:0/1972600977 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6414146960 0x7f6414161560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:12.266 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:12.265+0000 7f641a01d640 1 -- 172.21.15.167:0/1972600977 >> 172.21.15.167:0/1972600977 conn(0x7f64140b4ae0 msgr2=0x7f64140b4ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:07:12.267 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:12.265+0000 7f641a01d640 1 -- 172.21.15.167:0/1972600977 shutdown_connections 2024-05-11T08:07:12.267 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:12.265+0000 7f641a01d640 1 -- 172.21.15.167:0/1972600977 wait complete. 2024-05-11T08:07:12.269 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:12.267+0000 7f641a01d640 1 Processor -- start 2024-05-11T08:07:12.269 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:12.267+0000 7f641a01d640 1 -- start start 2024-05-11T08:07:12.269 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:12.268+0000 7f641a01d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6414070540 0x7f6414141570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:12.269 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:12.268+0000 7f641a01d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f64140c9a30 0x7f64141f2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:12.269 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:12.268+0000 7f641a01d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6414146960 0x7f64141f3420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:12.269 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:12.268+0000 7f641a01d640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6414146d40 con 0x7f6414070540 2024-05-11T08:07:12.269 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:12.268+0000 7f641a01d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6414161aa0 con 0x7f64140c9a30 2024-05-11T08:07:12.269 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:12.268+0000 7f641a01d640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f6414070cf0 con 0x7f6414146960 2024-05-11T08:07:12.269 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:12.268+0000 7f64137fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6414070540 0x7f6414141570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:12.269 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:12.268+0000 7f64137fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6414070540 0x7f6414141570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:55962/0 (socket says 172.21.15.167:55962) 2024-05-11T08:07:12.269 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:12.268+0000 7f64137fe640 1 -- 172.21.15.167:0/2236233555 learned_addr learned my addr 172.21.15.167:0/2236233555 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:07:12.269 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:12.268+0000 7f6412ffd640 1 --2- 172.21.15.167:0/2236233555 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f64140c9a30 0x7f64141f2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:12.270 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:12.268+0000 7f6413fff640 1 --2- 172.21.15.167:0/2236233555 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6414146960 0x7f64141f3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:12.270 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:12.268+0000 7f64137fe640 1 -- 172.21.15.167:0/2236233555 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6414146960 msgr2=0x7f64141f3420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:12.270 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:12.268+0000 7f64137fe640 1 --2- 172.21.15.167:0/2236233555 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6414146960 0x7f64141f3420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:12.270 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:12.268+0000 7f64137fe640 1 -- 172.21.15.167:0/2236233555 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f64140c9a30 msgr2=0x7f64141f2ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:12.270 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:12.268+0000 7f64137fe640 1 --2- 172.21.15.167:0/2236233555 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f64140c9a30 0x7f64141f2ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:12.270 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:12.268+0000 7f64137fe640 1 -- 172.21.15.167:0/2236233555 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f641413f9d0 con 0x7f6414070540 2024-05-11T08:07:12.270 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:12.268+0000 7f6413fff640 1 --2- 172.21.15.167:0/2236233555 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6414146960 0x7f64141f3420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T08:07:12.270 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:12.268+0000 7f6412ffd640 1 --2- 172.21.15.167:0/2236233555 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f64140c9a30 0x7f64141f2ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T08:07:12.270 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:12.268+0000 7f64137fe640 1 --2- 172.21.15.167:0/2236233555 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6414070540 0x7f6414141570 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f6404009db0 tx=0x7f640400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:12.270 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:12.268+0000 7f63fbfff640 1 -- 172.21.15.167:0/2236233555 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6404018020 con 0x7f6414070540 2024-05-11T08:07:12.270 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:12.268+0000 7f63fbfff640 1 -- 172.21.15.167:0/2236233555 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6404014480 con 0x7f6414070540 2024-05-11T08:07:12.270 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:12.268+0000 7f641a01d640 1 -- 172.21.15.167:0/2236233555 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f641406de90 con 0x7f6414070540 2024-05-11T08:07:12.270 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:12.268+0000 7f63fbfff640 1 -- 172.21.15.167:0/2236233555 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6404014850 con 0x7f6414070540 2024-05-11T08:07:12.271 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:12.268+0000 7f641a01d640 1 -- 172.21.15.167:0/2236233555 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f641413fcb0 con 0x7f6414070540 2024-05-11T08:07:12.271 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:12.269+0000 7f641a01d640 1 -- 172.21.15.167:0/2236233555 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64140752e0 con 0x7f6414070540 2024-05-11T08:07:12.271 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:12.270+0000 7f63fbfff640 1 -- 172.21.15.167:0/2236233555 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f6404005d10 con 0x7f6414070540 2024-05-11T08:07:12.271 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:12.270+0000 7f63fbfff640 1 --2- 172.21.15.167:0/2236233555 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f64000762d0 0x7f6400078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:12.272 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:12.270+0000 7f63fbfff640 1 -- 172.21.15.167:0/2236233555 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 6582+0+0 (secure 0 0 0) 0x7f6404098490 con 0x7f6414070540 2024-05-11T08:07:12.272 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:12.270+0000 7f6412ffd640 1 --2- 172.21.15.167:0/2236233555 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f64000762d0 0x7f6400078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:12.272 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:12.271+0000 7f6412ffd640 1 --2- 172.21.15.167:0/2236233555 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f64000762d0 0x7f6400078790 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f64141378e0 tx=0x7f6408008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:12.274 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:12.275+0000 7f63fbfff640 1 -- 172.21.15.167:0/2236233555 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f6404022070 con 0x7f6414070540 2024-05-11T08:07:12.458 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:12.458+0000 7f641a01d640 1 -- 172.21.15.167:0/2236233555 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f64140d0820 con 0x7f6414070540 2024-05-11T08:07:13.142 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:13.142+0000 7f63fbfff640 1 -- 172.21.15.167:0/2236233555 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v46) ==== 181+0+0 (secure 0 0 0) 0x7f64140d0820 con 0x7f6414070540 2024-05-11T08:07:13.142 INFO:teuthology.orchestra.run.smithi167.stderr:pool 'cephfs_metadata' removed 2024-05-11T08:07:13.145 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:13.144+0000 7f641a01d640 1 -- 172.21.15.167:0/2236233555 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f64000762d0 msgr2=0x7f6400078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:13.145 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:13.144+0000 7f641a01d640 1 --2- 172.21.15.167:0/2236233555 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f64000762d0 0x7f6400078790 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f64141378e0 tx=0x7f6408008040 comp rx=0 tx=0).stop 2024-05-11T08:07:13.145 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:13.144+0000 7f641a01d640 1 -- 172.21.15.167:0/2236233555 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6414070540 msgr2=0x7f6414141570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:13.145 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:13.144+0000 7f641a01d640 1 --2- 172.21.15.167:0/2236233555 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6414070540 0x7f6414141570 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f6404009db0 tx=0x7f640400c490 comp rx=0 tx=0).stop 2024-05-11T08:07:13.145 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:13.144+0000 7f641a01d640 1 -- 172.21.15.167:0/2236233555 shutdown_connections 2024-05-11T08:07:13.145 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:13.144+0000 7f641a01d640 1 --2- 172.21.15.167:0/2236233555 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f64000762d0 0x7f6400078790 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:13.145 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:13.144+0000 7f641a01d640 1 --2- 172.21.15.167:0/2236233555 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6414146960 0x7f64141f3420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:13.145 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:13.144+0000 7f641a01d640 1 --2- 172.21.15.167:0/2236233555 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f64140c9a30 0x7f64141f2ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:13.145 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:13.144+0000 7f641a01d640 1 --2- 172.21.15.167:0/2236233555 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6414070540 0x7f6414141570 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:13.145 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:13.145+0000 7f641a01d640 1 -- 172.21.15.167:0/2236233555 >> 172.21.15.167:0/2236233555 conn(0x7f64140b4ae0 msgr2=0x7f64140b2930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:07:13.145 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:13.145+0000 7f641a01d640 1 -- 172.21.15.167:0/2236233555 shutdown_connections 2024-05-11T08:07:13.145 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:13.145+0000 7f641a01d640 1 -- 172.21.15.167:0/2236233555 wait complete. 2024-05-11T08:07:13.156 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-11T08:07:13.263 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:13.262+0000 7f80590fc640 1 Processor -- start 2024-05-11T08:07:13.263 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:13.263+0000 7f80590fc640 1 -- start start 2024-05-11T08:07:13.265 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:13.263+0000 7f80590fc640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f8054163580 0x7f80540cd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:13.266 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:13.263+0000 7f80590fc640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80540cfad0 con 0x7f80540d2630 2024-05-11T08:07:13.266 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:13.263+0000 7f80590fc640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80540cfdb0 con 0x7f8054148d50 2024-05-11T08:07:13.266 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:13.263+0000 7f80590fc640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f80540cda40 con 0x7f8054163580 2024-05-11T08:07:13.266 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:13.264+0000 7f805359e640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f80540d2630 0x7f80540d0e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:47474/0 (socket says 172.21.15.167:47474) 2024-05-11T08:07:13.266 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:13.264+0000 7f805359e640 1 -- 172.21.15.167:0/54901798 learned_addr learned my addr 172.21.15.167:0/54901798 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T08:07:13.266 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:13.264+0000 7f805259c640 1 --2- 172.21.15.167:0/54901798 >> v2:172.21.15.204:3300/0 conn(0x7f8054163580 0x7f80540cd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:13.266 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:13.264+0000 7f8051d9b640 1 -- 172.21.15.167:0/54901798 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2534135648 0 0) 0x7f80540cfad0 con 0x7f80540d2630 2024-05-11T08:07:13.266 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:13.264+0000 7f8051d9b640 1 -- 172.21.15.167:0/54901798 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8040003520 con 0x7f80540d2630 2024-05-11T08:07:13.266 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:13.264+0000 7f8051d9b640 1 -- 172.21.15.167:0/54901798 <== mon.0 v1:172.21.15.167:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3823619649 0 0) 0x7f8040003520 con 0x7f80540d2630 2024-05-11T08:07:13.266 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:13.264+0000 7f8051d9b640 1 -- 172.21.15.167:0/54901798 >> v2:172.21.15.204:3300/0 conn(0x7f8054163580 msgr2=0x7f80540cd500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:07:13.266 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:13.264+0000 7f8051d9b640 1 --2- 172.21.15.167:0/54901798 >> v2:172.21.15.204:3300/0 conn(0x7f8054163580 0x7f80540cd500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:13.266 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:13.264+0000 7f8051d9b640 1 -- 172.21.15.167:0/54901798 >> v1:172.21.15.204:6789/0 conn(0x7f8054148d50 legacy=0x7f8054162e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:13.266 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:13.264+0000 7f8051d9b640 1 -- 172.21.15.167:0/54901798 --> v1:172.21.15.167:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80540d0090 con 0x7f80540d2630 2024-05-11T08:07:13.266 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:13.264+0000 7f8051d9b640 1 -- 172.21.15.167:0/54901798 <== mon.0 v1:172.21.15.167:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3602277846 0 0) 0x7f8044001a70 con 0x7f80540d2630 2024-05-11T08:07:13.267 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:13.264+0000 7f8051d9b640 1 -- 172.21.15.167:0/54901798 >> v1:172.21.15.167:6789/0 conn(0x7f80540d2630 legacy=0x7f80540d0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:13.267 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:13.264+0000 7f8051d9b640 1 --2- 172.21.15.167:0/54901798 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8040003a50 0x7f8040003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:13.267 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:13.264+0000 7f8051d9b640 1 --2- 172.21.15.167:0/54901798 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8040004800 0x7f8040004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:13.267 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:13.264+0000 7f8051d9b640 1 --2- 172.21.15.167:0/54901798 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f80400053c0 0x7f804000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:13.267 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:13.264+0000 7f8051d9b640 1 -- 172.21.15.167:0/54901798 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f80540cda40 con 0x7f80400053c0 2024-05-11T08:07:13.267 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:13.264+0000 7f8051d9b640 1 -- 172.21.15.167:0/54901798 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f804000c230 con 0x7f8040004800 2024-05-11T08:07:13.267 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:13.264+0000 7f8051d9b640 1 -- 172.21.15.167:0/54901798 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f804000c430 con 0x7f8040003a50 2024-05-11T08:07:13.267 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:13.264+0000 7f805359e640 1 --2- 172.21.15.167:0/54901798 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f80400053c0 0x7f804000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:13.267 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:13.264+0000 7f805259c640 1 --2- 172.21.15.167:0/54901798 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8040004800 0x7f8040004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:13.267 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:13.264+0000 7f8052d9d640 1 --2- 172.21.15.167:0/54901798 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8040003a50 0x7f8040003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:13.267 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:13.265+0000 7f805359e640 1 -- 172.21.15.167:0/54901798 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8040003a50 msgr2=0x7f8040003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:07:13.267 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:13.265+0000 7f805359e640 1 --2- 172.21.15.167:0/54901798 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8040003a50 0x7f8040003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:13.267 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:13.265+0000 7f805359e640 1 -- 172.21.15.167:0/54901798 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8040004800 msgr2=0x7f8040004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:07:13.267 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:13.265+0000 7f805359e640 1 --2- 172.21.15.167:0/54901798 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8040004800 0x7f8040004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:13.267 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:13.265+0000 7f805359e640 1 -- 172.21.15.167:0/54901798 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8040003520 con 0x7f80400053c0 2024-05-11T08:07:13.267 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:13.265+0000 7f805359e640 1 -- 172.21.15.167:0/54901798 reap_dead start 2024-05-11T08:07:13.268 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:13.265+0000 7f805359e640 1 --2- 172.21.15.167:0/54901798 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f80400053c0 0x7f804000b9c0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f804400a830 tx=0x7f8040005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c671ed80a97ac4b0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:13.268 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:13.265+0000 7f8051d9b640 1 -- 172.21.15.167:0/54901798 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f804400a450 con 0x7f80400053c0 2024-05-11T08:07:13.268 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:13.265+0000 7f8051d9b640 1 -- 172.21.15.167:0/54901798 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f804402f070 con 0x7f80400053c0 2024-05-11T08:07:13.268 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:13.265+0000 7f8051d9b640 1 -- 172.21.15.167:0/54901798 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f804402bc40 con 0x7f80400053c0 2024-05-11T08:07:13.271 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:13.266+0000 7f80590fc640 1 -- 172.21.15.167:0/54901798 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f80400053c0 msgr2=0x7f804000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:13.271 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:13.266+0000 7f80590fc640 1 --2- 172.21.15.167:0/54901798 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f80400053c0 0x7f804000b9c0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f804400a830 tx=0x7f8040005830 comp rx=0 tx=0).stop 2024-05-11T08:07:13.271 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:13.266+0000 7f80590fc640 1 -- 172.21.15.167:0/54901798 shutdown_connections 2024-05-11T08:07:13.271 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:13.266+0000 7f80590fc640 1 --2- 172.21.15.167:0/54901798 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f80400053c0 0x7f804000b9c0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:13.271 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:13.266+0000 7f80590fc640 1 -- 172.21.15.167:0/54901798 >> 172.21.15.167:0/54901798 conn(0x7f80540b4ae0 msgr2=0x7f80540b4ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:07:13.271 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:13.266+0000 7f80590fc640 1 -- 172.21.15.167:0/54901798 shutdown_connections 2024-05-11T08:07:13.271 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:13.266+0000 7f80590fc640 1 -- 172.21.15.167:0/54901798 wait complete. 2024-05-11T08:07:13.271 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:13.268+0000 7f80590fc640 1 Processor -- start 2024-05-11T08:07:13.271 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:13.268+0000 7f80590fc640 1 -- start start 2024-05-11T08:07:13.271 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:13.268+0000 7f80590fc640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f80400053c0 0x7f805415ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:13.272 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:13.268+0000 7f80590fc640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f80540cb840 0x7f80540cbca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:13.272 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:13.268+0000 7f80590fc640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8054160d80 0x7f8054161200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:13.272 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:13.268+0000 7f80590fc640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f80540cd500 con 0x7f8054160d80 2024-05-11T08:07:13.272 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:13.268+0000 7f80590fc640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f80540b5de0 con 0x7f80400053c0 2024-05-11T08:07:13.272 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:13.268+0000 7f80590fc640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f80540cd680 con 0x7f80540cb840 2024-05-11T08:07:13.272 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:13.268+0000 7f805359e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8054160d80 0x7f8054161200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:13.272 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:13.268+0000 7f805359e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8054160d80 0x7f8054161200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:55986/0 (socket says 172.21.15.167:55986) 2024-05-11T08:07:13.272 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:13.268+0000 7f805359e640 1 -- 172.21.15.167:0/113424794 learned_addr learned my addr 172.21.15.167:0/113424794 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:07:13.272 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:13.269+0000 7f8052d9d640 1 --2- 172.21.15.167:0/113424794 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f80400053c0 0x7f805415ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:13.272 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:13.269+0000 7f805259c640 1 --2- 172.21.15.167:0/113424794 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f80540cb840 0x7f80540cbca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:13.272 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:13.269+0000 7f805359e640 1 -- 172.21.15.167:0/113424794 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f80540cb840 msgr2=0x7f80540cbca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:13.272 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:13.269+0000 7f805359e640 1 --2- 172.21.15.167:0/113424794 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f80540cb840 0x7f80540cbca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:13.272 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:13.269+0000 7f805359e640 1 -- 172.21.15.167:0/113424794 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f80400053c0 msgr2=0x7f805415ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:13.272 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:13.269+0000 7f805359e640 1 --2- 172.21.15.167:0/113424794 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f80400053c0 0x7f805415ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:13.272 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:13.269+0000 7f805359e640 1 -- 172.21.15.167:0/113424794 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8054161b30 con 0x7f8054160d80 2024-05-11T08:07:13.273 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:13.269+0000 7f805259c640 1 --2- 172.21.15.167:0/113424794 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f80540cb840 0x7f80540cbca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T08:07:13.273 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:13.269+0000 7f8052d9d640 1 --2- 172.21.15.167:0/113424794 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f80400053c0 0x7f805415ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T08:07:13.273 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:13.269+0000 7f805359e640 1 --2- 172.21.15.167:0/113424794 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8054160d80 0x7f8054161200 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f804402b470 tx=0x7f80540cff80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:13.273 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:13.269+0000 7f804b7fe640 1 -- 172.21.15.167:0/113424794 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8044001fa0 con 0x7f8054160d80 2024-05-11T08:07:13.273 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:13.269+0000 7f804b7fe640 1 -- 172.21.15.167:0/113424794 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f804402b7c0 con 0x7f8054160d80 2024-05-11T08:07:13.273 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:13.269+0000 7f804b7fe640 1 -- 172.21.15.167:0/113424794 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f804402bdf0 con 0x7f8054160d80 2024-05-11T08:07:13.273 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:13.269+0000 7f80590fc640 1 -- 172.21.15.167:0/113424794 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8054161e10 con 0x7f8054160d80 2024-05-11T08:07:13.273 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:13.269+0000 7f80590fc640 1 -- 172.21.15.167:0/113424794 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80541e8e10 con 0x7f8054160d80 2024-05-11T08:07:13.273 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:13.270+0000 7f80590fc640 1 -- 172.21.15.167:0/113424794 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80540752e0 con 0x7f8054160d80 2024-05-11T08:07:13.273 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:13.271+0000 7f804b7fe640 1 -- 172.21.15.167:0/113424794 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f8044034470 con 0x7f8054160d80 2024-05-11T08:07:13.273 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:13.271+0000 7f804b7fe640 1 --2- 172.21.15.167:0/113424794 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f80340762d0 0x7f8034078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:13.273 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:13.271+0000 7f804b7fe640 1 -- 172.21.15.167:0/113424794 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 6144+0+0 (secure 0 0 0) 0x7f80440b7250 con 0x7f8054160d80 2024-05-11T08:07:13.273 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:13.271+0000 7f8052d9d640 1 --2- 172.21.15.167:0/113424794 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f80340762d0 0x7f8034078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:13.274 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:13.271+0000 7f8052d9d640 1 --2- 172.21.15.167:0/113424794 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f80340762d0 0x7f8034078790 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f804c0036b0 tx=0x7f804c009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:13.275 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:13.275+0000 7f804b7fe640 1 -- 172.21.15.167:0/113424794 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f8044080a00 con 0x7f8054160d80 2024-05-11T08:07:13.456 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:13.456+0000 7f80590fc640 1 -- 172.21.15.167:0/113424794 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f8054162330 con 0x7f8054160d80 2024-05-11T08:07:14.148 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:14.148+0000 7f804b7fe640 1 -- 172.21.15.167:0/113424794 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v47) ==== 169+0+0 (secure 0 0 0) 0x7f8054162330 con 0x7f8054160d80 2024-05-11T08:07:14.148 INFO:teuthology.orchestra.run.smithi167.stderr:pool 'cephfs_data' removed 2024-05-11T08:07:14.150 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:14.150+0000 7f80590fc640 1 -- 172.21.15.167:0/113424794 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f80340762d0 msgr2=0x7f8034078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:14.150 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:14.150+0000 7f80590fc640 1 --2- 172.21.15.167:0/113424794 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f80340762d0 0x7f8034078790 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f804c0036b0 tx=0x7f804c009040 comp rx=0 tx=0).stop 2024-05-11T08:07:14.150 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:14.150+0000 7f80590fc640 1 -- 172.21.15.167:0/113424794 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8054160d80 msgr2=0x7f8054161200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:14.150 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:14.150+0000 7f80590fc640 1 --2- 172.21.15.167:0/113424794 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8054160d80 0x7f8054161200 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f804402b470 tx=0x7f80540cff80 comp rx=0 tx=0).stop 2024-05-11T08:07:14.151 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:14.150+0000 7f80590fc640 1 -- 172.21.15.167:0/113424794 shutdown_connections 2024-05-11T08:07:14.151 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:14.150+0000 7f80590fc640 1 --2- 172.21.15.167:0/113424794 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f80340762d0 0x7f8034078790 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:14.151 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:14.150+0000 7f80590fc640 1 --2- 172.21.15.167:0/113424794 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8054160d80 0x7f8054161200 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:14.151 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:14.150+0000 7f80590fc640 1 --2- 172.21.15.167:0/113424794 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f80540cb840 0x7f80540cbca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:14.151 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:14.150+0000 7f80590fc640 1 --2- 172.21.15.167:0/113424794 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f80400053c0 0x7f805415ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:14.151 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:14.150+0000 7f80590fc640 1 -- 172.21.15.167:0/113424794 >> 172.21.15.167:0/113424794 conn(0x7f80540b4ae0 msgr2=0x7f80540b3df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:07:14.151 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:14.150+0000 7f80590fc640 1 -- 172.21.15.167:0/113424794 shutdown_connections 2024-05-11T08:07:14.151 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:14.150+0000 7f80590fc640 1 -- 172.21.15.167:0/113424794 wait complete. 2024-05-11T08:07:14.165 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm data_new data_new --yes-i-really-really-mean-it 2024-05-11T08:07:14.276 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:14.276+0000 7f149fd5f640 1 Processor -- start 2024-05-11T08:07:14.276 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:14.276+0000 7f149fd5f640 1 -- start start 2024-05-11T08:07:14.279 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:14.277+0000 7f149fd5f640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f1498070710 0x7f14980cd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:14.279 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:14.277+0000 7f149fd5f640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14980d11c0 con 0x7f14980c9a30 2024-05-11T08:07:14.280 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:14.277+0000 7f149fd5f640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14980d2ad0 con 0x7f1498146960 2024-05-11T08:07:14.280 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:14.277+0000 7f149fd5f640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f14980d0c00 con 0x7f1498070710 2024-05-11T08:07:14.280 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:14.277+0000 7f149d2fa640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f14980c9a30 0x7f1498070000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:47478/0 (socket says 172.21.15.167:47478) 2024-05-11T08:07:14.280 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:14.277+0000 7f149d2fa640 1 -- 172.21.15.167:0/3137127041 learned_addr learned my addr 172.21.15.167:0/3137127041 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T08:07:14.280 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:14.277+0000 7f149caf9640 1 -- 172.21.15.167:0/3137127041 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2442330308 0 0) 0x7f14980d11c0 con 0x7f14980c9a30 2024-05-11T08:07:14.280 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:14.277+0000 7f149caf9640 1 -- 172.21.15.167:0/3137127041 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1484003520 con 0x7f14980c9a30 2024-05-11T08:07:14.280 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:14.277+0000 7f149e2fc640 1 --2- 172.21.15.167:0/3137127041 >> v2:172.21.15.204:3300/0 conn(0x7f1498070710 0x7f14980cd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:14.280 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:14.277+0000 7f149caf9640 1 -- 172.21.15.167:0/3137127041 <== mon.0 v1:172.21.15.167:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2263695872 0 0) 0x7f1484003520 con 0x7f14980c9a30 2024-05-11T08:07:14.280 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:14.277+0000 7f149caf9640 1 -- 172.21.15.167:0/3137127041 >> v2:172.21.15.204:3300/0 conn(0x7f1498070710 msgr2=0x7f14980cd500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:07:14.280 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:14.277+0000 7f149caf9640 1 --2- 172.21.15.167:0/3137127041 >> v2:172.21.15.204:3300/0 conn(0x7f1498070710 0x7f14980cd500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:14.280 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:14.277+0000 7f149caf9640 1 -- 172.21.15.167:0/3137127041 >> v1:172.21.15.204:6789/0 conn(0x7f1498146960 legacy=0x7f1498161560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:14.280 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:14.277+0000 7f149caf9640 1 -- 172.21.15.167:0/3137127041 --> v1:172.21.15.167:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14980d0ee0 con 0x7f14980c9a30 2024-05-11T08:07:14.281 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:14.277+0000 7f149caf9640 1 -- 172.21.15.167:0/3137127041 <== mon.0 v1:172.21.15.167:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3602277846 0 0) 0x7f14880016f0 con 0x7f14980c9a30 2024-05-11T08:07:14.281 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:14.277+0000 7f149e2fc640 1 --2- 172.21.15.167:0/3137127041 >> v2:172.21.15.204:3300/0 conn(0x7f1498070710 0x7f14980cd500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T08:07:14.281 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:14.277+0000 7f149caf9640 1 -- 172.21.15.167:0/3137127041 >> v1:172.21.15.167:6789/0 conn(0x7f14980c9a30 legacy=0x7f1498070000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:14.281 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:14.277+0000 7f149caf9640 1 --2- 172.21.15.167:0/3137127041 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1484003a50 0x7f1484003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:14.281 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:14.277+0000 7f149caf9640 1 --2- 172.21.15.167:0/3137127041 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1484004800 0x7f1484004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:14.281 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:14.277+0000 7f149caf9640 1 --2- 172.21.15.167:0/3137127041 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f14840053c0 0x7f148400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:14.281 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:14.277+0000 7f149caf9640 1 -- 172.21.15.167:0/3137127041 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f14980d0c00 con 0x7f14840053c0 2024-05-11T08:07:14.281 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:14.277+0000 7f149caf9640 1 -- 172.21.15.167:0/3137127041 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f148400c250 con 0x7f1484004800 2024-05-11T08:07:14.281 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:14.277+0000 7f149caf9640 1 -- 172.21.15.167:0/3137127041 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f148400c450 con 0x7f1484003a50 2024-05-11T08:07:14.281 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:14.278+0000 7f149e2fc640 1 --2- 172.21.15.167:0/3137127041 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f14840053c0 0x7f148400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:14.281 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:14.278+0000 7f149d2fa640 1 --2- 172.21.15.167:0/3137127041 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1484004800 0x7f1484004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:14.281 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:14.278+0000 7f149dafb640 1 --2- 172.21.15.167:0/3137127041 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1484003a50 0x7f1484003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:14.281 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:14.278+0000 7f149e2fc640 1 -- 172.21.15.167:0/3137127041 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1484003a50 msgr2=0x7f1484003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:07:14.281 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:14.278+0000 7f149e2fc640 1 --2- 172.21.15.167:0/3137127041 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1484003a50 0x7f1484003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:14.281 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:14.278+0000 7f149e2fc640 1 -- 172.21.15.167:0/3137127041 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1484004800 msgr2=0x7f1484004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:07:14.282 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:14.278+0000 7f149e2fc640 1 --2- 172.21.15.167:0/3137127041 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1484004800 0x7f1484004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:14.282 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:14.278+0000 7f149e2fc640 1 -- 172.21.15.167:0/3137127041 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1484003520 con 0x7f14840053c0 2024-05-11T08:07:14.282 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:14.278+0000 7f149e2fc640 1 -- 172.21.15.167:0/3137127041 reap_dead start 2024-05-11T08:07:14.282 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:14.278+0000 7f149e2fc640 1 --2- 172.21.15.167:0/3137127041 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f14840053c0 0x7f148400b9c0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f14900073e0 tx=0x7f1484005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f8ca1567538454b server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:14.282 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:14.279+0000 7f149caf9640 1 -- 172.21.15.167:0/3137127041 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1490009c80 con 0x7f14840053c0 2024-05-11T08:07:14.282 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:14.279+0000 7f149caf9640 1 -- 172.21.15.167:0/3137127041 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f149002c6c0 con 0x7f14840053c0 2024-05-11T08:07:14.282 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:14.279+0000 7f149caf9640 1 -- 172.21.15.167:0/3137127041 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f149002cab0 con 0x7f14840053c0 2024-05-11T08:07:14.282 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:14.279+0000 7f149fd5f640 1 -- 172.21.15.167:0/3137127041 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f14840053c0 msgr2=0x7f148400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:14.282 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:14.279+0000 7f149fd5f640 1 --2- 172.21.15.167:0/3137127041 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f14840053c0 0x7f148400b9c0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f14900073e0 tx=0x7f1484005830 comp rx=0 tx=0).stop 2024-05-11T08:07:14.282 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:14.279+0000 7f149fd5f640 1 -- 172.21.15.167:0/3137127041 shutdown_connections 2024-05-11T08:07:14.282 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:14.279+0000 7f149fd5f640 1 --2- 172.21.15.167:0/3137127041 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f14840053c0 0x7f148400b9c0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:14.282 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:14.279+0000 7f149fd5f640 1 -- 172.21.15.167:0/3137127041 >> 172.21.15.167:0/3137127041 conn(0x7f14980b4ae0 msgr2=0x7f14980b4ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:07:14.282 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:14.279+0000 7f149fd5f640 1 -- 172.21.15.167:0/3137127041 shutdown_connections 2024-05-11T08:07:14.283 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:14.279+0000 7f149fd5f640 1 -- 172.21.15.167:0/3137127041 wait complete. 2024-05-11T08:07:14.285 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:14.280+0000 7f149fd5f640 1 Processor -- start 2024-05-11T08:07:14.285 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:14.280+0000 7f149fd5f640 1 -- start start 2024-05-11T08:07:14.285 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:14.281+0000 7f149fd5f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f14840053c0 0x7f14980b5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:14.285 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:14.281+0000 7f149fd5f640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1498158900 0x7f14980b5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:14.285 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:14.281+0000 7f149fd5f640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1498157db0 0x7f1498158230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:14.285 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:14.281+0000 7f149fd5f640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1498070e70 con 0x7f1498158900 2024-05-11T08:07:14.286 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:14.281+0000 7f149fd5f640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1498070cf0 con 0x7f14840053c0 2024-05-11T08:07:14.286 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:14.281+0000 7f149fd5f640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f14980d2660 con 0x7f1498157db0 2024-05-11T08:07:14.286 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:14.281+0000 7f149d2fa640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1498158900 0x7f14980b5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:14.286 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:14.281+0000 7f149d2fa640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1498158900 0x7f14980b5130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:56018/0 (socket says 172.21.15.167:56018) 2024-05-11T08:07:14.286 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:14.281+0000 7f149d2fa640 1 -- 172.21.15.167:0/579600835 learned_addr learned my addr 172.21.15.167:0/579600835 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:07:14.286 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:14.281+0000 7f149dafb640 1 --2- 172.21.15.167:0/579600835 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f14840053c0 0x7f14980b5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:14.286 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:14.281+0000 7f149e2fc640 1 --2- 172.21.15.167:0/579600835 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1498157db0 0x7f1498158230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:14.286 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:14.282+0000 7f149d2fa640 1 -- 172.21.15.167:0/579600835 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1498157db0 msgr2=0x7f1498158230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:14.286 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:14.282+0000 7f149d2fa640 1 --2- 172.21.15.167:0/579600835 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1498157db0 0x7f1498158230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:14.286 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:14.282+0000 7f149d2fa640 1 -- 172.21.15.167:0/579600835 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f14840053c0 msgr2=0x7f14980b5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:14.286 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:14.282+0000 7f149d2fa640 1 --2- 172.21.15.167:0/579600835 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f14840053c0 0x7f14980b5bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:14.286 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:14.282+0000 7f149d2fa640 1 -- 172.21.15.167:0/579600835 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14980b22a0 con 0x7f1498158900 2024-05-11T08:07:14.286 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:14.282+0000 7f149d2fa640 1 --2- 172.21.15.167:0/579600835 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1498158900 0x7f14980b5130 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f1488007310 tx=0x7f14880077d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:14.286 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:14.282+0000 7f148e7fc640 1 -- 172.21.15.167:0/579600835 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f148800f9c0 con 0x7f1498158900 2024-05-11T08:07:14.287 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:14.282+0000 7f148e7fc640 1 -- 172.21.15.167:0/579600835 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1488018070 con 0x7f1498158900 2024-05-11T08:07:14.287 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:14.282+0000 7f149fd5f640 1 -- 172.21.15.167:0/579600835 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f149806e360 con 0x7f1498158900 2024-05-11T08:07:14.287 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:14.282+0000 7f148e7fc640 1 -- 172.21.15.167:0/579600835 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f148800fdc0 con 0x7f1498158900 2024-05-11T08:07:14.287 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:14.282+0000 7f149fd5f640 1 -- 172.21.15.167:0/579600835 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14980b1fc0 con 0x7f1498158900 2024-05-11T08:07:14.287 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:14.283+0000 7f149fd5f640 1 -- 172.21.15.167:0/579600835 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14980752e0 con 0x7f1498158900 2024-05-11T08:07:14.287 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:14.283+0000 7f148e7fc640 1 -- 172.21.15.167:0/579600835 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f148801d460 con 0x7f1498158900 2024-05-11T08:07:14.287 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:14.284+0000 7f148e7fc640 1 --2- 172.21.15.167:0/579600835 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f147c0762d0 0x7f147c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:14.287 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:14.284+0000 7f149dafb640 1 --2- 172.21.15.167:0/579600835 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f147c0762d0 0x7f147c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:14.287 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:14.284+0000 7f148e7fc640 1 -- 172.21.15.167:0/579600835 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5730+0+0 (secure 0 0 0) 0x7f1488097490 con 0x7f1498158900 2024-05-11T08:07:14.287 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:14.285+0000 7f149dafb640 1 --2- 172.21.15.167:0/579600835 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f147c0762d0 0x7f147c078790 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f1494000c00 tx=0x7f1494009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:14.288 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:14.289+0000 7f148e7fc640 1 -- 172.21.15.167:0/579600835 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f1488060f00 con 0x7f1498158900 2024-05-11T08:07:14.467 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:14.467+0000 7f149fd5f640 1 -- 172.21.15.167:0/579600835 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "data_new", "pool2": "data_new", "yes_i_really_really_mean_it": true} v 0) -- 0x7f1498069590 con 0x7f1498158900 2024-05-11T08:07:15.149 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.149+0000 7f148e7fc640 1 -- 172.21.15.167:0/579600835 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "data_new", "pool2": "data_new", "yes_i_really_really_mean_it": true}]=0 pool 'data_new' removed v48) ==== 160+0+0 (secure 0 0 0) 0x7f1488060800 con 0x7f1498158900 2024-05-11T08:07:15.149 INFO:teuthology.orchestra.run.smithi167.stderr:pool 'data_new' removed 2024-05-11T08:07:15.152 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.151+0000 7f149fd5f640 1 -- 172.21.15.167:0/579600835 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f147c0762d0 msgr2=0x7f147c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:15.152 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.151+0000 7f149fd5f640 1 --2- 172.21.15.167:0/579600835 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f147c0762d0 0x7f147c078790 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f1494000c00 tx=0x7f1494009040 comp rx=0 tx=0).stop 2024-05-11T08:07:15.152 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.151+0000 7f149fd5f640 1 -- 172.21.15.167:0/579600835 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1498158900 msgr2=0x7f14980b5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:15.152 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.151+0000 7f149fd5f640 1 --2- 172.21.15.167:0/579600835 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1498158900 0x7f14980b5130 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f1488007310 tx=0x7f14880077d0 comp rx=0 tx=0).stop 2024-05-11T08:07:15.152 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.151+0000 7f149fd5f640 1 -- 172.21.15.167:0/579600835 shutdown_connections 2024-05-11T08:07:15.152 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.151+0000 7f149fd5f640 1 --2- 172.21.15.167:0/579600835 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f147c0762d0 0x7f147c078790 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:15.153 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.152+0000 7f149fd5f640 1 --2- 172.21.15.167:0/579600835 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1498157db0 0x7f1498158230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:15.153 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.152+0000 7f149fd5f640 1 --2- 172.21.15.167:0/579600835 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1498158900 0x7f14980b5130 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:15.153 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.152+0000 7f149fd5f640 1 --2- 172.21.15.167:0/579600835 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f14840053c0 0x7f14980b5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:15.153 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.152+0000 7f149fd5f640 1 -- 172.21.15.167:0/579600835 >> 172.21.15.167:0/579600835 conn(0x7f14980b4ae0 msgr2=0x7f1498070000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:07:15.153 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.152+0000 7f149fd5f640 1 -- 172.21.15.167:0/579600835 shutdown_connections 2024-05-11T08:07:15.153 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.152+0000 7f149fd5f640 1 -- 172.21.15.167:0/579600835 wait complete. 2024-05-11T08:07:15.167 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_symlink_recovery 2024-05-11T08:07:15.271 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.271+0000 7f160c2e1640 1 Processor -- start 2024-05-11T08:07:15.272 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.272+0000 7f160c2e1640 1 -- start start 2024-05-11T08:07:15.274 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.272+0000 7f160c2e1640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f160415f910 0x7f160406c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:15.274 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.272+0000 7f160c2e1640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f160406cc30 0x7f160415fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:15.274 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.273+0000 7f160c2e1640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f160406dcc0 con 0x7f1604160230 2024-05-11T08:07:15.274 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.273+0000 7f160c2e1640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f1604156820 con 0x7f160415f910 2024-05-11T08:07:15.274 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.273+0000 7f160c2e1640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f160406d140 con 0x7f160406cc30 2024-05-11T08:07:15.274 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.273+0000 7f160a07d640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f160415f910 0x7f160406c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:15.274 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.273+0000 7f160a07d640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f160415f910 0x7f160406c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:56562/0 (socket says 172.21.15.167:56562) 2024-05-11T08:07:15.274 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.273+0000 7f160a07d640 1 -- 172.21.15.167:0/769482339 learned_addr learned my addr 172.21.15.167:0/769482339 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:07:15.274 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.273+0000 7f160987c640 1 --2- 172.21.15.167:0/769482339 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f160406cc30 0x7f160415fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:15.274 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.273+0000 7f160a07d640 1 -- 172.21.15.167:0/769482339 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f160406cc30 msgr2=0x7f160415fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:07:15.274 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.273+0000 7f160a07d640 1 --2- 172.21.15.167:0/769482339 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f160406cc30 0x7f160415fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:15.275 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.273+0000 7f160a07d640 1 -- 172.21.15.167:0/769482339 >> v1:172.21.15.204:6789/0 conn(0x7f1604160230 legacy=0x7f16040d3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:15.275 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.273+0000 7f160a07d640 1 -- 172.21.15.167:0/769482339 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f160406d420 con 0x7f160415f910 2024-05-11T08:07:15.275 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.274+0000 7f160a07d640 1 --2- 172.21.15.167:0/769482339 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f160415f910 0x7f160406c6f0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f1600009870 tx=0x7f160002a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2fd540dff40ce8c server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:15.275 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.274+0000 7f160907b640 1 -- 172.21.15.167:0/769482339 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1600037070 con 0x7f160415f910 2024-05-11T08:07:15.275 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.274+0000 7f160907b640 1 -- 172.21.15.167:0/769482339 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1600032440 con 0x7f160415f910 2024-05-11T08:07:15.275 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.274+0000 7f160907b640 1 -- 172.21.15.167:0/769482339 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1600032810 con 0x7f160415f910 2024-05-11T08:07:15.278 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.274+0000 7f160c2e1640 1 -- 172.21.15.167:0/769482339 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f160415f910 msgr2=0x7f160406c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:15.278 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.274+0000 7f160c2e1640 1 --2- 172.21.15.167:0/769482339 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f160415f910 0x7f160406c6f0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f1600009870 tx=0x7f160002a250 comp rx=0 tx=0).stop 2024-05-11T08:07:15.278 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.274+0000 7f160c2e1640 1 -- 172.21.15.167:0/769482339 shutdown_connections 2024-05-11T08:07:15.278 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.274+0000 7f160c2e1640 1 --2- 172.21.15.167:0/769482339 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f160406cc30 0x7f160415fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:15.278 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.274+0000 7f160c2e1640 1 --2- 172.21.15.167:0/769482339 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f160415f910 0x7f160406c6f0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:15.278 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.274+0000 7f160c2e1640 1 -- 172.21.15.167:0/769482339 >> 172.21.15.167:0/769482339 conn(0x7f16040d51e0 msgr2=0x7f16040d55f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:07:15.278 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.274+0000 7f160c2e1640 1 -- 172.21.15.167:0/769482339 shutdown_connections 2024-05-11T08:07:15.278 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.275+0000 7f160c2e1640 1 -- 172.21.15.167:0/769482339 wait complete. 2024-05-11T08:07:15.278 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.276+0000 7f160c2e1640 1 Processor -- start 2024-05-11T08:07:15.279 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.276+0000 7f160c2e1640 1 -- start start 2024-05-11T08:07:15.279 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.276+0000 7f160c2e1640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f160406cc30 0x7f16040c9f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:15.279 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.276+0000 7f160c2e1640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f160415f910 0x7f16041525c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:15.279 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.276+0000 7f160c2e1640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1604160230 0x7f16040cc2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:15.279 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.276+0000 7f160c2e1640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f16041606b0 con 0x7f160415f910 2024-05-11T08:07:15.279 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.276+0000 7f160c2e1640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f160406d9e0 con 0x7f1604160230 2024-05-11T08:07:15.279 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.276+0000 7f160c2e1640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f16040d0060 con 0x7f160406cc30 2024-05-11T08:07:15.279 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.276+0000 7f160987c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f160415f910 0x7f16041525c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:15.279 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.276+0000 7f160987c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f160415f910 0x7f16041525c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:56566/0 (socket says 172.21.15.167:56566) 2024-05-11T08:07:15.279 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.277+0000 7f160987c640 1 -- 172.21.15.167:0/4091935975 learned_addr learned my addr 172.21.15.167:0/4091935975 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:07:15.279 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.277+0000 7f160a07d640 1 --2- 172.21.15.167:0/4091935975 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f160406cc30 0x7f16040c9f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:15.279 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.277+0000 7f160a87e640 1 --2- 172.21.15.167:0/4091935975 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1604160230 0x7f16040cc2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:15.279 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.277+0000 7f160987c640 1 -- 172.21.15.167:0/4091935975 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f160406cc30 msgr2=0x7f16040c9f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:15.279 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.277+0000 7f160987c640 1 --2- 172.21.15.167:0/4091935975 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f160406cc30 0x7f16040c9f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:15.280 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.277+0000 7f160987c640 1 -- 172.21.15.167:0/4091935975 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1604160230 msgr2=0x7f16040cc2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:15.280 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.277+0000 7f160987c640 1 --2- 172.21.15.167:0/4091935975 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1604160230 0x7f16040cc2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:15.280 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.277+0000 7f160987c640 1 -- 172.21.15.167:0/4091935975 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16040cf970 con 0x7f160415f910 2024-05-11T08:07:15.280 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.277+0000 7f160a87e640 1 --2- 172.21.15.167:0/4091935975 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1604160230 0x7f16040cc2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T08:07:15.280 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.277+0000 7f160987c640 1 --2- 172.21.15.167:0/4091935975 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f160415f910 0x7f16041525c0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f15f400d5a0 tx=0x7f15f400da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:15.280 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.277+0000 7f15faffd640 1 -- 172.21.15.167:0/4091935975 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15f40125f0 con 0x7f160415f910 2024-05-11T08:07:15.280 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.277+0000 7f15faffd640 1 -- 172.21.15.167:0/4091935975 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15f4019070 con 0x7f160415f910 2024-05-11T08:07:15.280 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.277+0000 7f160c2e1640 1 -- 172.21.15.167:0/4091935975 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1604157f90 con 0x7f160415f910 2024-05-11T08:07:15.280 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.277+0000 7f15faffd640 1 -- 172.21.15.167:0/4091935975 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15f40129b0 con 0x7f160415f910 2024-05-11T08:07:15.280 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.277+0000 7f160c2e1640 1 -- 172.21.15.167:0/4091935975 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1604158550 con 0x7f160415f910 2024-05-11T08:07:15.280 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.278+0000 7f160c2e1640 1 -- 172.21.15.167:0/4091935975 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1604069520 con 0x7f160415f910 2024-05-11T08:07:15.281 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.279+0000 7f15faffd640 1 -- 172.21.15.167:0/4091935975 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f15f4012b50 con 0x7f160415f910 2024-05-11T08:07:15.281 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.279+0000 7f15faffd640 1 --2- 172.21.15.167:0/4091935975 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f15e80762d0 0x7f15e8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:15.281 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.279+0000 7f15faffd640 1 -- 172.21.15.167:0/4091935975 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5335+0+0 (secure 0 0 0) 0x7f15f40977f0 con 0x7f160415f910 2024-05-11T08:07:15.281 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.279+0000 7f160a07d640 1 --2- 172.21.15.167:0/4091935975 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f15e80762d0 0x7f15e8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:15.281 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.280+0000 7f160a07d640 1 --2- 172.21.15.167:0/4091935975 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f15e80762d0 0x7f15e8078790 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f1600034000 tx=0x7f160002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:15.283 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.284+0000 7f15faffd640 1 -- 172.21.15.167:0/4091935975 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f15f40612c0 con 0x7f160415f910 2024-05-11T08:07:15.463 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.463+0000 7f160c2e1640 1 -- 172.21.15.167:0/4091935975 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_symlink_recovery"} v 0) -- 0x7f16040c8450 con 0x7f160415f910 2024-05-11T08:07:15.466 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.466+0000 7f15faffd640 1 -- 172.21.15.167:0/4091935975 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_symlink_recovery"}]=0 v13) ==== 103+0+0 (secure 0 0 0) 0x7f15f4060bc0 con 0x7f160415f910 2024-05-11T08:07:15.469 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.468+0000 7f160c2e1640 1 -- 172.21.15.167:0/4091935975 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f15e80762d0 msgr2=0x7f15e8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:15.469 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.468+0000 7f160c2e1640 1 --2- 172.21.15.167:0/4091935975 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f15e80762d0 0x7f15e8078790 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f1600034000 tx=0x7f160002e000 comp rx=0 tx=0).stop 2024-05-11T08:07:15.469 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.468+0000 7f160c2e1640 1 -- 172.21.15.167:0/4091935975 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f160415f910 msgr2=0x7f16041525c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:15.469 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.468+0000 7f160c2e1640 1 --2- 172.21.15.167:0/4091935975 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f160415f910 0x7f16041525c0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f15f400d5a0 tx=0x7f15f400da60 comp rx=0 tx=0).stop 2024-05-11T08:07:15.469 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.468+0000 7f160c2e1640 1 -- 172.21.15.167:0/4091935975 shutdown_connections 2024-05-11T08:07:15.469 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.468+0000 7f160c2e1640 1 --2- 172.21.15.167:0/4091935975 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f15e80762d0 0x7f15e8078790 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:15.469 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.468+0000 7f160c2e1640 1 --2- 172.21.15.167:0/4091935975 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1604160230 0x7f16040cc2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:15.469 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.468+0000 7f160c2e1640 1 --2- 172.21.15.167:0/4091935975 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f160415f910 0x7f16041525c0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:15.470 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.468+0000 7f160c2e1640 1 --2- 172.21.15.167:0/4091935975 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f160406cc30 0x7f16040c9f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:15.470 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.468+0000 7f160c2e1640 1 -- 172.21.15.167:0/4091935975 >> 172.21.15.167:0/4091935975 conn(0x7f16040d51e0 msgr2=0x7f16040b97f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:07:15.470 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.469+0000 7f160c2e1640 1 -- 172.21.15.167:0/4091935975 shutdown_connections 2024-05-11T08:07:15.470 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.469+0000 7f160c2e1640 1 -- 172.21.15.167:0/4091935975 wait complete. 2024-05-11T08:07:15.480 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-05-11T08:07:15.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.582+0000 7f7b6eebd640 1 Processor -- start 2024-05-11T08:07:15.706 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.582+0000 7f7b6eebd640 1 -- start start 2024-05-11T08:07:15.711 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.583+0000 7f7b6eebd640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f7b68147ff0 0x7f7b680d24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:15.711 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.583+0000 7f7b6eebd640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f7b68148cf0 0x7f7b680b5de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:15.711 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.583+0000 7f7b6eebd640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7b680d1430 con 0x7f7b680d2e80 2024-05-11T08:07:15.711 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.583+0000 7f7b6eebd640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f7b680d2040 con 0x7f7b68147ff0 2024-05-11T08:07:15.711 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.583+0000 7f7b6eebd640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f7b680d1710 con 0x7f7b68148cf0 2024-05-11T08:07:15.711 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.583+0000 7f7b6cc59640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f7b68147ff0 0x7f7b680d24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:15.711 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.583+0000 7f7b6cc59640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f7b68147ff0 0x7f7b680d24d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:56580/0 (socket says 172.21.15.167:56580) 2024-05-11T08:07:15.711 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.583+0000 7f7b6cc59640 1 -- 172.21.15.167:0/1862781302 learned_addr learned my addr 172.21.15.167:0/1862781302 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:07:15.712 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.583+0000 7f7b67fff640 1 --2- 172.21.15.167:0/1862781302 >> v2:172.21.15.204:3300/0 conn(0x7f7b68148cf0 0x7f7b680b5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:15.712 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.583+0000 7f7b6cc59640 1 -- 172.21.15.167:0/1862781302 >> v2:172.21.15.204:3300/0 conn(0x7f7b68148cf0 msgr2=0x7f7b680b5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:07:15.712 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.583+0000 7f7b6cc59640 1 --2- 172.21.15.167:0/1862781302 >> v2:172.21.15.204:3300/0 conn(0x7f7b68148cf0 0x7f7b680b5de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:15.712 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.583+0000 7f7b6cc59640 1 -- 172.21.15.167:0/1862781302 >> v1:172.21.15.204:6789/0 conn(0x7f7b680d2e80 legacy=0x7f7b6815f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:07:15.712 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.583+0000 7f7b6cc59640 1 -- 172.21.15.167:0/1862781302 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b680d19f0 con 0x7f7b68147ff0 2024-05-11T08:07:15.712 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.584+0000 7f7b6cc59640 1 --2- 172.21.15.167:0/1862781302 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7b68147ff0 0x7f7b680d24d0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f7b68072060 tx=0x7f7b5c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49073b4ecf71a1b2 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:15.712 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.584+0000 7f7b677fe640 1 -- 172.21.15.167:0/1862781302 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b5c037070 con 0x7f7b68147ff0 2024-05-11T08:07:15.712 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.584+0000 7f7b677fe640 1 -- 172.21.15.167:0/1862781302 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b5c032440 con 0x7f7b68147ff0 2024-05-11T08:07:15.712 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.584+0000 7f7b677fe640 1 -- 172.21.15.167:0/1862781302 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b5c032810 con 0x7f7b68147ff0 2024-05-11T08:07:15.712 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.585+0000 7f7b6eebd640 1 -- 172.21.15.167:0/1862781302 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7b68147ff0 msgr2=0x7f7b680d24d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:15.712 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.585+0000 7f7b6eebd640 1 --2- 172.21.15.167:0/1862781302 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7b68147ff0 0x7f7b680d24d0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f7b68072060 tx=0x7f7b5c02a270 comp rx=0 tx=0).stop 2024-05-11T08:07:15.713 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.585+0000 7f7b6eebd640 1 -- 172.21.15.167:0/1862781302 shutdown_connections 2024-05-11T08:07:15.713 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.585+0000 7f7b6eebd640 1 --2- 172.21.15.167:0/1862781302 >> v2:172.21.15.204:3300/0 conn(0x7f7b68148cf0 0x7f7b680b5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:15.713 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.585+0000 7f7b6eebd640 1 --2- 172.21.15.167:0/1862781302 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7b68147ff0 0x7f7b680d24d0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:15.713 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.585+0000 7f7b6eebd640 1 -- 172.21.15.167:0/1862781302 >> 172.21.15.167:0/1862781302 conn(0x7f7b680b8300 msgr2=0x7f7b680b86e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:07:15.713 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.585+0000 7f7b6eebd640 1 -- 172.21.15.167:0/1862781302 shutdown_connections 2024-05-11T08:07:15.713 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.585+0000 7f7b6eebd640 1 -- 172.21.15.167:0/1862781302 wait complete. 2024-05-11T08:07:15.713 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.586+0000 7f7b6eebd640 1 Processor -- start 2024-05-11T08:07:15.713 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.586+0000 7f7b6eebd640 1 -- start start 2024-05-11T08:07:15.713 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.587+0000 7f7b6eebd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7b680d2e80 0x7f7b6815eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:15.713 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.587+0000 7f7b6eebd640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7b68147ff0 0x7f7b68143cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:15.713 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.587+0000 7f7b6eebd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7b68148cf0 0x7f7b680ce7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:15.713 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.587+0000 7f7b6eebd640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7b680d2a10 con 0x7f7b68148cf0 2024-05-11T08:07:15.713 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.587+0000 7f7b6eebd640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7b681483d0 con 0x7f7b680d2e80 2024-05-11T08:07:15.713 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.587+0000 7f7b6eebd640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f7b680b6320 con 0x7f7b68147ff0 2024-05-11T08:07:15.714 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.587+0000 7f7b6d45a640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7b68148cf0 0x7f7b680ce7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:15.714 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.587+0000 7f7b6d45a640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7b68148cf0 0x7f7b680ce7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:56596/0 (socket says 172.21.15.167:56596) 2024-05-11T08:07:15.714 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.587+0000 7f7b6d45a640 1 -- 172.21.15.167:0/3312522867 learned_addr learned my addr 172.21.15.167:0/3312522867 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:07:15.714 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.587+0000 7f7b67fff640 1 --2- 172.21.15.167:0/3312522867 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7b68147ff0 0x7f7b68143cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:15.714 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.587+0000 7f7b6cc59640 1 --2- 172.21.15.167:0/3312522867 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7b680d2e80 0x7f7b6815eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:15.714 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.587+0000 7f7b6d45a640 1 -- 172.21.15.167:0/3312522867 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7b68147ff0 msgr2=0x7f7b68143cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:15.714 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.587+0000 7f7b6d45a640 1 --2- 172.21.15.167:0/3312522867 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7b68147ff0 0x7f7b68143cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:15.714 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.587+0000 7f7b6d45a640 1 -- 172.21.15.167:0/3312522867 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7b680d2e80 msgr2=0x7f7b6815eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:15.714 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.587+0000 7f7b6d45a640 1 --2- 172.21.15.167:0/3312522867 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7b680d2e80 0x7f7b6815eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:15.714 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.587+0000 7f7b6d45a640 1 -- 172.21.15.167:0/3312522867 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b680ccd40 con 0x7f7b68148cf0 2024-05-11T08:07:15.715 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.587+0000 7f7b6cc59640 1 --2- 172.21.15.167:0/3312522867 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7b680d2e80 0x7f7b6815eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T08:07:15.715 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.587+0000 7f7b6d45a640 1 --2- 172.21.15.167:0/3312522867 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7b68148cf0 0x7f7b680ce7b0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f7b6000c700 tx=0x7f7b6000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:15.715 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.587+0000 7f7b657fa640 1 -- 172.21.15.167:0/3312522867 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b60003b70 con 0x7f7b68148cf0 2024-05-11T08:07:15.715 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.588+0000 7f7b657fa640 1 -- 172.21.15.167:0/3312522867 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b60003d10 con 0x7f7b68148cf0 2024-05-11T08:07:15.715 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.588+0000 7f7b657fa640 1 -- 172.21.15.167:0/3312522867 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b6000aa70 con 0x7f7b68148cf0 2024-05-11T08:07:15.715 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.588+0000 7f7b6eebd640 1 -- 172.21.15.167:0/3312522867 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b680cd020 con 0x7f7b68148cf0 2024-05-11T08:07:15.715 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.588+0000 7f7b6eebd640 1 -- 172.21.15.167:0/3312522867 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b680cd5e0 con 0x7f7b68148cf0 2024-05-11T08:07:15.715 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.588+0000 7f7b6eebd640 1 -- 172.21.15.167:0/3312522867 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b680b90c0 con 0x7f7b68148cf0 2024-05-11T08:07:15.715 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.589+0000 7f7b657fa640 1 -- 172.21.15.167:0/3312522867 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f7b6000ac10 con 0x7f7b68148cf0 2024-05-11T08:07:15.715 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.590+0000 7f7b657fa640 1 --2- 172.21.15.167:0/3312522867 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f7b4c0762d0 0x7f7b4c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:15.715 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.590+0000 7f7b657fa640 1 -- 172.21.15.167:0/3312522867 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5335+0+0 (secure 0 0 0) 0x7f7b60097300 con 0x7f7b68148cf0 2024-05-11T08:07:15.715 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.590+0000 7f7b6cc59640 1 --2- 172.21.15.167:0/3312522867 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f7b4c0762d0 0x7f7b4c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:15.715 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.590+0000 7f7b6cc59640 1 --2- 172.21.15.167:0/3312522867 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f7b4c0762d0 0x7f7b4c078790 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c034000 tx=0x7f7b5c02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:15.715 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.594+0000 7f7b657fa640 1 -- 172.21.15.167:0/3312522867 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f7b680b90c0 con 0x7f7b68148cf0 2024-05-11T08:07:15.777 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.776+0000 7f7b6eebd640 1 -- 172.21.15.167:0/3312522867 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f7b68069520 con 0x7f7b68148cf0 2024-05-11T08:07:15.782 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.782+0000 7f7b657fa640 1 -- 172.21.15.167:0/3312522867 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v14) ==== 104+0+0 (secure 0 0 0) 0x7f7b60060dd0 con 0x7f7b68148cf0 2024-05-11T08:07:15.785 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.784+0000 7f7b6eebd640 1 -- 172.21.15.167:0/3312522867 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f7b4c0762d0 msgr2=0x7f7b4c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:15.785 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.784+0000 7f7b6eebd640 1 --2- 172.21.15.167:0/3312522867 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f7b4c0762d0 0x7f7b4c078790 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c034000 tx=0x7f7b5c02e000 comp rx=0 tx=0).stop 2024-05-11T08:07:15.785 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.784+0000 7f7b6eebd640 1 -- 172.21.15.167:0/3312522867 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7b68148cf0 msgr2=0x7f7b680ce7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:15.785 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.784+0000 7f7b6eebd640 1 --2- 172.21.15.167:0/3312522867 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7b68148cf0 0x7f7b680ce7b0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f7b6000c700 tx=0x7f7b6000cbc0 comp rx=0 tx=0).stop 2024-05-11T08:07:15.785 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.784+0000 7f7b6eebd640 1 -- 172.21.15.167:0/3312522867 shutdown_connections 2024-05-11T08:07:15.785 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.784+0000 7f7b6eebd640 1 --2- 172.21.15.167:0/3312522867 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f7b4c0762d0 0x7f7b4c078790 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:15.785 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.784+0000 7f7b6eebd640 1 --2- 172.21.15.167:0/3312522867 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7b68148cf0 0x7f7b680ce7b0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:15.785 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.784+0000 7f7b6eebd640 1 --2- 172.21.15.167:0/3312522867 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7b68147ff0 0x7f7b68143cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:15.786 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.784+0000 7f7b6eebd640 1 --2- 172.21.15.167:0/3312522867 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7b680d2e80 0x7f7b6815eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:15.786 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.784+0000 7f7b6eebd640 1 -- 172.21.15.167:0/3312522867 >> 172.21.15.167:0/3312522867 conn(0x7f7b680b8300 msgr2=0x7f7b68141ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:07:15.786 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.784+0000 7f7b6eebd640 1 -- 172.21.15.167:0/3312522867 shutdown_connections 2024-05-11T08:07:15.786 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.784+0000 7f7b6eebd640 1 -- 172.21.15.167:0/3312522867 wait complete. 2024-05-11T08:07:15.796 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_backtrace.TestBacktrace.test_backtrace' 2024-05-11T08:07:15.900 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.898+0000 7f1b2d7c1640 1 Processor -- start 2024-05-11T08:07:15.900 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.898+0000 7f1b2d7c1640 1 -- start start 2024-05-11T08:07:15.901 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.899+0000 7f1b2d7c1640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1b280cf390 0x7f1b280d1470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:15.902 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.899+0000 7f1b2d7c1640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f1b28070650 0x7f1b28165a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:15.902 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.899+0000 7f1b2d7c1640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b28145ca0 con 0x7f1b280cf770 2024-05-11T08:07:15.902 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.899+0000 7f1b2d7c1640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f1b280b5960 con 0x7f1b28070650 2024-05-11T08:07:15.902 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.899+0000 7f1b2d7c1640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f1b280b5c40 con 0x7f1b280cf390 2024-05-11T08:07:15.902 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.899+0000 7f1b277fe640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f1b28070650 0x7f1b28165a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:15.902 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.899+0000 7f1b267fc640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f1b280cf770 0x7f1b280cfbd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:59718/0 (socket says 172.21.15.167:59718) 2024-05-11T08:07:15.902 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.899+0000 7f1b267fc640 1 -- 172.21.15.167:0/2803930335 learned_addr learned my addr 172.21.15.167:0/2803930335 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T08:07:15.902 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.900+0000 7f1b26ffd640 1 --2- 172.21.15.167:0/2803930335 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1b280cf390 0x7f1b280d1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:15.902 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.900+0000 7f1b277fe640 1 -- 172.21.15.167:0/2803930335 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1b280cf390 msgr2=0x7f1b280d1470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:07:15.902 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.900+0000 7f1b277fe640 1 --2- 172.21.15.167:0/2803930335 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1b280cf390 0x7f1b280d1470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:15.902 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.900+0000 7f1b25ffb640 1 -- 172.21.15.167:0/2803930335 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1847238002 0 0) 0x7f1b28145ca0 con 0x7f1b280cf770 2024-05-11T08:07:15.903 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.900+0000 7f1b277fe640 1 -- 172.21.15.167:0/2803930335 >> v1:172.21.15.167:6789/0 conn(0x7f1b280cf770 legacy=0x7f1b280cfbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:15.903 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.900+0000 7f1b277fe640 1 -- 172.21.15.167:0/2803930335 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b280b5f20 con 0x7f1b28070650 2024-05-11T08:07:15.903 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.900+0000 7f1b26ffd640 1 --2- 172.21.15.167:0/2803930335 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1b280cf390 0x7f1b280d1470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T08:07:15.903 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.900+0000 7f1b277fe640 1 --2- 172.21.15.167:0/2803930335 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b28070650 0x7f1b28165a80 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f1b2000ba40 tx=0x7f1b2002c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93e26521193381c6 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:15.903 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.901+0000 7f1b25ffb640 1 -- 172.21.15.167:0/2803930335 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b20031710 con 0x7f1b28070650 2024-05-11T08:07:15.903 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.901+0000 7f1b25ffb640 1 -- 172.21.15.167:0/2803930335 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b200318b0 con 0x7f1b28070650 2024-05-11T08:07:15.903 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.901+0000 7f1b25ffb640 1 -- 172.21.15.167:0/2803930335 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b20031c80 con 0x7f1b28070650 2024-05-11T08:07:15.906 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.901+0000 7f1b2d7c1640 1 -- 172.21.15.167:0/2803930335 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b28070650 msgr2=0x7f1b28165a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:15.906 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.901+0000 7f1b2d7c1640 1 --2- 172.21.15.167:0/2803930335 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b28070650 0x7f1b28165a80 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f1b2000ba40 tx=0x7f1b2002c7b0 comp rx=0 tx=0).stop 2024-05-11T08:07:15.906 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.901+0000 7f1b2d7c1640 1 -- 172.21.15.167:0/2803930335 shutdown_connections 2024-05-11T08:07:15.906 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.901+0000 7f1b2d7c1640 1 --2- 172.21.15.167:0/2803930335 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b28070650 0x7f1b28165a80 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:15.906 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.901+0000 7f1b2d7c1640 1 --2- 172.21.15.167:0/2803930335 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1b280cf390 0x7f1b280d1470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:15.906 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.901+0000 7f1b2d7c1640 1 -- 172.21.15.167:0/2803930335 >> 172.21.15.167:0/2803930335 conn(0x7f1b280d1e40 msgr2=0x7f1b280d2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:07:15.907 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.901+0000 7f1b2d7c1640 1 -- 172.21.15.167:0/2803930335 shutdown_connections 2024-05-11T08:07:15.907 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.901+0000 7f1b2d7c1640 1 -- 172.21.15.167:0/2803930335 wait complete. 2024-05-11T08:07:15.907 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.903+0000 7f1b2d7c1640 1 Processor -- start 2024-05-11T08:07:15.907 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.903+0000 7f1b2d7c1640 1 -- start start 2024-05-11T08:07:15.907 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.903+0000 7f1b2d7c1640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1b28070650 0x7f1b2813e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:15.907 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.903+0000 7f1b2d7c1640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b280cf390 0x7f1b2813b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:15.907 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.903+0000 7f1b2d7c1640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1b280cf770 0x7f1b2813ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:15.907 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.903+0000 7f1b2d7c1640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1b280d19b0 con 0x7f1b280cf390 2024-05-11T08:07:15.907 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.903+0000 7f1b2d7c1640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1b280b64e0 con 0x7f1b28070650 2024-05-11T08:07:15.907 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.903+0000 7f1b2d7c1640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f1b280d1b30 con 0x7f1b280cf770 2024-05-11T08:07:15.907 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.903+0000 7f1b267fc640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b280cf390 0x7f1b2813b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:15.907 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.903+0000 7f1b267fc640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b280cf390 0x7f1b2813b660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:56608/0 (socket says 172.21.15.167:56608) 2024-05-11T08:07:15.907 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.903+0000 7f1b267fc640 1 -- 172.21.15.167:0/2507576217 learned_addr learned my addr 172.21.15.167:0/2507576217 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:07:15.907 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.904+0000 7f1b26ffd640 1 --2- 172.21.15.167:0/2507576217 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1b28070650 0x7f1b2813e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:15.908 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.904+0000 7f1b277fe640 1 --2- 172.21.15.167:0/2507576217 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1b280cf770 0x7f1b2813ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:15.908 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.904+0000 7f1b267fc640 1 -- 172.21.15.167:0/2507576217 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1b280cf770 msgr2=0x7f1b2813ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:15.908 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.904+0000 7f1b267fc640 1 --2- 172.21.15.167:0/2507576217 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1b280cf770 0x7f1b2813ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:15.908 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.904+0000 7f1b267fc640 1 -- 172.21.15.167:0/2507576217 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1b28070650 msgr2=0x7f1b2813e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:15.908 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.904+0000 7f1b267fc640 1 --2- 172.21.15.167:0/2507576217 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1b28070650 0x7f1b2813e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:15.908 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.904+0000 7f1b267fc640 1 -- 172.21.15.167:0/2507576217 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b2815bc80 con 0x7f1b280cf390 2024-05-11T08:07:15.908 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.904+0000 7f1b267fc640 1 --2- 172.21.15.167:0/2507576217 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b280cf390 0x7f1b2813b660 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f1b1c00c410 tx=0x7f1b1c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:15.908 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.904+0000 7f1b0f7fe640 1 -- 172.21.15.167:0/2507576217 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b1c0074e0 con 0x7f1b280cf390 2024-05-11T08:07:15.908 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.904+0000 7f1b0f7fe640 1 -- 172.21.15.167:0/2507576217 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b1c0047e0 con 0x7f1b280cf390 2024-05-11T08:07:15.908 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.904+0000 7f1b2d7c1640 1 -- 172.21.15.167:0/2507576217 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b2813bdb0 con 0x7f1b280cf390 2024-05-11T08:07:15.908 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.904+0000 7f1b0f7fe640 1 -- 172.21.15.167:0/2507576217 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b1c004bd0 con 0x7f1b280cf390 2024-05-11T08:07:15.908 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.904+0000 7f1b2d7c1640 1 -- 172.21.15.167:0/2507576217 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b2813c370 con 0x7f1b280cf390 2024-05-11T08:07:15.908 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.905+0000 7f1b2d7c1640 1 -- 172.21.15.167:0/2507576217 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b28069bc0 con 0x7f1b280cf390 2024-05-11T08:07:15.909 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.906+0000 7f1b0f7fe640 1 -- 172.21.15.167:0/2507576217 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f1b1c011070 con 0x7f1b280cf390 2024-05-11T08:07:15.909 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.906+0000 7f1b0f7fe640 1 --2- 172.21.15.167:0/2507576217 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f1b140762d0 0x7f1b14078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:15.909 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.906+0000 7f1b0f7fe640 1 -- 172.21.15.167:0/2507576217 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5335+0+0 (secure 0 0 0) 0x7f1b1c096ff0 con 0x7f1b280cf390 2024-05-11T08:07:15.909 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.906+0000 7f1b26ffd640 1 --2- 172.21.15.167:0/2507576217 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f1b140762d0 0x7f1b14078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:15.910 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.907+0000 7f1b26ffd640 1 --2- 172.21.15.167:0/2507576217 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f1b140762d0 0x7f1b14078790 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f1b180075b0 tx=0x7f1b18007540 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:15.911 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:15.911+0000 7f1b0f7fe640 1 -- 172.21.15.167:0/2507576217 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f1b2815bc80 con 0x7f1b280cf390 2024-05-11T08:07:16.163 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:16.162+0000 7f1b2d7c1640 1 -- 172.21.15.167:0/2507576217 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_backtrace.TestBacktrace.test_backtrace"]} v 0) -- 0x7f1b280ce2c0 con 0x7f1b280cf390 2024-05-11T08:07:17.161 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:17.160+0000 7f1b0f7fe640 1 -- 172.21.15.167:0/2507576217 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_backtrace.TestBacktrace.test_backtrace"]}]=0 v97) ==== 135+0+0 (secure 0 0 0) 0x7f1b1c060ac0 con 0x7f1b280cf390 2024-05-11T08:07:17.164 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:17.163+0000 7f1b2d7c1640 1 -- 172.21.15.167:0/2507576217 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f1b140762d0 msgr2=0x7f1b14078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:17.164 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:17.163+0000 7f1b2d7c1640 1 --2- 172.21.15.167:0/2507576217 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f1b140762d0 0x7f1b14078790 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f1b180075b0 tx=0x7f1b18007540 comp rx=0 tx=0).stop 2024-05-11T08:07:17.164 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:17.163+0000 7f1b2d7c1640 1 -- 172.21.15.167:0/2507576217 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b280cf390 msgr2=0x7f1b2813b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:17.165 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:17.163+0000 7f1b2d7c1640 1 --2- 172.21.15.167:0/2507576217 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b280cf390 0x7f1b2813b660 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f1b1c00c410 tx=0x7f1b1c00c8d0 comp rx=0 tx=0).stop 2024-05-11T08:07:17.165 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:17.163+0000 7f1b2d7c1640 1 -- 172.21.15.167:0/2507576217 shutdown_connections 2024-05-11T08:07:17.165 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:17.163+0000 7f1b2d7c1640 1 --2- 172.21.15.167:0/2507576217 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f1b140762d0 0x7f1b14078790 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:17.165 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:17.163+0000 7f1b2d7c1640 1 --2- 172.21.15.167:0/2507576217 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1b280cf770 0x7f1b2813ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:17.165 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:17.163+0000 7f1b2d7c1640 1 --2- 172.21.15.167:0/2507576217 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b280cf390 0x7f1b2813b660 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:17.165 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:17.163+0000 7f1b2d7c1640 1 --2- 172.21.15.167:0/2507576217 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1b28070650 0x7f1b2813e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:17.165 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:17.163+0000 7f1b2d7c1640 1 -- 172.21.15.167:0/2507576217 >> 172.21.15.167:0/2507576217 conn(0x7f1b280d1e40 msgr2=0x7f1b280b6d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:07:17.165 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:17.163+0000 7f1b2d7c1640 1 -- 172.21.15.167:0/2507576217 shutdown_connections 2024-05-11T08:07:17.165 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:17.163+0000 7f1b2d7c1640 1 -- 172.21.15.167:0/2507576217 wait complete. 2024-05-11T08:07:17.176 INFO:tasks.cephfs_test_runner:That the 'parent' 'layout' and 'symlink' xattrs on the head objects of files ... ok 2024-05-11T08:07:17.177 INFO:tasks.cephfs_test_runner: 2024-05-11T08:07:17.177 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-05-11T08:07:17.177 INFO:tasks.cephfs_test_runner:Ran 1 test in 46.714s 2024-05-11T08:07:17.177 INFO:tasks.cephfs_test_runner: 2024-05-11T08:07:17.177 INFO:tasks.cephfs_test_runner:OK 2024-05-11T08:07:17.177 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-05-11T08:07:17.186 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-05-11T08:07:17.247 INFO:tasks.kclient:Unmounting kernel clients... 2024-05-11T08:07:17.247 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-05-11T08:07:17.247 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T08:07:17.264 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056335,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20691 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-11T08:07:17.264 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-05-11T08:07:17.264 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T08:07:17.321 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056335,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20691 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-11T08:07:17.321 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-11T08:07:17.321 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T08:07:17.341 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056314,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497640k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1336 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248820k,nr_inodes=812205,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497640k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-11T08:07:17.341 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-05-11T08:07:17.341 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T08:07:17.401 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056314,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497640k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1336 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248820k,nr_inodes=812205,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497640k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-11T08:07:17.401 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T08:07:17.401 DEBUG:teuthology.orchestra.run.smithi167:> ip netns list 2024-05-11T08:07:17.417 INFO:teuthology.orchestra.run.smithi167.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-11T08:07:17.417 INFO:teuthology.orchestra.run.smithi167.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-11T08:07:17.417 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T08:07:17.417 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-11T08:07:17.484 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T08:07:17.485 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-11T08:07:17.552 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T08:07:17.553 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip link delete ceph-brx 2024-05-11T08:07:17.652 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T08:07:17.652 DEBUG:teuthology.orchestra.run.smithi167:> ip netns list 2024-05-11T08:07:17.668 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T08:07:17.668 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip link delete ceph-brx 2024-05-11T08:07:17.736 INFO:teuthology.orchestra.run.smithi167.stderr:Cannot find device "ceph-brx" 2024-05-11T08:07:17.737 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-11T08:07:17.738 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T08:07:17.738 DEBUG:teuthology.orchestra.run.smithi204:> ip netns list 2024-05-11T08:07:17.757 INFO:teuthology.orchestra.run.smithi204.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-05-11T08:07:17.757 INFO:teuthology.orchestra.run.smithi204.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-05-11T08:07:17.757 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T08:07:17.757 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-05-11T08:07:17.828 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T08:07:17.828 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-11T08:07:17.899 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T08:07:17.899 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip link delete ceph-brx 2024-05-11T08:07:17.998 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T08:07:17.999 DEBUG:teuthology.orchestra.run.smithi204:> ip netns list 2024-05-11T08:07:18.014 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T08:07:18.014 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip link delete ceph-brx 2024-05-11T08:07:18.083 INFO:teuthology.orchestra.run.smithi204.stderr:Cannot find device "ceph-brx" 2024-05-11T08:07:18.085 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-11T08:07:18.085 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-05-11T08:07:18.098 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-11T08:07:18.099 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-11T08:07:18.201 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.200+0000 7f4fafa09640 1 Processor -- start 2024-05-11T08:07:18.202 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.200+0000 7f4fafa09640 1 -- start start 2024-05-11T08:07:18.205 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.201+0000 7f4fafa09640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f4fa813eeb0 0x7f4fa806f480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:18.205 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.201+0000 7f4fafa09640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f4fa80b50a0 0x7f4fa8073cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:18.205 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.201+0000 7f4fafa09640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4fa815fb90 con 0x7f4fa813f290 2024-05-11T08:07:18.205 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.201+0000 7f4fafa09640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f4fa815d5b0 con 0x7f4fa80b50a0 2024-05-11T08:07:18.205 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.201+0000 7f4fafa09640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f4fa815fe70 con 0x7f4fa813eeb0 2024-05-11T08:07:18.205 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.201+0000 7f4fadfa6640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f4fa80b50a0 0x7f4fa8073cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:18.205 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.201+0000 7f4fadfa6640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f4fa80b50a0 0x7f4fa8073cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:56618/0 (socket says 172.21.15.167:56618) 2024-05-11T08:07:18.205 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.201+0000 7f4fadfa6640 1 -- 172.21.15.167:0/3466340381 learned_addr learned my addr 172.21.15.167:0/3466340381 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:07:18.205 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.202+0000 7f4fadfa6640 1 -- 172.21.15.167:0/3466340381 >> v2:172.21.15.204:3300/0 conn(0x7f4fa813eeb0 msgr2=0x7f4fa806f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:07:18.205 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.202+0000 7f4fad7a5640 1 --2- 172.21.15.167:0/3466340381 >> v2:172.21.15.204:3300/0 conn(0x7f4fa813eeb0 0x7f4fa806f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:18.205 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.202+0000 7f4fadfa6640 1 --2- 172.21.15.167:0/3466340381 >> v2:172.21.15.204:3300/0 conn(0x7f4fa813eeb0 0x7f4fa806f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:18.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.202+0000 7f4fadfa6640 1 -- 172.21.15.167:0/3466340381 >> v1:172.21.15.204:6789/0 conn(0x7f4fa813f290 legacy=0x7f4fa80735c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:07:18.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.202+0000 7f4fadfa6640 1 -- 172.21.15.167:0/3466340381 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fa8160150 con 0x7f4fa80b50a0 2024-05-11T08:07:18.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.202+0000 7f4fad7a5640 1 --2- 172.21.15.167:0/3466340381 >> v2:172.21.15.204:3300/0 conn(0x7f4fa813eeb0 0x7f4fa806f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T08:07:18.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.202+0000 7f4fadfa6640 1 --2- 172.21.15.167:0/3466340381 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4fa80b50a0 0x7f4fa8073cd0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f4fa80b6590 tx=0x7f4fa002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38f7cedac67c9a0a server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:18.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.202+0000 7f4f9ffff640 1 -- 172.21.15.167:0/3466340381 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fa0037070 con 0x7f4fa80b50a0 2024-05-11T08:07:18.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.202+0000 7f4f9ffff640 1 -- 172.21.15.167:0/3466340381 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4fa0032440 con 0x7f4fa80b50a0 2024-05-11T08:07:18.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.202+0000 7f4f9ffff640 1 -- 172.21.15.167:0/3466340381 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fa0032810 con 0x7f4fa80b50a0 2024-05-11T08:07:18.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.203+0000 7f4fafa09640 1 -- 172.21.15.167:0/3466340381 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4fa80b50a0 msgr2=0x7f4fa8073cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:18.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.203+0000 7f4fafa09640 1 --2- 172.21.15.167:0/3466340381 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4fa80b50a0 0x7f4fa8073cd0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f4fa80b6590 tx=0x7f4fa002a140 comp rx=0 tx=0).stop 2024-05-11T08:07:18.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.203+0000 7f4fafa09640 1 -- 172.21.15.167:0/3466340381 shutdown_connections 2024-05-11T08:07:18.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.203+0000 7f4fafa09640 1 --2- 172.21.15.167:0/3466340381 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4fa80b50a0 0x7f4fa8073cd0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:18.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.203+0000 7f4fafa09640 1 --2- 172.21.15.167:0/3466340381 >> v2:172.21.15.204:3300/0 conn(0x7f4fa813eeb0 0x7f4fa806f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:18.207 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.203+0000 7f4fafa09640 1 -- 172.21.15.167:0/3466340381 >> 172.21.15.167:0/3466340381 conn(0x7f4fa80cd8f0 msgr2=0x7f4fa80cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:07:18.207 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.203+0000 7f4fafa09640 1 -- 172.21.15.167:0/3466340381 shutdown_connections 2024-05-11T08:07:18.207 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.203+0000 7f4fafa09640 1 -- 172.21.15.167:0/3466340381 wait complete. 2024-05-11T08:07:18.210 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.204+0000 7f4fafa09640 1 Processor -- start 2024-05-11T08:07:18.210 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.204+0000 7f4fafa09640 1 -- start start 2024-05-11T08:07:18.210 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.205+0000 7f4fafa09640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4fa80b50a0 0x7f4fa8138520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:18.210 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.205+0000 7f4fafa09640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4fa813eeb0 0x7f4fa813a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:18.210 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.205+0000 7f4fafa09640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4fa813f290 0x7f4fa8154a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:18.210 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.205+0000 7f4fafa09640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4fa806f9c0 con 0x7f4fa813eeb0 2024-05-11T08:07:18.210 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.205+0000 7f4fafa09640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4fa8160710 con 0x7f4fa80b50a0 2024-05-11T08:07:18.210 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.205+0000 7f4fafa09640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f4fa806fb40 con 0x7f4fa813f290 2024-05-11T08:07:18.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.205+0000 7f4facfa4640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4fa813eeb0 0x7f4fa813a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:18.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.205+0000 7f4facfa4640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4fa813eeb0 0x7f4fa813a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:56626/0 (socket says 172.21.15.167:56626) 2024-05-11T08:07:18.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.205+0000 7f4facfa4640 1 -- 172.21.15.167:0/3761933439 learned_addr learned my addr 172.21.15.167:0/3761933439 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:07:18.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.205+0000 7f4fad7a5640 1 --2- 172.21.15.167:0/3761933439 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4fa80b50a0 0x7f4fa8138520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:18.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.205+0000 7f4fadfa6640 1 --2- 172.21.15.167:0/3761933439 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4fa813f290 0x7f4fa8154a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:18.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.205+0000 7f4facfa4640 1 -- 172.21.15.167:0/3761933439 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4fa813f290 msgr2=0x7f4fa8154a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:18.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.205+0000 7f4facfa4640 1 --2- 172.21.15.167:0/3761933439 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4fa813f290 0x7f4fa8154a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:18.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.205+0000 7f4facfa4640 1 -- 172.21.15.167:0/3761933439 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4fa80b50a0 msgr2=0x7f4fa8138520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:18.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.205+0000 7f4facfa4640 1 --2- 172.21.15.167:0/3761933439 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4fa80b50a0 0x7f4fa8138520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:18.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.205+0000 7f4facfa4640 1 -- 172.21.15.167:0/3761933439 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fa81525f0 con 0x7f4fa813eeb0 2024-05-11T08:07:18.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.205+0000 7f4fad7a5640 1 --2- 172.21.15.167:0/3761933439 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4fa80b50a0 0x7f4fa8138520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T08:07:18.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.206+0000 7f4facfa4640 1 --2- 172.21.15.167:0/3761933439 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4fa813eeb0 0x7f4fa813a440 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f4f9800d990 tx=0x7f4f9800de50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:18.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.206+0000 7f4f9dffb640 1 -- 172.21.15.167:0/3761933439 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f9801b070 con 0x7f4fa813eeb0 2024-05-11T08:07:18.212 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.206+0000 7f4f9dffb640 1 -- 172.21.15.167:0/3761933439 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f9800ed60 con 0x7f4fa813eeb0 2024-05-11T08:07:18.212 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.206+0000 7f4f9dffb640 1 -- 172.21.15.167:0/3761933439 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f98016680 con 0x7f4fa813eeb0 2024-05-11T08:07:18.212 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.206+0000 7f4fafa09640 1 -- 172.21.15.167:0/3761933439 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fa813ad90 con 0x7f4fa813eeb0 2024-05-11T08:07:18.212 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.206+0000 7f4fafa09640 1 -- 172.21.15.167:0/3761933439 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fa813b070 con 0x7f4fa813eeb0 2024-05-11T08:07:18.212 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.207+0000 7f4fafa09640 1 -- 172.21.15.167:0/3761933439 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4fa8069590 con 0x7f4fa813eeb0 2024-05-11T08:07:18.212 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.207+0000 7f4f9dffb640 1 -- 172.21.15.167:0/3761933439 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f4f9801e070 con 0x7f4fa813eeb0 2024-05-11T08:07:18.212 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.208+0000 7f4f9dffb640 1 --2- 172.21.15.167:0/3761933439 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f4f8c0762d0 0x7f4f8c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:18.212 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.208+0000 7f4f9dffb640 1 -- 172.21.15.167:0/3761933439 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5335+0+0 (secure 0 0 0) 0x7f4f98099180 con 0x7f4fa813eeb0 2024-05-11T08:07:18.212 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.208+0000 7f4fad7a5640 1 --2- 172.21.15.167:0/3761933439 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f4f8c0762d0 0x7f4f8c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:18.212 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.208+0000 7f4fad7a5640 1 --2- 172.21.15.167:0/3761933439 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f4f8c0762d0 0x7f4f8c078790 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f4fa813a230 tx=0x7f4fa4007550 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:18.213 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.212+0000 7f4f9dffb640 1 -- 172.21.15.167:0/3761933439 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f4f98062c50 con 0x7f4fa813eeb0 2024-05-11T08:07:18.389 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.388+0000 7f4fafa09640 1 -- 172.21.15.167:0/3761933439 --> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4fa8138b20 con 0x7f4f8c0762d0 2024-05-11T08:07:18.389 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.388+0000 7f4f9dffb640 1 -- 172.21.15.167:0/3761933439 <== mgr.4113 v2:172.21.15.167:6832/253569496 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13486 (secure 0 0 0) 0x7f4fa8138b20 con 0x7f4f8c0762d0 2024-05-11T08:07:18.390 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T08:07:18.390 INFO:teuthology.orchestra.run.smithi167.stderr:dumped all 2024-05-11T08:07:18.392 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.391+0000 7f4fafa09640 1 -- 172.21.15.167:0/3761933439 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f4f8c0762d0 msgr2=0x7f4f8c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:18.392 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.391+0000 7f4fafa09640 1 --2- 172.21.15.167:0/3761933439 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f4f8c0762d0 0x7f4f8c078790 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f4fa813a230 tx=0x7f4fa4007550 comp rx=0 tx=0).stop 2024-05-11T08:07:18.392 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.391+0000 7f4fafa09640 1 -- 172.21.15.167:0/3761933439 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4fa813eeb0 msgr2=0x7f4fa813a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:18.392 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.391+0000 7f4fafa09640 1 --2- 172.21.15.167:0/3761933439 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4fa813eeb0 0x7f4fa813a440 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f4f9800d990 tx=0x7f4f9800de50 comp rx=0 tx=0).stop 2024-05-11T08:07:18.393 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.391+0000 7f4fafa09640 1 -- 172.21.15.167:0/3761933439 shutdown_connections 2024-05-11T08:07:18.393 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.391+0000 7f4fafa09640 1 --2- 172.21.15.167:0/3761933439 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f4f8c0762d0 0x7f4f8c078790 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:18.393 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.391+0000 7f4fafa09640 1 --2- 172.21.15.167:0/3761933439 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4fa813f290 0x7f4fa8154a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:18.393 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.391+0000 7f4fafa09640 1 --2- 172.21.15.167:0/3761933439 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4fa813eeb0 0x7f4fa813a440 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:18.393 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.391+0000 7f4fafa09640 1 --2- 172.21.15.167:0/3761933439 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4fa80b50a0 0x7f4fa8138520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:18.393 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.391+0000 7f4fafa09640 1 -- 172.21.15.167:0/3761933439 >> 172.21.15.167:0/3761933439 conn(0x7f4fa80cd8f0 msgr2=0x7f4fa80728e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:07:18.393 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.391+0000 7f4fafa09640 1 -- 172.21.15.167:0/3761933439 shutdown_connections 2024-05-11T08:07:18.393 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.391+0000 7f4fafa09640 1 -- 172.21.15.167:0/3761933439 wait complete. 2024-05-11T08:07:18.403 INFO:teuthology.orchestra.run.smithi167.stdout:{"pg_ready":true,"pg_map":{"version":90,"stamp":"2024-05-11T08:07:17.932857+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":235,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":225708,"kb_used_data":5388,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754749012,"statfs":{"total":773094113280,"available":772862988288,"internally_reserved":0,"allocated":5517312,"data_stored":3138368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":225496625},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-53547,"num_objects":-33,"num_object_clones":0,"num_object_copies":-66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-33,"num_whiteouts":0,"num_read":-21,"num_read_kb":-10,"num_write":-194,"num_write_kb":-135,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-19,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000938"},"pg_stats":[{"pgid":"1.0","version":"18'72","reported_seq":154,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-11T08:07:11.645103+0000","last_change":"2024-05-11T08:05:45.772368+0000","last_active":"2024-05-11T08:07:11.645103+0000","last_peered":"2024-05-11T08:07:11.645103+0000","last_clean":"2024-05-11T08:07:11.645103+0000","last_became_active":"2024-05-11T08:05:45.771373+0000","last_became_peered":"2024-05-11T08:05:45.771373+0000","last_unstale":"2024-05-11T08:07:11.645103+0000","last_undegraded":"2024-05-11T08:07:11.645103+0000","last_fullsized":"2024-05-11T08:07:11.645103+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:44.616617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:44.616617+0000","last_clean_scrub_stamp":"2024-05-11T08:05:44.616617+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:02:08.078748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509462,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28172,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343668,"statfs":{"total":96636764160,"available":96607916032,"internally_reserved":0,"allocated":1179648,"data_stored":876570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509462,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27528,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344312,"statfs":{"total":96636764160,"available":96608575488,"internally_reserved":0,"allocated":520192,"data_stored":229032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509462,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27532,"kb_used_data":512,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344308,"statfs":{"total":96636764160,"available":96608571392,"internally_reserved":0,"allocated":524288,"data_stored":228562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509462,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27532,"kb_used_data":512,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344308,"statfs":{"total":96636764160,"available":96608571392,"internally_reserved":0,"allocated":524288,"data_stored":229054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509463,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27592,"kb_used_data":572,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344248,"statfs":{"total":96636764160,"available":96608509952,"internally_reserved":0,"allocated":585728,"data_stored":286166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509463,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27528,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344312,"statfs":{"total":96636764160,"available":96608575488,"internally_reserved":0,"allocated":520192,"data_stored":228540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509462,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28108,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343732,"statfs":{"total":96636764160,"available":96607981568,"internally_reserved":0,"allocated":1114112,"data_stored":818852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574871,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31716,"kb_used_data":536,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340124,"statfs":{"total":96636764160,"available":96604286976,"internally_reserved":0,"allocated":548864,"data_stored":241592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-11T08:07:18.403 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-11T08:07:18.502 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.501+0000 7fbcc26c4640 1 Processor -- start 2024-05-11T08:07:18.503 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.501+0000 7fbcc26c4640 1 -- start start 2024-05-11T08:07:18.504 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.502+0000 7fbcc26c4640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbcbc157750 0x7fbcbc15d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:18.504 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.502+0000 7fbcc26c4640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fbcbc06d460 0x7fbcbc06d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:18.504 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.502+0000 7fbcc26c4640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbcbc15dc80 con 0x7fbcbc0b3ea0 2024-05-11T08:07:18.504 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.502+0000 7fbcc26c4640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fbcbc161000 con 0x7fbcbc06d460 2024-05-11T08:07:18.504 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.502+0000 7fbcc26c4640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fbcbc15df30 con 0x7fbcbc157750 2024-05-11T08:07:18.504 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.502+0000 7fbcc0c61640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fbcbc06d460 0x7fbcbc06d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:18.504 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.502+0000 7fbcc0c61640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fbcbc06d460 0x7fbcbc06d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.167:57418/0 (socket says 172.21.15.167:57418) 2024-05-11T08:07:18.504 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.502+0000 7fbcb3fff640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7fbcbc0b3ea0 0x7fbcbc161490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.167:58140/0 (socket says 172.21.15.167:58140) 2024-05-11T08:07:18.505 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.502+0000 7fbcc0c61640 1 -- 172.21.15.167:0/2455084067 learned_addr learned my addr 172.21.15.167:0/2455084067 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:07:18.505 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.502+0000 7fbcbbfff640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbcbc157750 0x7fbcbc15d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:18.505 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.503+0000 7fbcc0c61640 1 -- 172.21.15.167:0/2455084067 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbcbc157750 msgr2=0x7fbcbc15d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:07:18.505 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.503+0000 7fbcc0c61640 1 --2- 172.21.15.167:0/2455084067 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbcbc157750 0x7fbcbc15d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:18.505 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.503+0000 7fbcc0c61640 1 -- 172.21.15.167:0/2455084067 >> v1:172.21.15.204:6789/0 conn(0x7fbcbc0b3ea0 legacy=0x7fbcbc161490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:18.505 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.503+0000 7fbcc0c61640 1 -- 172.21.15.167:0/2455084067 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcbc15e210 con 0x7fbcbc06d460 2024-05-11T08:07:18.507 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.504+0000 7fbcc0c61640 1 --2- 172.21.15.167:0/2455084067 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbcbc06d460 0x7fbcbc06d8f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fbcb4009e50 tx=0x7fbcb402a710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=639598a5d65c7ebb server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:18.507 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.504+0000 7fbcbb7fe640 1 -- 172.21.15.167:0/2455084067 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcb4036070 con 0x7fbcbc06d460 2024-05-11T08:07:18.507 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.504+0000 7fbcbb7fe640 1 -- 172.21.15.167:0/2455084067 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbcb4031600 con 0x7fbcbc06d460 2024-05-11T08:07:18.507 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.504+0000 7fbcbb7fe640 1 -- 172.21.15.167:0/2455084067 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcb40319d0 con 0x7fbcbc06d460 2024-05-11T08:07:18.507 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.504+0000 7fbcc26c4640 1 -- 172.21.15.167:0/2455084067 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbcbc06d460 msgr2=0x7fbcbc06d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:18.507 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.504+0000 7fbcc26c4640 1 --2- 172.21.15.167:0/2455084067 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbcbc06d460 0x7fbcbc06d8f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fbcb4009e50 tx=0x7fbcb402a710 comp rx=0 tx=0).stop 2024-05-11T08:07:18.507 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.504+0000 7fbcc26c4640 1 -- 172.21.15.167:0/2455084067 shutdown_connections 2024-05-11T08:07:18.507 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.504+0000 7fbcc26c4640 1 --2- 172.21.15.167:0/2455084067 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbcbc06d460 0x7fbcbc06d8f0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:18.507 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.504+0000 7fbcc26c4640 1 --2- 172.21.15.167:0/2455084067 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbcbc157750 0x7fbcbc15d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:18.507 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.504+0000 7fbcc26c4640 1 -- 172.21.15.167:0/2455084067 >> 172.21.15.167:0/2455084067 conn(0x7fbcbc0cd8f0 msgr2=0x7fbcbc0cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:07:18.507 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.504+0000 7fbcc26c4640 1 -- 172.21.15.167:0/2455084067 shutdown_connections 2024-05-11T08:07:18.508 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.504+0000 7fbcc26c4640 1 -- 172.21.15.167:0/2455084067 wait complete. 2024-05-11T08:07:18.508 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.505+0000 7fbcc26c4640 1 Processor -- start 2024-05-11T08:07:18.508 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.506+0000 7fbcc26c4640 1 -- start start 2024-05-11T08:07:18.511 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.506+0000 7fbcc26c4640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbcbc06d460 0x7fbcbc158380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:18.511 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.506+0000 7fbcc26c4640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbcbc0b3ea0 0x7fbcbc134370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:18.511 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.506+0000 7fbcc26c4640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbcbc157750 0x7fbcbc15a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:18.511 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.506+0000 7fbcc26c4640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fbcbc161ba0 con 0x7fbcbc157750 2024-05-11T08:07:18.511 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.506+0000 7fbcc26c4640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fbcbc157b30 con 0x7fbcbc0b3ea0 2024-05-11T08:07:18.511 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.506+0000 7fbcc26c4640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fbcbc0cf1c0 con 0x7fbcbc06d460 2024-05-11T08:07:18.511 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.506+0000 7fbcc0c61640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbcbc157750 0x7fbcbc15a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:18.511 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.506+0000 7fbcc0c61640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbcbc157750 0x7fbcbc15a7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:56634/0 (socket says 172.21.15.167:56634) 2024-05-11T08:07:18.511 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.506+0000 7fbcc0c61640 1 -- 172.21.15.167:0/1992425914 learned_addr learned my addr 172.21.15.167:0/1992425914 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:07:18.511 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.506+0000 7fbcb3fff640 1 --2- 172.21.15.167:0/1992425914 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbcbc0b3ea0 0x7fbcbc134370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:18.511 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.506+0000 7fbcbbfff640 1 --2- 172.21.15.167:0/1992425914 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbcbc06d460 0x7fbcbc158380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:18.512 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.507+0000 7fbcc0c61640 1 -- 172.21.15.167:0/1992425914 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbcbc06d460 msgr2=0x7fbcbc158380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:18.512 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.507+0000 7fbcc0c61640 1 --2- 172.21.15.167:0/1992425914 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbcbc06d460 0x7fbcbc158380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:18.512 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.507+0000 7fbcc0c61640 1 -- 172.21.15.167:0/1992425914 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbcbc0b3ea0 msgr2=0x7fbcbc134370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:18.512 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.507+0000 7fbcc0c61640 1 --2- 172.21.15.167:0/1992425914 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbcbc0b3ea0 0x7fbcbc134370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:18.512 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.507+0000 7fbcc0c61640 1 -- 172.21.15.167:0/1992425914 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcbc135280 con 0x7fbcbc157750 2024-05-11T08:07:18.512 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.507+0000 7fbcb3fff640 1 --2- 172.21.15.167:0/1992425914 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbcbc0b3ea0 0x7fbcbc134370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T08:07:18.512 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.507+0000 7fbcc0c61640 1 --2- 172.21.15.167:0/1992425914 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbcbc157750 0x7fbcbc15a7d0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fbcb4009f70 tx=0x7fbcb4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:18.512 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.507+0000 7fbcb97fa640 1 -- 172.21.15.167:0/1992425914 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcb403f020 con 0x7fbcbc157750 2024-05-11T08:07:18.512 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.507+0000 7fbcb97fa640 1 -- 172.21.15.167:0/1992425914 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbcb4037040 con 0x7fbcbc157750 2024-05-11T08:07:18.512 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.507+0000 7fbcc26c4640 1 -- 172.21.15.167:0/1992425914 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcbc134cc0 con 0x7fbcbc157750 2024-05-11T08:07:18.512 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.507+0000 7fbcb97fa640 1 -- 172.21.15.167:0/1992425914 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcb4036070 con 0x7fbcbc157750 2024-05-11T08:07:18.512 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.507+0000 7fbcc26c4640 1 -- 172.21.15.167:0/1992425914 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcbc159fd0 con 0x7fbcbc157750 2024-05-11T08:07:18.512 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.508+0000 7fbcc26c4640 1 -- 172.21.15.167:0/1992425914 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbcbc069590 con 0x7fbcbc157750 2024-05-11T08:07:18.513 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.509+0000 7fbcb97fa640 1 -- 172.21.15.167:0/1992425914 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fbcb4042020 con 0x7fbcbc157750 2024-05-11T08:07:18.513 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.509+0000 7fbcb97fa640 1 --2- 172.21.15.167:0/1992425914 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fbc9c0762d0 0x7fbc9c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:18.513 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.510+0000 7fbcbbfff640 1 --2- 172.21.15.167:0/1992425914 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fbc9c0762d0 0x7fbc9c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:18.513 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.510+0000 7fbcb97fa640 1 -- 172.21.15.167:0/1992425914 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5335+0+0 (secure 0 0 0) 0x7fbcb40b7700 con 0x7fbcbc157750 2024-05-11T08:07:18.513 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.510+0000 7fbcbbfff640 1 --2- 172.21.15.167:0/1992425914 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fbc9c0762d0 0x7fbc9c078790 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fbcac009aa0 tx=0x7fbcac009190 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:18.515 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.515+0000 7fbcb97fa640 1 -- 172.21.15.167:0/1992425914 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fbcb4049030 con 0x7fbcbc157750 2024-05-11T08:07:18.690 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.689+0000 7fbcc26c4640 1 -- 172.21.15.167:0/1992425914 --> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbcbc075b40 con 0x7fbc9c0762d0 2024-05-11T08:07:18.691 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.690+0000 7fbcb97fa640 1 -- 172.21.15.167:0/1992425914 <== mgr.4113 v2:172.21.15.167:6832/253569496 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13486 (secure 0 0 0) 0x7fbcbc075b40 con 0x7fbc9c0762d0 2024-05-11T08:07:18.692 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T08:07:18.692 INFO:teuthology.orchestra.run.smithi167.stderr:dumped all 2024-05-11T08:07:18.694 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.692+0000 7fbcc26c4640 1 -- 172.21.15.167:0/1992425914 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fbc9c0762d0 msgr2=0x7fbc9c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:18.694 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.692+0000 7fbcc26c4640 1 --2- 172.21.15.167:0/1992425914 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fbc9c0762d0 0x7fbc9c078790 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fbcac009aa0 tx=0x7fbcac009190 comp rx=0 tx=0).stop 2024-05-11T08:07:18.694 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.692+0000 7fbcc26c4640 1 -- 172.21.15.167:0/1992425914 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbcbc157750 msgr2=0x7fbcbc15a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:18.694 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.692+0000 7fbcc26c4640 1 --2- 172.21.15.167:0/1992425914 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbcbc157750 0x7fbcbc15a7d0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fbcb4009f70 tx=0x7fbcb4004a40 comp rx=0 tx=0).stop 2024-05-11T08:07:18.694 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.692+0000 7fbcc26c4640 1 -- 172.21.15.167:0/1992425914 shutdown_connections 2024-05-11T08:07:18.694 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.692+0000 7fbcc26c4640 1 --2- 172.21.15.167:0/1992425914 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fbc9c0762d0 0x7fbc9c078790 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:18.694 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.692+0000 7fbcc26c4640 1 --2- 172.21.15.167:0/1992425914 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbcbc157750 0x7fbcbc15a7d0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:18.694 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.692+0000 7fbcc26c4640 1 --2- 172.21.15.167:0/1992425914 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbcbc0b3ea0 0x7fbcbc134370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:18.695 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.692+0000 7fbcc26c4640 1 --2- 172.21.15.167:0/1992425914 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbcbc06d460 0x7fbcbc158380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:18.695 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.692+0000 7fbcc26c4640 1 -- 172.21.15.167:0/1992425914 >> 172.21.15.167:0/1992425914 conn(0x7fbcbc0cd8f0 msgr2=0x7fbcbc161490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:07:18.695 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.692+0000 7fbcc26c4640 1 -- 172.21.15.167:0/1992425914 shutdown_connections 2024-05-11T08:07:18.695 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.692+0000 7fbcc26c4640 1 -- 172.21.15.167:0/1992425914 wait complete. 2024-05-11T08:07:18.704 INFO:teuthology.orchestra.run.smithi167.stdout:{"pg_ready":true,"pg_map":{"version":90,"stamp":"2024-05-11T08:07:17.932857+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":235,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":225708,"kb_used_data":5388,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754749012,"statfs":{"total":773094113280,"available":772862988288,"internally_reserved":0,"allocated":5517312,"data_stored":3138368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":225496625},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-53547,"num_objects":-33,"num_object_clones":0,"num_object_copies":-66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-33,"num_whiteouts":0,"num_read":-21,"num_read_kb":-10,"num_write":-194,"num_write_kb":-135,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-19,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000938"},"pg_stats":[{"pgid":"1.0","version":"18'72","reported_seq":154,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-11T08:07:11.645103+0000","last_change":"2024-05-11T08:05:45.772368+0000","last_active":"2024-05-11T08:07:11.645103+0000","last_peered":"2024-05-11T08:07:11.645103+0000","last_clean":"2024-05-11T08:07:11.645103+0000","last_became_active":"2024-05-11T08:05:45.771373+0000","last_became_peered":"2024-05-11T08:05:45.771373+0000","last_unstale":"2024-05-11T08:07:11.645103+0000","last_undegraded":"2024-05-11T08:07:11.645103+0000","last_fullsized":"2024-05-11T08:07:11.645103+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:44.616617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:44.616617+0000","last_clean_scrub_stamp":"2024-05-11T08:05:44.616617+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:02:08.078748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509462,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28172,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343668,"statfs":{"total":96636764160,"available":96607916032,"internally_reserved":0,"allocated":1179648,"data_stored":876570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509462,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27528,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344312,"statfs":{"total":96636764160,"available":96608575488,"internally_reserved":0,"allocated":520192,"data_stored":229032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509462,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27532,"kb_used_data":512,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344308,"statfs":{"total":96636764160,"available":96608571392,"internally_reserved":0,"allocated":524288,"data_stored":228562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509462,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27532,"kb_used_data":512,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344308,"statfs":{"total":96636764160,"available":96608571392,"internally_reserved":0,"allocated":524288,"data_stored":229054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509463,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27592,"kb_used_data":572,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344248,"statfs":{"total":96636764160,"available":96608509952,"internally_reserved":0,"allocated":585728,"data_stored":286166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509463,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27528,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344312,"statfs":{"total":96636764160,"available":96608575488,"internally_reserved":0,"allocated":520192,"data_stored":228540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509462,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28108,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343732,"statfs":{"total":96636764160,"available":96607981568,"internally_reserved":0,"allocated":1114112,"data_stored":818852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574871,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31716,"kb_used_data":536,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340124,"statfs":{"total":96636764160,"available":96604286976,"internally_reserved":0,"allocated":548864,"data_stored":241592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-11T08:07:18.705 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-11T08:07:18.705 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-11T08:07:18.806 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.805+0000 7f84e2acf640 1 Processor -- start 2024-05-11T08:07:18.806 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.805+0000 7f84e2acf640 1 -- start start 2024-05-11T08:07:18.808 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.806+0000 7f84e2acf640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f84dc0736a0 0x7f84dc072b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:18.808 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.806+0000 7f84e2acf640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f84dc072640 0x7f84dc073040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:18.808 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.806+0000 7f84e2acf640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84dc15bc40 con 0x7f84dc15b860 2024-05-11T08:07:18.808 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.806+0000 7f84e2acf640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f84dc06e410 con 0x7f84dc072640 2024-05-11T08:07:18.808 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.806+0000 7f84e2acf640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f84dc06e6f0 con 0x7f84dc0736a0 2024-05-11T08:07:18.808 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.806+0000 7f84e106c640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f84dc072640 0x7f84dc073040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:18.808 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.806+0000 7f84e086b640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f84dc15b860 0x7f84dc06f4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:59728/0 (socket says 172.21.15.167:59728) 2024-05-11T08:07:18.808 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.806+0000 7f84e086b640 1 -- 172.21.15.167:0/1389397579 learned_addr learned my addr 172.21.15.167:0/1389397579 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T08:07:18.808 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.806+0000 7f84d3fff640 1 --2- 172.21.15.167:0/1389397579 >> v2:172.21.15.204:3300/0 conn(0x7f84dc0736a0 0x7f84dc072b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:18.808 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.806+0000 7f84e106c640 1 -- 172.21.15.167:0/1389397579 >> v2:172.21.15.204:3300/0 conn(0x7f84dc0736a0 msgr2=0x7f84dc072b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:07:18.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.806+0000 7f84e106c640 1 --2- 172.21.15.167:0/1389397579 >> v2:172.21.15.204:3300/0 conn(0x7f84dc0736a0 0x7f84dc072b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:18.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.806+0000 7f84e106c640 1 -- 172.21.15.167:0/1389397579 >> v1:172.21.15.167:6789/0 conn(0x7f84dc15b860 legacy=0x7f84dc06f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:18.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.806+0000 7f84e106c640 1 -- 172.21.15.167:0/1389397579 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84dc06e9d0 con 0x7f84dc072640 2024-05-11T08:07:18.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.806+0000 7f84d37fe640 1 -- 172.21.15.167:0/1389397579 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 414643131 0 0) 0x7f84d8001110 con 0x7f84dc15b860 2024-05-11T08:07:18.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.806+0000 7f84d3fff640 1 --2- 172.21.15.167:0/1389397579 >> v2:172.21.15.204:3300/0 conn(0x7f84dc0736a0 0x7f84dc072b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T08:07:18.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.807+0000 7f84e106c640 1 --2- 172.21.15.167:0/1389397579 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f84dc072640 0x7f84dc073040 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f84dc0700b0 tx=0x7f84d402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28856598f98d5617 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:18.812 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.807+0000 7f84d37fe640 1 -- 172.21.15.167:0/1389397579 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84d4007740 con 0x7f84dc072640 2024-05-11T08:07:18.812 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.807+0000 7f84d37fe640 1 -- 172.21.15.167:0/1389397579 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84d40078e0 con 0x7f84dc072640 2024-05-11T08:07:18.812 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.807+0000 7f84d37fe640 1 -- 172.21.15.167:0/1389397579 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84d4007cb0 con 0x7f84dc072640 2024-05-11T08:07:18.812 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.808+0000 7f84e2acf640 1 -- 172.21.15.167:0/1389397579 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f84dc072640 msgr2=0x7f84dc073040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:18.812 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.808+0000 7f84e2acf640 1 --2- 172.21.15.167:0/1389397579 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f84dc072640 0x7f84dc073040 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f84dc0700b0 tx=0x7f84d402a680 comp rx=0 tx=0).stop 2024-05-11T08:07:18.813 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.808+0000 7f84e2acf640 1 -- 172.21.15.167:0/1389397579 shutdown_connections 2024-05-11T08:07:18.813 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.808+0000 7f84e2acf640 1 --2- 172.21.15.167:0/1389397579 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f84dc072640 0x7f84dc073040 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:18.813 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.808+0000 7f84e2acf640 1 --2- 172.21.15.167:0/1389397579 >> v2:172.21.15.204:3300/0 conn(0x7f84dc0736a0 0x7f84dc072b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:18.813 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.808+0000 7f84e2acf640 1 -- 172.21.15.167:0/1389397579 >> 172.21.15.167:0/1389397579 conn(0x7f84dc0cd8f0 msgr2=0x7f84dc0cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:07:18.813 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.808+0000 7f84e2acf640 1 -- 172.21.15.167:0/1389397579 shutdown_connections 2024-05-11T08:07:18.813 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.808+0000 7f84e2acf640 1 -- 172.21.15.167:0/1389397579 wait complete. 2024-05-11T08:07:18.813 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.809+0000 7f84e2acf640 1 Processor -- start 2024-05-11T08:07:18.813 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.809+0000 7f84e2acf640 1 -- start start 2024-05-11T08:07:18.813 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.810+0000 7f84e2acf640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f84dc072640 0x7f84dc14c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:18.813 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.810+0000 7f84e2acf640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f84dc0736a0 0x7f84dc15fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:18.813 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.810+0000 7f84e2acf640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f84dc15b860 0x7f84dc156b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:18.813 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.810+0000 7f84e2acf640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f84dc06fbc0 con 0x7f84dc072640 2024-05-11T08:07:18.813 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.810+0000 7f84e2acf640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f84dc06ef90 con 0x7f84dc15b860 2024-05-11T08:07:18.814 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.810+0000 7f84e2acf640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f84dc0cf620 con 0x7f84dc0736a0 2024-05-11T08:07:18.814 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.810+0000 7f84e086b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f84dc072640 0x7f84dc14c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:18.814 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.810+0000 7f84e086b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f84dc072640 0x7f84dc14c1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:56646/0 (socket says 172.21.15.167:56646) 2024-05-11T08:07:18.814 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.810+0000 7f84e086b640 1 -- 172.21.15.167:0/4231716461 learned_addr learned my addr 172.21.15.167:0/4231716461 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:07:18.814 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.810+0000 7f84e106c640 1 --2- 172.21.15.167:0/4231716461 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f84dc15b860 0x7f84dc156b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:18.814 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.810+0000 7f84d3fff640 1 --2- 172.21.15.167:0/4231716461 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f84dc0736a0 0x7f84dc15fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:18.814 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.810+0000 7f84e086b640 1 -- 172.21.15.167:0/4231716461 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f84dc0736a0 msgr2=0x7f84dc15fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:18.814 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.810+0000 7f84e086b640 1 --2- 172.21.15.167:0/4231716461 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f84dc0736a0 0x7f84dc15fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:18.814 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.810+0000 7f84e086b640 1 -- 172.21.15.167:0/4231716461 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f84dc15b860 msgr2=0x7f84dc156b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:18.814 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.810+0000 7f84e086b640 1 --2- 172.21.15.167:0/4231716461 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f84dc15b860 0x7f84dc156b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:18.814 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.810+0000 7f84e086b640 1 -- 172.21.15.167:0/4231716461 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84dc160170 con 0x7f84dc072640 2024-05-11T08:07:18.814 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.810+0000 7f84e086b640 1 --2- 172.21.15.167:0/4231716461 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f84dc072640 0x7f84dc14c1d0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f84d800c400 tx=0x7f84d800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:18.814 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.810+0000 7f84d17fa640 1 -- 172.21.15.167:0/4231716461 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84d8017070 con 0x7f84dc072640 2024-05-11T08:07:18.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.810+0000 7f84d17fa640 1 -- 172.21.15.167:0/4231716461 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84d8005ce0 con 0x7f84dc072640 2024-05-11T08:07:18.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.810+0000 7f84e2acf640 1 -- 172.21.15.167:0/4231716461 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84dc152460 con 0x7f84dc072640 2024-05-11T08:07:18.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.810+0000 7f84d17fa640 1 -- 172.21.15.167:0/4231716461 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84d8007310 con 0x7f84dc072640 2024-05-11T08:07:18.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.811+0000 7f84e2acf640 1 -- 172.21.15.167:0/4231716461 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84dc152a20 con 0x7f84dc072640 2024-05-11T08:07:18.816 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.812+0000 7f84e2acf640 1 -- 172.21.15.167:0/4231716461 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84dc069590 con 0x7f84dc072640 2024-05-11T08:07:18.816 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.812+0000 7f84d17fa640 1 -- 172.21.15.167:0/4231716461 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f84d801a070 con 0x7f84dc072640 2024-05-11T08:07:18.816 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.813+0000 7f84d17fa640 1 --2- 172.21.15.167:0/4231716461 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f84c80762d0 0x7f84c8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:18.816 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.813+0000 7f84d3fff640 1 --2- 172.21.15.167:0/4231716461 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f84c80762d0 0x7f84c8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:18.816 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.813+0000 7f84d17fa640 1 -- 172.21.15.167:0/4231716461 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5335+0+0 (secure 0 0 0) 0x7f84d8096840 con 0x7f84dc072640 2024-05-11T08:07:18.816 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.813+0000 7f84d3fff640 1 --2- 172.21.15.167:0/4231716461 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f84c80762d0 0x7f84c8078790 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f84dc06da90 tx=0x7f84cc008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:18.818 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.818+0000 7f84d17fa640 1 -- 172.21.15.167:0/4231716461 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f84dc15bc40 con 0x7f84dc072640 2024-05-11T08:07:18.994 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.993+0000 7f84e2acf640 1 -- 172.21.15.167:0/4231716461 --> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f84dc151500 con 0x7f84c80762d0 2024-05-11T08:07:18.994 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.993+0000 7f84d17fa640 1 -- 172.21.15.167:0/4231716461 <== mgr.4113 v2:172.21.15.167:6832/253569496 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13486 (secure 0 0 0) 0x7f84dc151500 con 0x7f84c80762d0 2024-05-11T08:07:18.995 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T08:07:18.995 INFO:teuthology.orchestra.run.smithi167.stderr:dumped all 2024-05-11T08:07:18.997 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.995+0000 7f84e2acf640 1 -- 172.21.15.167:0/4231716461 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f84c80762d0 msgr2=0x7f84c8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:18.997 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.995+0000 7f84e2acf640 1 --2- 172.21.15.167:0/4231716461 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f84c80762d0 0x7f84c8078790 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f84dc06da90 tx=0x7f84cc008040 comp rx=0 tx=0).stop 2024-05-11T08:07:18.997 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.995+0000 7f84e2acf640 1 -- 172.21.15.167:0/4231716461 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f84dc072640 msgr2=0x7f84dc14c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:18.997 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.995+0000 7f84e2acf640 1 --2- 172.21.15.167:0/4231716461 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f84dc072640 0x7f84dc14c1d0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f84d800c400 tx=0x7f84d800c8c0 comp rx=0 tx=0).stop 2024-05-11T08:07:18.997 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.995+0000 7f84e2acf640 1 -- 172.21.15.167:0/4231716461 shutdown_connections 2024-05-11T08:07:18.997 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.995+0000 7f84e2acf640 1 --2- 172.21.15.167:0/4231716461 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f84c80762d0 0x7f84c8078790 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:18.997 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.995+0000 7f84e2acf640 1 --2- 172.21.15.167:0/4231716461 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f84dc15b860 0x7f84dc156b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:18.997 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.995+0000 7f84e2acf640 1 --2- 172.21.15.167:0/4231716461 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f84dc0736a0 0x7f84dc15fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:18.997 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.995+0000 7f84e2acf640 1 --2- 172.21.15.167:0/4231716461 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f84dc072640 0x7f84dc14c1d0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:18.998 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.996+0000 7f84e2acf640 1 -- 172.21.15.167:0/4231716461 >> 172.21.15.167:0/4231716461 conn(0x7f84dc0cd8f0 msgr2=0x7f84dc151820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:07:18.998 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.996+0000 7f84e2acf640 1 -- 172.21.15.167:0/4231716461 shutdown_connections 2024-05-11T08:07:18.998 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:18.996+0000 7f84e2acf640 1 -- 172.21.15.167:0/4231716461 wait complete. 2024-05-11T08:07:19.007 INFO:teuthology.orchestra.run.smithi167.stdout:{"pg_ready":true,"pg_map":{"version":90,"stamp":"2024-05-11T08:07:17.932857+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":235,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":225708,"kb_used_data":5388,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754749012,"statfs":{"total":773094113280,"available":772862988288,"internally_reserved":0,"allocated":5517312,"data_stored":3138368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":225496625},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-53547,"num_objects":-33,"num_object_clones":0,"num_object_copies":-66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-33,"num_whiteouts":0,"num_read":-21,"num_read_kb":-10,"num_write":-194,"num_write_kb":-135,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-19,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000938"},"pg_stats":[{"pgid":"1.0","version":"18'72","reported_seq":154,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-11T08:07:11.645103+0000","last_change":"2024-05-11T08:05:45.772368+0000","last_active":"2024-05-11T08:07:11.645103+0000","last_peered":"2024-05-11T08:07:11.645103+0000","last_clean":"2024-05-11T08:07:11.645103+0000","last_became_active":"2024-05-11T08:05:45.771373+0000","last_became_peered":"2024-05-11T08:05:45.771373+0000","last_unstale":"2024-05-11T08:07:11.645103+0000","last_undegraded":"2024-05-11T08:07:11.645103+0000","last_fullsized":"2024-05-11T08:07:11.645103+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:44.616617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:44.616617+0000","last_clean_scrub_stamp":"2024-05-11T08:05:44.616617+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:02:08.078748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509462,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28172,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343668,"statfs":{"total":96636764160,"available":96607916032,"internally_reserved":0,"allocated":1179648,"data_stored":876570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509462,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27528,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344312,"statfs":{"total":96636764160,"available":96608575488,"internally_reserved":0,"allocated":520192,"data_stored":229032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509462,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27532,"kb_used_data":512,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344308,"statfs":{"total":96636764160,"available":96608571392,"internally_reserved":0,"allocated":524288,"data_stored":228562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509462,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27532,"kb_used_data":512,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344308,"statfs":{"total":96636764160,"available":96608571392,"internally_reserved":0,"allocated":524288,"data_stored":229054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509463,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27592,"kb_used_data":572,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344248,"statfs":{"total":96636764160,"available":96608509952,"internally_reserved":0,"allocated":585728,"data_stored":286166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509463,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27528,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344312,"statfs":{"total":96636764160,"available":96608575488,"internally_reserved":0,"allocated":520192,"data_stored":228540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509462,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28108,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343732,"statfs":{"total":96636764160,"available":96607981568,"internally_reserved":0,"allocated":1114112,"data_stored":818852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574871,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31716,"kb_used_data":536,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340124,"statfs":{"total":96636764160,"available":96604286976,"internally_reserved":0,"allocated":548864,"data_stored":241592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-11T08:07:19.008 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-11T08:07:19.108 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:19.107+0000 7f1c9e015640 1 Processor -- start 2024-05-11T08:07:19.108 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:19.107+0000 7f1c9e015640 1 -- start start 2024-05-11T08:07:19.109 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:19.108+0000 7f1c9e015640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f1c9815b860 0x7f1c9806f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:19.109 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:19.108+0000 7f1c9e015640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1c980736a0 0x7f1c98072b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:19.109 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:19.108+0000 7f1c9e015640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c9815bc40 con 0x7f1c98072640 2024-05-11T08:07:19.109 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:19.108+0000 7f1c9e015640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f1c9806e410 con 0x7f1c9815b860 2024-05-11T08:07:19.109 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:19.108+0000 7f1c9e015640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f1c9806e6f0 con 0x7f1c980736a0 2024-05-11T08:07:19.110 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:19.108+0000 7f1c977fe640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f1c9815b860 0x7f1c9806f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:19.110 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:19.108+0000 7f1c977fe640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f1c9815b860 0x7f1c9806f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:56648/0 (socket says 172.21.15.167:56648) 2024-05-11T08:07:19.110 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:19.108+0000 7f1c977fe640 1 -- 172.21.15.167:0/721948751 learned_addr learned my addr 172.21.15.167:0/721948751 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:07:19.110 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:19.108+0000 7f1c96ffd640 1 --2- 172.21.15.167:0/721948751 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1c980736a0 0x7f1c98072b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:19.110 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:19.108+0000 7f1c977fe640 1 -- 172.21.15.167:0/721948751 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1c980736a0 msgr2=0x7f1c98072b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:07:19.110 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:19.108+0000 7f1c977fe640 1 --2- 172.21.15.167:0/721948751 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1c980736a0 0x7f1c98072b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:19.110 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:19.108+0000 7f1c977fe640 1 -- 172.21.15.167:0/721948751 >> v1:172.21.15.204:6789/0 conn(0x7f1c98072640 legacy=0x7f1c980b47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:19.110 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:19.108+0000 7f1c977fe640 1 -- 172.21.15.167:0/721948751 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c9806e9d0 con 0x7f1c9815b860 2024-05-11T08:07:19.112 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:19.109+0000 7f1c977fe640 1 --2- 172.21.15.167:0/721948751 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1c9815b860 0x7f1c9806f4b0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f1c88009e50 tx=0x7f1c8802a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1605659720c3a0a server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:19.112 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:19.109+0000 7f1c967fc640 1 -- 172.21.15.167:0/721948751 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c88036070 con 0x7f1c9815b860 2024-05-11T08:07:19.112 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:19.109+0000 7f1c967fc640 1 -- 172.21.15.167:0/721948751 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c88031600 con 0x7f1c9815b860 2024-05-11T08:07:19.112 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:19.109+0000 7f1c967fc640 1 -- 172.21.15.167:0/721948751 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c880319d0 con 0x7f1c9815b860 2024-05-11T08:07:19.112 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:19.109+0000 7f1c9e015640 1 -- 172.21.15.167:0/721948751 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1c9815b860 msgr2=0x7f1c9806f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:19.112 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:19.109+0000 7f1c9e015640 1 --2- 172.21.15.167:0/721948751 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1c9815b860 0x7f1c9806f4b0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f1c88009e50 tx=0x7f1c8802a710 comp rx=0 tx=0).stop 2024-05-11T08:07:19.112 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:19.109+0000 7f1c9e015640 1 -- 172.21.15.167:0/721948751 shutdown_connections 2024-05-11T08:07:19.112 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:19.109+0000 7f1c9e015640 1 --2- 172.21.15.167:0/721948751 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1c980736a0 0x7f1c98072b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:19.112 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:19.109+0000 7f1c9e015640 1 --2- 172.21.15.167:0/721948751 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1c9815b860 0x7f1c9806f4b0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:19.112 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:19.109+0000 7f1c9e015640 1 -- 172.21.15.167:0/721948751 >> 172.21.15.167:0/721948751 conn(0x7f1c980cd8f0 msgr2=0x7f1c980cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:07:19.112 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:19.109+0000 7f1c9e015640 1 -- 172.21.15.167:0/721948751 shutdown_connections 2024-05-11T08:07:19.112 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:19.109+0000 7f1c9e015640 1 -- 172.21.15.167:0/721948751 wait complete. 2024-05-11T08:07:19.112 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:19.111+0000 7f1c9e015640 1 Processor -- start 2024-05-11T08:07:19.112 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:19.111+0000 7f1c9e015640 1 -- start start 2024-05-11T08:07:19.114 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:19.111+0000 7f1c9e015640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1c98072640 0x7f1c9814ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:19.115 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:19.111+0000 7f1c9e015640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1c980736a0 0x7f1c9815c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:19.115 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:19.111+0000 7f1c9e015640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1c9815b860 0x7f1c981f6a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:19.115 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:19.111+0000 7f1c9e015640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1c980b4ee0 con 0x7f1c980736a0 2024-05-11T08:07:19.115 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:19.111+0000 7f1c9e015640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1c9806ef90 con 0x7f1c98072640 2024-05-11T08:07:19.115 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:19.111+0000 7f1c9e015640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f1c980b5060 con 0x7f1c9815b860 2024-05-11T08:07:19.115 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:19.112+0000 7f1c96ffd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1c980736a0 0x7f1c9815c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:19.115 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:19.112+0000 7f1c96ffd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1c980736a0 0x7f1c9815c3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:56656/0 (socket says 172.21.15.167:56656) 2024-05-11T08:07:19.115 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:19.112+0000 7f1c96ffd640 1 -- 172.21.15.167:0/3926740230 learned_addr learned my addr 172.21.15.167:0/3926740230 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:07:19.115 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:19.112+0000 7f1c977fe640 1 --2- 172.21.15.167:0/3926740230 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1c98072640 0x7f1c9814ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:19.115 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:19.112+0000 7f1c97fff640 1 --2- 172.21.15.167:0/3926740230 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1c9815b860 0x7f1c981f6a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:19.115 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:19.112+0000 7f1c96ffd640 1 -- 172.21.15.167:0/3926740230 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1c9815b860 msgr2=0x7f1c981f6a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:19.115 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:19.112+0000 7f1c96ffd640 1 --2- 172.21.15.167:0/3926740230 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1c9815b860 0x7f1c981f6a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:19.115 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:19.112+0000 7f1c96ffd640 1 -- 172.21.15.167:0/3926740230 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1c98072640 msgr2=0x7f1c9814ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:19.116 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:19.112+0000 7f1c96ffd640 1 --2- 172.21.15.167:0/3926740230 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1c98072640 0x7f1c9814ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:19.116 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:19.112+0000 7f1c96ffd640 1 -- 172.21.15.167:0/3926740230 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c9814b310 con 0x7f1c980736a0 2024-05-11T08:07:19.116 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:19.112+0000 7f1c977fe640 1 --2- 172.21.15.167:0/3926740230 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1c98072640 0x7f1c9814ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T08:07:19.116 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:19.112+0000 7f1c97fff640 1 --2- 172.21.15.167:0/3926740230 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1c9815b860 0x7f1c981f6a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T08:07:19.116 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:19.112+0000 7f1c96ffd640 1 --2- 172.21.15.167:0/3926740230 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1c980736a0 0x7f1c9815c3c0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f1c8000d7d0 tx=0x7f1c8000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:19.116 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:19.112+0000 7f1c8ffff640 1 -- 172.21.15.167:0/3926740230 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c8000b6c0 con 0x7f1c980736a0 2024-05-11T08:07:19.116 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:19.112+0000 7f1c8ffff640 1 -- 172.21.15.167:0/3926740230 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c80019070 con 0x7f1c980736a0 2024-05-11T08:07:19.116 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:19.112+0000 7f1c9e015640 1 -- 172.21.15.167:0/3926740230 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c9814b5f0 con 0x7f1c980736a0 2024-05-11T08:07:19.116 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:19.112+0000 7f1c8ffff640 1 -- 172.21.15.167:0/3926740230 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c8000ba80 con 0x7f1c980736a0 2024-05-11T08:07:19.116 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:19.112+0000 7f1c9e015640 1 -- 172.21.15.167:0/3926740230 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c9815e460 con 0x7f1c980736a0 2024-05-11T08:07:19.116 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:19.113+0000 7f1c9e015640 1 -- 172.21.15.167:0/3926740230 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c98069590 con 0x7f1c980736a0 2024-05-11T08:07:19.116 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:19.114+0000 7f1c8ffff640 1 -- 172.21.15.167:0/3926740230 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f1c8000bc20 con 0x7f1c980736a0 2024-05-11T08:07:19.117 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:19.114+0000 7f1c8ffff640 1 --2- 172.21.15.167:0/3926740230 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f1c780762d0 0x7f1c78078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:19.117 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:19.115+0000 7f1c8ffff640 1 -- 172.21.15.167:0/3926740230 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5335+0+0 (secure 0 0 0) 0x7f1c800978a0 con 0x7f1c980736a0 2024-05-11T08:07:19.117 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:19.115+0000 7f1c977fe640 1 --2- 172.21.15.167:0/3926740230 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f1c780762d0 0x7f1c78078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:19.117 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:19.115+0000 7f1c977fe640 1 --2- 172.21.15.167:0/3926740230 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f1c780762d0 0x7f1c78078790 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f1c88033000 tx=0x7f1c88009000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:19.120 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:19.119+0000 7f1c8ffff640 1 -- 172.21.15.167:0/3926740230 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f1c80061400 con 0x7f1c980736a0 2024-05-11T08:07:19.307 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:19.306+0000 7f1c9e015640 1 -- 172.21.15.167:0/3926740230 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1c980cf1d0 con 0x7f1c980736a0 2024-05-11T08:07:19.308 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:19.307+0000 7f1c8ffff640 1 -- 172.21.15.167:0/3926740230 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v48) ==== 74+0+13341 (secure 0 0 0) 0x7f1c80060d00 con 0x7f1c980736a0 2024-05-11T08:07:19.308 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T08:07:19.308 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":48,"fsid":"2034778d-8ff1-4823-aa44-ded32b518241","created":"2024-05-11T08:05:31.612421+0000","modified":"2024-05-11T08:07:15.148924+0000","last_up_change":"2024-05-11T08:05:42.557382+0000","last_in_change":"2024-05-11T08:05:33.757311+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-11T08:05:43.931263+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"587eaced-5ded-406d-884b-30c1caa8686b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":3863307356},{"type":"v1","addr":"172.21.15.167:6806","nonce":3863307356}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":3863307356},{"type":"v1","addr":"172.21.15.167:6810","nonce":3863307356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":3863307356},{"type":"v1","addr":"172.21.15.167:6815","nonce":3863307356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":3863307356},{"type":"v1","addr":"172.21.15.167:6813","nonce":3863307356}]},"public_addr":"172.21.15.167:6806/3863307356","cluster_addr":"172.21.15.167:6810/3863307356","heartbeat_back_addr":"172.21.15.167:6815/3863307356","heartbeat_front_addr":"172.21.15.167:6813/3863307356","state":["exists","up"]},{"osd":1,"uuid":"37c59104-2dd0-4cf9-85ea-56636f6b02a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":1585024048},{"type":"v1","addr":"172.21.15.167:6817","nonce":1585024048}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1585024048},{"type":"v1","addr":"172.21.15.167:6819","nonce":1585024048}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1585024048},{"type":"v1","addr":"172.21.15.167:6823","nonce":1585024048}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":1585024048},{"type":"v1","addr":"172.21.15.167:6821","nonce":1585024048}]},"public_addr":"172.21.15.167:6817/1585024048","cluster_addr":"172.21.15.167:6819/1585024048","heartbeat_back_addr":"172.21.15.167:6823/1585024048","heartbeat_front_addr":"172.21.15.167:6821/1585024048","state":["exists","up"]},{"osd":2,"uuid":"2bb4f7f5-c698-451b-b4bb-78e716921291","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":3242166900},{"type":"v1","addr":"172.21.15.167:6801","nonce":3242166900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":3242166900},{"type":"v1","addr":"172.21.15.167:6803","nonce":3242166900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6809","nonce":3242166900},{"type":"v1","addr":"172.21.15.167:6811","nonce":3242166900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6805","nonce":3242166900},{"type":"v1","addr":"172.21.15.167:6807","nonce":3242166900}]},"public_addr":"172.21.15.167:6801/3242166900","cluster_addr":"172.21.15.167:6803/3242166900","heartbeat_back_addr":"172.21.15.167:6811/3242166900","heartbeat_front_addr":"172.21.15.167:6807/3242166900","state":["exists","up"]},{"osd":3,"uuid":"9dd26759-a338-49d2-8103-45d068b9c000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":453689293},{"type":"v1","addr":"172.21.15.167:6825","nonce":453689293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":453689293},{"type":"v1","addr":"172.21.15.167:6827","nonce":453689293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":453689293},{"type":"v1","addr":"172.21.15.167:6831","nonce":453689293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":453689293},{"type":"v1","addr":"172.21.15.167:6829","nonce":453689293}]},"public_addr":"172.21.15.167:6825/453689293","cluster_addr":"172.21.15.167:6827/453689293","heartbeat_back_addr":"172.21.15.167:6831/453689293","heartbeat_front_addr":"172.21.15.167:6829/453689293","state":["exists","up"]},{"osd":4,"uuid":"cb27250d-2a80-4c7b-8cef-3a7af05c8a16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1311831500},{"type":"v1","addr":"172.21.15.204:6801","nonce":1311831500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1311831500},{"type":"v1","addr":"172.21.15.204:6803","nonce":1311831500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1311831500},{"type":"v1","addr":"172.21.15.204:6807","nonce":1311831500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1311831500},{"type":"v1","addr":"172.21.15.204:6805","nonce":1311831500}]},"public_addr":"172.21.15.204:6801/1311831500","cluster_addr":"172.21.15.204:6803/1311831500","heartbeat_back_addr":"172.21.15.204:6807/1311831500","heartbeat_front_addr":"172.21.15.204:6805/1311831500","state":["exists","up"]},{"osd":5,"uuid":"92998541-ffec-430b-ab76-8c3cb7c2752d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2921847521},{"type":"v1","addr":"172.21.15.204:6825","nonce":2921847521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2921847521},{"type":"v1","addr":"172.21.15.204:6827","nonce":2921847521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2921847521},{"type":"v1","addr":"172.21.15.204:6831","nonce":2921847521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2921847521},{"type":"v1","addr":"172.21.15.204:6829","nonce":2921847521}]},"public_addr":"172.21.15.204:6825/2921847521","cluster_addr":"172.21.15.204:6827/2921847521","heartbeat_back_addr":"172.21.15.204:6831/2921847521","heartbeat_front_addr":"172.21.15.204:6829/2921847521","state":["exists","up"]},{"osd":6,"uuid":"9543479c-2ec5-42a8-8933-707bfed2e048","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":2889851077},{"type":"v1","addr":"172.21.15.204:6817","nonce":2889851077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":2889851077},{"type":"v1","addr":"172.21.15.204:6819","nonce":2889851077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":2889851077},{"type":"v1","addr":"172.21.15.204:6823","nonce":2889851077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":2889851077},{"type":"v1","addr":"172.21.15.204:6821","nonce":2889851077}]},"public_addr":"172.21.15.204:6817/2889851077","cluster_addr":"172.21.15.204:6819/2889851077","heartbeat_back_addr":"172.21.15.204:6823/2889851077","heartbeat_front_addr":"172.21.15.204:6821/2889851077","state":["exists","up"]},{"osd":7,"uuid":"c45a5647-8358-409e-af91-c05dda65f0b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":1949534184},{"type":"v1","addr":"172.21.15.204:6809","nonce":1949534184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":1949534184},{"type":"v1","addr":"172.21.15.204:6811","nonce":1949534184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1949534184},{"type":"v1","addr":"172.21.15.204:6815","nonce":1949534184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1949534184},{"type":"v1","addr":"172.21.15.204:6813","nonce":1949534184}]},"public_addr":"172.21.15.204:6809/1949534184","cluster_addr":"172.21.15.204:6811/1949534184","heartbeat_back_addr":"172.21.15.204:6815/1949534184","heartbeat_front_addr":"172.21.15.204:6813/1949534184","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.952253+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.940689+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.860682+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.929350+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.901627+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.957451+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.987249+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T08:05:38.951197+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.204:6835/469864006":"2024-05-12T08:07:11.138945+0000","172.21.15.204:6833/469864006":"2024-05-12T08:07:11.138945+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-11T08:07:19.310 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:19.309+0000 7f1c9e015640 1 -- 172.21.15.167:0/3926740230 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f1c780762d0 msgr2=0x7f1c78078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:19.310 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:19.309+0000 7f1c9e015640 1 --2- 172.21.15.167:0/3926740230 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f1c780762d0 0x7f1c78078790 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f1c88033000 tx=0x7f1c88009000 comp rx=0 tx=0).stop 2024-05-11T08:07:19.310 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:19.309+0000 7f1c9e015640 1 -- 172.21.15.167:0/3926740230 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1c980736a0 msgr2=0x7f1c9815c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:19.310 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:19.309+0000 7f1c9e015640 1 --2- 172.21.15.167:0/3926740230 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1c980736a0 0x7f1c9815c3c0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f1c8000d7d0 tx=0x7f1c8000dc90 comp rx=0 tx=0).stop 2024-05-11T08:07:19.311 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:19.309+0000 7f1c9e015640 1 -- 172.21.15.167:0/3926740230 shutdown_connections 2024-05-11T08:07:19.311 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:19.309+0000 7f1c9e015640 1 --2- 172.21.15.167:0/3926740230 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f1c780762d0 0x7f1c78078790 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:19.311 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:19.309+0000 7f1c9e015640 1 --2- 172.21.15.167:0/3926740230 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1c9815b860 0x7f1c981f6a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:19.311 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:19.309+0000 7f1c9e015640 1 --2- 172.21.15.167:0/3926740230 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1c980736a0 0x7f1c9815c3c0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:19.311 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:19.309+0000 7f1c9e015640 1 --2- 172.21.15.167:0/3926740230 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1c98072640 0x7f1c9814ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:19.311 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:19.309+0000 7f1c9e015640 1 -- 172.21.15.167:0/3926740230 >> 172.21.15.167:0/3926740230 conn(0x7f1c980cd8f0 msgr2=0x7f1c981482e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:07:19.311 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:19.309+0000 7f1c9e015640 1 -- 172.21.15.167:0/3926740230 shutdown_connections 2024-05-11T08:07:19.311 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:19.309+0000 7f1c9e015640 1 -- 172.21.15.167:0/3926740230 wait complete. 2024-05-11T08:07:20.322 INFO:tasks.ceph:Scrubbing osd.0 2024-05-11T08:07:20.322 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-05-11T08:07:20.440 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.440+0000 7f811d493640 1 Processor -- start 2024-05-11T08:07:20.440 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.440+0000 7f811d493640 1 -- start start 2024-05-11T08:07:20.444 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.441+0000 7f811d493640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f81181628c0 0x7f81180d3da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:20.444 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.441+0000 7f811d493640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81180d0630 con 0x7f81180b4460 2024-05-11T08:07:20.444 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.441+0000 7f811d493640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8118163420 con 0x7f81180b3f10 2024-05-11T08:07:20.444 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.441+0000 7f811d493640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f8118164030 con 0x7f81181628c0 2024-05-11T08:07:20.444 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.441+0000 7f810e7fc640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f81180b4460 0x7f81180cf810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:59740/0 (socket says 172.21.15.167:59740) 2024-05-11T08:07:20.444 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.441+0000 7f810e7fc640 1 -- 172.21.15.167:0/1475483570 learned_addr learned my addr 172.21.15.167:0/1475483570 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T08:07:20.444 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.441+0000 7f8116ffd640 1 --2- 172.21.15.167:0/1475483570 >> v2:172.21.15.204:3300/0 conn(0x7f81181628c0 0x7f81180d3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:20.444 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.441+0000 7f81167fc640 1 -- 172.21.15.167:0/1475483570 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3759932067 0 0) 0x7f81180d0630 con 0x7f81180b4460 2024-05-11T08:07:20.444 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.441+0000 7f81167fc640 1 -- 172.21.15.167:0/1475483570 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8104003520 con 0x7f81180b4460 2024-05-11T08:07:20.445 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.441+0000 7f81167fc640 1 -- 172.21.15.167:0/1475483570 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2621090814 0 0) 0x7f8118163420 con 0x7f81180b3f10 2024-05-11T08:07:20.445 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.441+0000 7f81167fc640 1 -- 172.21.15.167:0/1475483570 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f81180d0630 con 0x7f81180b3f10 2024-05-11T08:07:20.445 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.441+0000 7f81167fc640 1 -- 172.21.15.167:0/1475483570 <== mon.0 v1:172.21.15.167:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3284554873 0 0) 0x7f8104003520 con 0x7f81180b4460 2024-05-11T08:07:20.445 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.441+0000 7f81167fc640 1 -- 172.21.15.167:0/1475483570 >> v2:172.21.15.204:3300/0 conn(0x7f81181628c0 msgr2=0x7f81180d3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:07:20.445 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.441+0000 7f81167fc640 1 --2- 172.21.15.167:0/1475483570 >> v2:172.21.15.204:3300/0 conn(0x7f81181628c0 0x7f81180d3da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:20.445 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.441+0000 7f81167fc640 1 -- 172.21.15.167:0/1475483570 >> v1:172.21.15.204:6789/0 conn(0x7f81180b3f10 legacy=0x7f81180cff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:20.445 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.441+0000 7f81167fc640 1 -- 172.21.15.167:0/1475483570 --> v1:172.21.15.167:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8118163700 con 0x7f81180b4460 2024-05-11T08:07:20.445 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.441+0000 7f81167fc640 1 -- 172.21.15.167:0/1475483570 <== mon.0 v1:172.21.15.167:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3602277846 0 0) 0x7f81000016f0 con 0x7f81180b4460 2024-05-11T08:07:20.445 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.441+0000 7f81167fc640 1 -- 172.21.15.167:0/1475483570 >> v1:172.21.15.167:6789/0 conn(0x7f81180b4460 legacy=0x7f81180cf810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:20.445 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.441+0000 7f81167fc640 1 --2- 172.21.15.167:0/1475483570 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8104003c30 0x7f8104004030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:20.445 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.441+0000 7f81167fc640 1 --2- 172.21.15.167:0/1475483570 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8104004800 0x7f8104004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:20.445 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.441+0000 7f81167fc640 1 --2- 172.21.15.167:0/1475483570 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8104005210 0x7f810400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:20.445 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.441+0000 7f81167fc640 1 -- 172.21.15.167:0/1475483570 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f810400c110 con 0x7f8104003c30 2024-05-11T08:07:20.445 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.441+0000 7f81167fc640 1 -- 172.21.15.167:0/1475483570 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f810400c2c0 con 0x7f8104004800 2024-05-11T08:07:20.446 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.441+0000 7f81167fc640 1 -- 172.21.15.167:0/1475483570 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f810400c4c0 con 0x7f8104005210 2024-05-11T08:07:20.446 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.441+0000 7f8116ffd640 1 --2- 172.21.15.167:0/1475483570 >> v2:172.21.15.204:3300/0 conn(0x7f81181628c0 0x7f81180d3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T08:07:20.446 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.442+0000 7f810e7fc640 1 --2- 172.21.15.167:0/1475483570 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8104004800 0x7f8104004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:20.446 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.442+0000 7f8116ffd640 1 --2- 172.21.15.167:0/1475483570 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8104003c30 0x7f8104004030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:20.446 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.442+0000 7f81177fe640 1 --2- 172.21.15.167:0/1475483570 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8104005210 0x7f810400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:20.446 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.442+0000 7f8116ffd640 1 -- 172.21.15.167:0/1475483570 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8104005210 msgr2=0x7f810400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:07:20.446 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.442+0000 7f8116ffd640 1 --2- 172.21.15.167:0/1475483570 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8104005210 0x7f810400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:20.446 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.442+0000 7f8116ffd640 1 -- 172.21.15.167:0/1475483570 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8104004800 msgr2=0x7f8104004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:07:20.446 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.442+0000 7f8116ffd640 1 --2- 172.21.15.167:0/1475483570 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8104004800 0x7f8104004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:20.446 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.442+0000 7f8116ffd640 1 -- 172.21.15.167:0/1475483570 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8104003520 con 0x7f8104003c30 2024-05-11T08:07:20.446 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.442+0000 7f81177fe640 1 -- 172.21.15.167:0/1475483570 reap_dead start 2024-05-11T08:07:20.446 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.443+0000 7f8116ffd640 1 --2- 172.21.15.167:0/1475483570 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8104003c30 0x7f8104004030 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f8108009c70 tx=0x7f8104005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99aeef2570f3ba86 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:20.446 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.443+0000 7f81167fc640 1 -- 172.21.15.167:0/1475483570 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8108009840 con 0x7f8104003c30 2024-05-11T08:07:20.446 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.443+0000 7f81167fc640 1 -- 172.21.15.167:0/1475483570 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8108004020 con 0x7f8104003c30 2024-05-11T08:07:20.447 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.443+0000 7f81167fc640 1 -- 172.21.15.167:0/1475483570 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81080043d0 con 0x7f8104003c30 2024-05-11T08:07:20.447 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.443+0000 7f811d493640 1 -- 172.21.15.167:0/1475483570 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8104003c30 msgr2=0x7f8104004030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:20.447 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.443+0000 7f811d493640 1 --2- 172.21.15.167:0/1475483570 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8104003c30 0x7f8104004030 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f8108009c70 tx=0x7f8104005830 comp rx=0 tx=0).stop 2024-05-11T08:07:20.447 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.443+0000 7f811d493640 1 -- 172.21.15.167:0/1475483570 shutdown_connections 2024-05-11T08:07:20.447 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.443+0000 7f811d493640 1 --2- 172.21.15.167:0/1475483570 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8104003c30 0x7f8104004030 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:20.447 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.443+0000 7f811d493640 1 -- 172.21.15.167:0/1475483570 >> 172.21.15.167:0/1475483570 conn(0x7f81180b6cb0 msgr2=0x7f81180b7090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:07:20.447 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.443+0000 7f811d493640 1 -- 172.21.15.167:0/1475483570 shutdown_connections 2024-05-11T08:07:20.447 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.443+0000 7f811d493640 1 -- 172.21.15.167:0/1475483570 wait complete. 2024-05-11T08:07:20.450 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.445+0000 7f811d493640 1 Processor -- start 2024-05-11T08:07:20.450 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.445+0000 7f811d493640 1 -- start start 2024-05-11T08:07:20.450 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.446+0000 7f811d493640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8104003c30 0x7f8118159dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:20.450 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.446+0000 7f811d493640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8118158e70 0x7f811814c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:20.450 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.446+0000 7f811d493640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f811815c550 0x7f811815c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:20.450 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.446+0000 7f811d493640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f81181630d0 con 0x7f811815c550 2024-05-11T08:07:20.451 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.446+0000 7f811d493640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8118163cc0 con 0x7f8118158e70 2024-05-11T08:07:20.451 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.446+0000 7f811d493640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f8118162ca0 con 0x7f8104003c30 2024-05-11T08:07:20.451 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.446+0000 7f81177fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f811815c550 0x7f811815c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:20.451 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.446+0000 7f81177fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f811815c550 0x7f811815c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:56674/0 (socket says 172.21.15.167:56674) 2024-05-11T08:07:20.451 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.446+0000 7f8116ffd640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8104003c30 0x7f8118159dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:20.451 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.446+0000 7f81177fe640 1 -- 172.21.15.167:0/78408712 learned_addr learned my addr 172.21.15.167:0/78408712 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:07:20.451 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.446+0000 7f810e7fc640 1 --2- 172.21.15.167:0/78408712 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8118158e70 0x7f811814c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:20.451 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.446+0000 7f81177fe640 1 -- 172.21.15.167:0/78408712 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8104003c30 msgr2=0x7f8118159dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:20.451 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.446+0000 7f81177fe640 1 --2- 172.21.15.167:0/78408712 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8104003c30 0x7f8118159dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:20.451 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.446+0000 7f81177fe640 1 -- 172.21.15.167:0/78408712 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8118158e70 msgr2=0x7f811814c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:20.451 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.446+0000 7f81177fe640 1 --2- 172.21.15.167:0/78408712 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8118158e70 0x7f811814c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:20.451 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.446+0000 7f81177fe640 1 -- 172.21.15.167:0/78408712 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f811814cba0 con 0x7f811815c550 2024-05-11T08:07:20.451 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.446+0000 7f81177fe640 1 --2- 172.21.15.167:0/78408712 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f811815c550 0x7f811815c9c0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f81180b7cb0 tx=0x7f811000ff70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:20.451 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.447+0000 7f810ffff640 1 -- 172.21.15.167:0/78408712 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8110003b80 con 0x7f811815c550 2024-05-11T08:07:20.451 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.447+0000 7f810ffff640 1 -- 172.21.15.167:0/78408712 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8110018070 con 0x7f811815c550 2024-05-11T08:07:20.452 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.447+0000 7f811d493640 1 -- 172.21.15.167:0/78408712 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f811815c2e0 con 0x7f811815c550 2024-05-11T08:07:20.452 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.447+0000 7f810ffff640 1 -- 172.21.15.167:0/78408712 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81100050d0 con 0x7f811815c550 2024-05-11T08:07:20.452 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.447+0000 7f811d493640 1 -- 172.21.15.167:0/78408712 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8118158390 con 0x7f811815c550 2024-05-11T08:07:20.452 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.447+0000 7f811d493640 1 -- 172.21.15.167:0/78408712 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8118158a50 con 0x7f811815c550 2024-05-11T08:07:20.452 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.448+0000 7f810ffff640 1 -- 172.21.15.167:0/78408712 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f8110005270 con 0x7f811815c550 2024-05-11T08:07:20.452 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.449+0000 7f810ffff640 1 --2- 172.21.15.167:0/78408712 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f80f80762d0 0x7f80f8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:20.452 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.449+0000 7f8116ffd640 1 --2- 172.21.15.167:0/78408712 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f80f80762d0 0x7f80f8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:20.452 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.449+0000 7f810ffff640 1 -- 172.21.15.167:0/78408712 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5335+0+0 (secure 0 0 0) 0x7f8110097330 con 0x7f811815c550 2024-05-11T08:07:20.452 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.449+0000 7f810ffff640 1 --2- 172.21.15.167:0/78408712 >> [v2:172.21.15.167:6804/3863307356,v1:172.21.15.167:6806/3863307356] conn(0x7f80f807c770 0x7f80f807ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:20.452 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.449+0000 7f810ffff640 1 -- 172.21.15.167:0/78408712 --> [v2:172.21.15.167:6804/3863307356,v1:172.21.15.167:6806/3863307356] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f80f807f280 con 0x7f80f807c770 2024-05-11T08:07:20.452 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.449+0000 7f810ffff640 1 -- 172.21.15.167:0/78408712 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f811009e050 con 0x7f811815c550 2024-05-11T08:07:20.452 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.449+0000 7f810e7fc640 1 --2- 172.21.15.167:0/78408712 >> [v2:172.21.15.167:6804/3863307356,v1:172.21.15.167:6806/3863307356] conn(0x7f80f807c770 0x7f80f807ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:20.452 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.449+0000 7f8116ffd640 1 --2- 172.21.15.167:0/78408712 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f80f80762d0 0x7f80f8078790 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f8108032000 tx=0x7f8108008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:20.452 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.449+0000 7f810e7fc640 1 --2- 172.21.15.167:0/78408712 >> [v2:172.21.15.167:6804/3863307356,v1:172.21.15.167:6806/3863307356] conn(0x7f80f807c770 0x7f80f807ebb0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:20.453 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.450+0000 7f810ffff640 1 -- 172.21.15.167:0/78408712 <== osd.0 v2:172.21.15.167:6804/3863307356 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f80f807f280 con 0x7f80f807c770 2024-05-11T08:07:20.463 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.463+0000 7f811d493640 1 -- 172.21.15.167:0/78408712 --> [v2:172.21.15.167:6804/3863307356,v1:172.21.15.167:6806/3863307356] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f811806eb80 con 0x7f80f807c770 2024-05-11T08:07:20.463 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.463+0000 7f810ffff640 1 -- 172.21.15.167:0/78408712 <== osd.0 v2:172.21.15.167:6804/3863307356 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f811806eb80 con 0x7f80f807c770 2024-05-11T08:07:20.463 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-05-11T08:07:20.463 INFO:teuthology.orchestra.run.smithi167.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-11T08:07:20.464 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-05-11T08:07:20.464 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.463+0000 7f811d493640 1 -- 172.21.15.167:0/78408712 >> [v2:172.21.15.167:6804/3863307356,v1:172.21.15.167:6806/3863307356] conn(0x7f80f807c770 msgr2=0x7f80f807ebb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:20.464 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.463+0000 7f811d493640 1 --2- 172.21.15.167:0/78408712 >> [v2:172.21.15.167:6804/3863307356,v1:172.21.15.167:6806/3863307356] conn(0x7f80f807c770 0x7f80f807ebb0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:20.465 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.464+0000 7f811d493640 1 -- 172.21.15.167:0/78408712 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f80f80762d0 msgr2=0x7f80f8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:20.465 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.464+0000 7f811d493640 1 --2- 172.21.15.167:0/78408712 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f80f80762d0 0x7f80f8078790 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f8108032000 tx=0x7f8108008030 comp rx=0 tx=0).stop 2024-05-11T08:07:20.465 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.464+0000 7f811d493640 1 -- 172.21.15.167:0/78408712 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f811815c550 msgr2=0x7f811815c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:20.465 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.464+0000 7f811d493640 1 --2- 172.21.15.167:0/78408712 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f811815c550 0x7f811815c9c0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f81180b7cb0 tx=0x7f811000ff70 comp rx=0 tx=0).stop 2024-05-11T08:07:20.465 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.464+0000 7f81177fe640 1 -- 172.21.15.167:0/78408712 reap_dead start 2024-05-11T08:07:20.465 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.464+0000 7f811d493640 1 -- 172.21.15.167:0/78408712 shutdown_connections 2024-05-11T08:07:20.465 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.464+0000 7f811d493640 1 -- 172.21.15.167:0/78408712 >> 172.21.15.167:0/78408712 conn(0x7f81180b6cb0 msgr2=0x7f81180c9620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:07:20.465 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.464+0000 7f811d493640 1 -- 172.21.15.167:0/78408712 shutdown_connections 2024-05-11T08:07:20.465 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.464+0000 7f811d493640 1 -- 172.21.15.167:0/78408712 wait complete. 2024-05-11T08:07:20.475 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-05-11T08:07:20.576 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.575+0000 7f892ba59640 1 Processor -- start 2024-05-11T08:07:20.576 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.576+0000 7f892ba59640 1 -- start start 2024-05-11T08:07:20.577 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.576+0000 7f892ba59640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f892413eeb0 0x7f892406f480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:20.577 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.577+0000 7f892ba59640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f892413f290 0x7f892406f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:20.577 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.577+0000 7f892ba59640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f892415fb90 con 0x7f89240b50a0 2024-05-11T08:07:20.577 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.577+0000 7f892ba59640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f892415d5b0 con 0x7f892413eeb0 2024-05-11T08:07:20.577 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.577+0000 7f892ba59640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f892415fe70 con 0x7f892413f290 2024-05-11T08:07:20.578 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.577+0000 7f89297f5640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f892413eeb0 0x7f892406f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:20.578 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.577+0000 7f8929ff6640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f89240b50a0 0x7f89240735c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:59748/0 (socket says 172.21.15.167:59748) 2024-05-11T08:07:20.578 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.577+0000 7f89297f5640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f892413eeb0 0x7f892406f480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:56688/0 (socket says 172.21.15.167:56688) 2024-05-11T08:07:20.578 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.577+0000 7f8929ff6640 1 -- 172.21.15.167:0/2344915554 learned_addr learned my addr 172.21.15.167:0/2344915554 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T08:07:20.578 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.577+0000 7f8928ff4640 1 --2- 172.21.15.167:0/2344915554 >> v2:172.21.15.204:3300/0 conn(0x7f892413f290 0x7f892406f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:20.578 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.577+0000 7f89297f5640 1 -- 172.21.15.167:0/2344915554 >> v2:172.21.15.204:3300/0 conn(0x7f892413f290 msgr2=0x7f892406f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:07:20.578 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.577+0000 7f89297f5640 1 --2- 172.21.15.167:0/2344915554 >> v2:172.21.15.204:3300/0 conn(0x7f892413f290 0x7f892406f9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:20.578 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.577+0000 7f89297f5640 1 -- 172.21.15.167:0/2344915554 >> v1:172.21.15.167:6789/0 conn(0x7f89240b50a0 legacy=0x7f89240735c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:20.578 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.577+0000 7f8917fff640 1 -- 172.21.15.167:0/2344915554 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2556242365 0 0) 0x7f892415fb90 con 0x7f89240b50a0 2024-05-11T08:07:20.578 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.577+0000 7f89297f5640 1 -- 172.21.15.167:0/2344915554 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8924160150 con 0x7f892413eeb0 2024-05-11T08:07:20.580 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.578+0000 7f89297f5640 1 --2- 172.21.15.167:0/2344915554 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f892413eeb0 0x7f892406f480 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f8924074520 tx=0x7f891802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f55d3c2e22c2bd1d server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:20.580 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.578+0000 7f8917fff640 1 -- 172.21.15.167:0/2344915554 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8918037070 con 0x7f892413eeb0 2024-05-11T08:07:20.580 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.578+0000 7f8917fff640 1 -- 172.21.15.167:0/2344915554 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8918032440 con 0x7f892413eeb0 2024-05-11T08:07:20.580 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.578+0000 7f8917fff640 1 -- 172.21.15.167:0/2344915554 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f891802ae60 con 0x7f892413eeb0 2024-05-11T08:07:20.580 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.578+0000 7f892ba59640 1 -- 172.21.15.167:0/2344915554 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f892413eeb0 msgr2=0x7f892406f480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:20.580 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.578+0000 7f892ba59640 1 --2- 172.21.15.167:0/2344915554 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f892413eeb0 0x7f892406f480 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f8924074520 tx=0x7f891802a110 comp rx=0 tx=0).stop 2024-05-11T08:07:20.580 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.579+0000 7f892ba59640 1 -- 172.21.15.167:0/2344915554 shutdown_connections 2024-05-11T08:07:20.580 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.579+0000 7f892ba59640 1 --2- 172.21.15.167:0/2344915554 >> v2:172.21.15.204:3300/0 conn(0x7f892413f290 0x7f892406f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:20.580 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.579+0000 7f892ba59640 1 --2- 172.21.15.167:0/2344915554 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f892413eeb0 0x7f892406f480 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:20.580 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.579+0000 7f892ba59640 1 -- 172.21.15.167:0/2344915554 >> 172.21.15.167:0/2344915554 conn(0x7f89240cd8f0 msgr2=0x7f89240cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:07:20.580 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.579+0000 7f892ba59640 1 -- 172.21.15.167:0/2344915554 shutdown_connections 2024-05-11T08:07:20.580 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.579+0000 7f892ba59640 1 -- 172.21.15.167:0/2344915554 wait complete. 2024-05-11T08:07:20.580 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.580+0000 7f892ba59640 1 Processor -- start 2024-05-11T08:07:20.581 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.580+0000 7f892ba59640 1 -- start start 2024-05-11T08:07:20.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.581+0000 7f892ba59640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f89240b50a0 0x7f892414ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:20.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.581+0000 7f892ba59640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f892413eeb0 0x7f892414efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:20.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.581+0000 7f892ba59640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f892413f290 0x7f892412efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:20.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.581+0000 7f892ba59640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8924073cd0 con 0x7f89240b50a0 2024-05-11T08:07:20.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.581+0000 7f892ba59640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8924160710 con 0x7f892413f290 2024-05-11T08:07:20.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.581+0000 7f892ba59640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f8924073e50 con 0x7f892413eeb0 2024-05-11T08:07:20.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.581+0000 7f89297f5640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f89240b50a0 0x7f892414ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:20.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.581+0000 7f89297f5640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f89240b50a0 0x7f892414ea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:56698/0 (socket says 172.21.15.167:56698) 2024-05-11T08:07:20.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.581+0000 7f89297f5640 1 -- 172.21.15.167:0/2457283106 learned_addr learned my addr 172.21.15.167:0/2457283106 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:07:20.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.581+0000 7f8928ff4640 1 --2- 172.21.15.167:0/2457283106 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f892413eeb0 0x7f892414efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:20.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.581+0000 7f8929ff6640 1 --2- 172.21.15.167:0/2457283106 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f892413f290 0x7f892412efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:20.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.581+0000 7f89297f5640 1 -- 172.21.15.167:0/2457283106 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f892413eeb0 msgr2=0x7f892414efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:20.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.581+0000 7f89297f5640 1 --2- 172.21.15.167:0/2457283106 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f892413eeb0 0x7f892414efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:20.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.581+0000 7f89297f5640 1 -- 172.21.15.167:0/2457283106 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f892413f290 msgr2=0x7f892412efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:20.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.581+0000 7f89297f5640 1 --2- 172.21.15.167:0/2457283106 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f892413f290 0x7f892412efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:20.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.581+0000 7f89297f5640 1 -- 172.21.15.167:0/2457283106 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89241f4f10 con 0x7f89240b50a0 2024-05-11T08:07:20.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.581+0000 7f8929ff6640 1 --2- 172.21.15.167:0/2457283106 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f892413f290 0x7f892412efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T08:07:20.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.581+0000 7f89297f5640 1 --2- 172.21.15.167:0/2457283106 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f89240b50a0 0x7f892414ea80 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f8918000c00 tx=0x7f89180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:20.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.581+0000 7f8915ffb640 1 -- 172.21.15.167:0/2457283106 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8918037040 con 0x7f89240b50a0 2024-05-11T08:07:20.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.581+0000 7f8915ffb640 1 -- 172.21.15.167:0/2457283106 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8918042070 con 0x7f89240b50a0 2024-05-11T08:07:20.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.581+0000 7f8915ffb640 1 -- 172.21.15.167:0/2457283106 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8918030bd0 con 0x7f89240b50a0 2024-05-11T08:07:20.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.581+0000 7f892ba59640 1 -- 172.21.15.167:0/2457283106 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f892412fff0 con 0x7f89240b50a0 2024-05-11T08:07:20.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.582+0000 7f892ba59640 1 -- 172.21.15.167:0/2457283106 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89240cef00 con 0x7f89240b50a0 2024-05-11T08:07:20.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.582+0000 7f892ba59640 1 -- 172.21.15.167:0/2457283106 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89240cecf0 con 0x7f89240b50a0 2024-05-11T08:07:20.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.583+0000 7f8915ffb640 1 -- 172.21.15.167:0/2457283106 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f8918047460 con 0x7f89240b50a0 2024-05-11T08:07:20.585 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.583+0000 7f8915ffb640 1 --2- 172.21.15.167:0/2457283106 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f89080762d0 0x7f8908078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:20.585 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.584+0000 7f8928ff4640 1 --2- 172.21.15.167:0/2457283106 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f89080762d0 0x7f8908078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:20.585 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.584+0000 7f8915ffb640 1 -- 172.21.15.167:0/2457283106 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5335+0+0 (secure 0 0 0) 0x7f89180b8050 con 0x7f89240b50a0 2024-05-11T08:07:20.585 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.584+0000 7f8928ff4640 1 --2- 172.21.15.167:0/2457283106 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f89080762d0 0x7f8908078790 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f8920006fd0 tx=0x7f8920008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:20.588 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.588+0000 7f8915ffb640 1 -- 172.21.15.167:0/2457283106 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f8918032e20 con 0x7f89240b50a0 2024-05-11T08:07:20.768 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.767+0000 7f892ba59640 1 -- 172.21.15.167:0/2457283106 --> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7f8924069590 con 0x7f89080762d0 2024-05-11T08:07:20.768 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.768+0000 7f8915ffb640 1 -- 172.21.15.167:0/2457283106 <== mgr.4113 v2:172.21.15.167:6832/253569496 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f8924069590 con 0x7f89080762d0 2024-05-11T08:07:20.768 INFO:teuthology.orchestra.run.smithi167.stderr:instructed osd(s) 0 to deep-scrub 2024-05-11T08:07:20.770 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.770+0000 7f892ba59640 1 -- 172.21.15.167:0/2457283106 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f89080762d0 msgr2=0x7f8908078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:20.770 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.770+0000 7f892ba59640 1 --2- 172.21.15.167:0/2457283106 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f89080762d0 0x7f8908078790 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f8920006fd0 tx=0x7f8920008040 comp rx=0 tx=0).stop 2024-05-11T08:07:20.770 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.770+0000 7f892ba59640 1 -- 172.21.15.167:0/2457283106 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f89240b50a0 msgr2=0x7f892414ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:20.770 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.770+0000 7f892ba59640 1 --2- 172.21.15.167:0/2457283106 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f89240b50a0 0x7f892414ea80 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f8918000c00 tx=0x7f89180046c0 comp rx=0 tx=0).stop 2024-05-11T08:07:20.770 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.770+0000 7f892ba59640 1 -- 172.21.15.167:0/2457283106 shutdown_connections 2024-05-11T08:07:20.770 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.770+0000 7f892ba59640 1 --2- 172.21.15.167:0/2457283106 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f89080762d0 0x7f8908078790 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:20.771 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.770+0000 7f892ba59640 1 --2- 172.21.15.167:0/2457283106 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f892413f290 0x7f892412efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:20.771 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.770+0000 7f892ba59640 1 --2- 172.21.15.167:0/2457283106 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f892413eeb0 0x7f892414efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:20.771 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.770+0000 7f892ba59640 1 --2- 172.21.15.167:0/2457283106 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f89240b50a0 0x7f892414ea80 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:20.771 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.770+0000 7f892ba59640 1 -- 172.21.15.167:0/2457283106 >> 172.21.15.167:0/2457283106 conn(0x7f89240cd8f0 msgr2=0x7f892406e870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:07:20.771 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.770+0000 7f892ba59640 1 -- 172.21.15.167:0/2457283106 shutdown_connections 2024-05-11T08:07:20.771 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.770+0000 7f892ba59640 1 -- 172.21.15.167:0/2457283106 wait complete. 2024-05-11T08:07:20.781 INFO:tasks.ceph:Scrubbing osd.1 2024-05-11T08:07:20.781 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-05-11T08:07:20.885 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.885+0000 7fe7322bd640 1 Processor -- start 2024-05-11T08:07:20.885 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.885+0000 7fe7322bd640 1 -- start start 2024-05-11T08:07:20.886 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.886+0000 7fe7322bd640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fe72c148d50 0x7fe72c0cd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:20.886 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.886+0000 7fe7322bd640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe72c0cfad0 con 0x7fe72c163580 2024-05-11T08:07:20.886 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.886+0000 7fe7322bd640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe72c0cfdb0 con 0x7fe72c0d2630 2024-05-11T08:07:20.887 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.886+0000 7fe7322bd640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7fe72c0cda40 con 0x7fe72c148d50 2024-05-11T08:07:20.887 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.886+0000 7fe72bfff640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fe72c148d50 0x7fe72c0cd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:20.887 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.886+0000 7fe73085a640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7fe72c163580 0x7fe72c0d0e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:59762/0 (socket says 172.21.15.167:59762) 2024-05-11T08:07:20.887 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.886+0000 7fe73085a640 1 -- 172.21.15.167:0/3286766665 learned_addr learned my addr 172.21.15.167:0/3286766665 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T08:07:20.887 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.886+0000 7fe72bfff640 1 -- 172.21.15.167:0/3286766665 >> v1:172.21.15.204:6789/0 conn(0x7fe72c0d2630 legacy=0x7fe72c162e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:07:20.887 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.886+0000 7fe72bfff640 1 -- 172.21.15.167:0/3286766665 >> v1:172.21.15.167:6789/0 conn(0x7fe72c163580 legacy=0x7fe72c0d0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:20.887 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.886+0000 7fe72affd640 1 -- 172.21.15.167:0/3286766665 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2826079916 0 0) 0x7fe72c0cfad0 con 0x7fe72c163580 2024-05-11T08:07:20.887 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.886+0000 7fe72bfff640 1 -- 172.21.15.167:0/3286766665 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe72c0d0090 con 0x7fe72c148d50 2024-05-11T08:07:20.888 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.887+0000 7fe72bfff640 1 --2- 172.21.15.167:0/3286766665 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe72c148d50 0x7fe72c0cd500 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fe71c0097a0 tx=0x7fe71c02a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9e4b385a3fbf08c server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:20.888 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.887+0000 7fe72affd640 1 -- 172.21.15.167:0/3286766665 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe71c037070 con 0x7fe72c148d50 2024-05-11T08:07:20.888 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.887+0000 7fe72affd640 1 -- 172.21.15.167:0/3286766665 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe71c02adb0 con 0x7fe72c148d50 2024-05-11T08:07:20.888 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.887+0000 7fe72affd640 1 -- 172.21.15.167:0/3286766665 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe71c032660 con 0x7fe72c148d50 2024-05-11T08:07:20.889 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.888+0000 7fe7322bd640 1 -- 172.21.15.167:0/3286766665 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe72c148d50 msgr2=0x7fe72c0cd500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:20.889 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.888+0000 7fe7322bd640 1 --2- 172.21.15.167:0/3286766665 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe72c148d50 0x7fe72c0cd500 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fe71c0097a0 tx=0x7fe71c02a060 comp rx=0 tx=0).stop 2024-05-11T08:07:20.889 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.888+0000 7fe7322bd640 1 -- 172.21.15.167:0/3286766665 shutdown_connections 2024-05-11T08:07:20.889 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.888+0000 7fe7322bd640 1 --2- 172.21.15.167:0/3286766665 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe72c148d50 0x7fe72c0cd500 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:20.889 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.888+0000 7fe7322bd640 1 -- 172.21.15.167:0/3286766665 >> 172.21.15.167:0/3286766665 conn(0x7fe72c0b4ae0 msgr2=0x7fe72c0b4ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:07:20.889 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.888+0000 7fe7322bd640 1 -- 172.21.15.167:0/3286766665 shutdown_connections 2024-05-11T08:07:20.889 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.888+0000 7fe7322bd640 1 -- 172.21.15.167:0/3286766665 wait complete. 2024-05-11T08:07:20.891 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.889+0000 7fe7322bd640 1 Processor -- start 2024-05-11T08:07:20.891 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.889+0000 7fe7322bd640 1 -- start start 2024-05-11T08:07:20.892 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.890+0000 7fe7322bd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe72c0d2630 0x7fe72c0b7b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:20.892 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.890+0000 7fe7322bd640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe72c148d50 0x7fe72c0c9f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:20.892 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.890+0000 7fe7322bd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe72c163580 0x7fe72c1527a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:20.892 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.890+0000 7fe7322bd640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe72c0d1550 con 0x7fe72c163580 2024-05-11T08:07:20.892 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.890+0000 7fe7322bd640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe72c0b5de0 con 0x7fe72c0d2630 2024-05-11T08:07:20.892 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.890+0000 7fe7322bd640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fe72c0d16d0 con 0x7fe72c148d50 2024-05-11T08:07:20.892 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.890+0000 7fe73085a640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe72c163580 0x7fe72c1527a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:20.892 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.890+0000 7fe73085a640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe72c163580 0x7fe72c1527a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:56714/0 (socket says 172.21.15.167:56714) 2024-05-11T08:07:20.892 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.890+0000 7fe73085a640 1 -- 172.21.15.167:0/1760842058 learned_addr learned my addr 172.21.15.167:0/1760842058 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:07:20.892 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.890+0000 7fe72bfff640 1 --2- 172.21.15.167:0/1760842058 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe72c0d2630 0x7fe72c0b7b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:20.892 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.890+0000 7fe72b7fe640 1 --2- 172.21.15.167:0/1760842058 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe72c148d50 0x7fe72c0c9f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:20.892 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.890+0000 7fe73085a640 1 -- 172.21.15.167:0/1760842058 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe72c148d50 msgr2=0x7fe72c0c9f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:20.892 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.890+0000 7fe73085a640 1 --2- 172.21.15.167:0/1760842058 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe72c148d50 0x7fe72c0c9f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:20.892 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.890+0000 7fe73085a640 1 -- 172.21.15.167:0/1760842058 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe72c0d2630 msgr2=0x7fe72c0b7b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:20.893 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.890+0000 7fe73085a640 1 --2- 172.21.15.167:0/1760842058 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe72c0d2630 0x7fe72c0b7b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:20.893 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.890+0000 7fe73085a640 1 -- 172.21.15.167:0/1760842058 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe72c0caa00 con 0x7fe72c163580 2024-05-11T08:07:20.893 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.890+0000 7fe72b7fe640 1 --2- 172.21.15.167:0/1760842058 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe72c148d50 0x7fe72c0c9f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T08:07:20.893 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.890+0000 7fe73085a640 1 --2- 172.21.15.167:0/1760842058 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe72c163580 0x7fe72c1527a0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fe72400c6b0 tx=0x7fe72400cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:20.893 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.891+0000 7fe728ff9640 1 -- 172.21.15.167:0/1760842058 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe724018070 con 0x7fe72c163580 2024-05-11T08:07:20.893 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.891+0000 7fe728ff9640 1 -- 172.21.15.167:0/1760842058 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe724004da0 con 0x7fe72c163580 2024-05-11T08:07:20.893 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.891+0000 7fe7322bd640 1 -- 172.21.15.167:0/1760842058 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe72c0cace0 con 0x7fe72c163580 2024-05-11T08:07:20.893 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.891+0000 7fe728ff9640 1 -- 172.21.15.167:0/1760842058 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe72400a820 con 0x7fe72c163580 2024-05-11T08:07:20.893 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.891+0000 7fe7322bd640 1 -- 172.21.15.167:0/1760842058 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe72c0cb580 con 0x7fe72c163580 2024-05-11T08:07:20.893 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.891+0000 7fe7322bd640 1 -- 172.21.15.167:0/1760842058 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe72c0cb0d0 con 0x7fe72c163580 2024-05-11T08:07:20.895 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.892+0000 7fe728ff9640 1 -- 172.21.15.167:0/1760842058 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fe72401b070 con 0x7fe72c163580 2024-05-11T08:07:20.895 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.893+0000 7fe728ff9640 1 --2- 172.21.15.167:0/1760842058 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fe7180762d0 0x7fe718078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:20.895 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.893+0000 7fe728ff9640 1 -- 172.21.15.167:0/1760842058 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5335+0+0 (secure 0 0 0) 0x7fe724097120 con 0x7fe72c163580 2024-05-11T08:07:20.895 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.893+0000 7fe72bfff640 1 --2- 172.21.15.167:0/1760842058 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fe7180762d0 0x7fe718078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:20.895 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.893+0000 7fe728ff9640 1 --2- 172.21.15.167:0/1760842058 >> [v2:172.21.15.167:6816/1585024048,v1:172.21.15.167:6817/1585024048] conn(0x7fe71807c770 0x7fe71807ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:20.895 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.893+0000 7fe728ff9640 1 -- 172.21.15.167:0/1760842058 --> [v2:172.21.15.167:6816/1585024048,v1:172.21.15.167:6817/1585024048] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe724015ec0 con 0x7fe71807c770 2024-05-11T08:07:20.895 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.893+0000 7fe728ff9640 1 -- 172.21.15.167:0/1760842058 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fe7240975b0 con 0x7fe72c163580 2024-05-11T08:07:20.895 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.893+0000 7fe72b7fe640 1 --2- 172.21.15.167:0/1760842058 >> [v2:172.21.15.167:6816/1585024048,v1:172.21.15.167:6817/1585024048] conn(0x7fe71807c770 0x7fe71807ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:20.895 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.893+0000 7fe72bfff640 1 --2- 172.21.15.167:0/1760842058 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fe7180762d0 0x7fe718078790 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fe71c034000 tx=0x7fe71c02e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:20.895 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.893+0000 7fe72b7fe640 1 --2- 172.21.15.167:0/1760842058 >> [v2:172.21.15.167:6816/1585024048,v1:172.21.15.167:6817/1585024048] conn(0x7fe71807c770 0x7fe71807ebb0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:20.895 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.894+0000 7fe728ff9640 1 -- 172.21.15.167:0/1760842058 <== osd.1 v2:172.21.15.167:6816/1585024048 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe724015ec0 con 0x7fe71807c770 2024-05-11T08:07:20.907 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.907+0000 7fe7322bd640 1 -- 172.21.15.167:0/1760842058 --> [v2:172.21.15.167:6816/1585024048,v1:172.21.15.167:6817/1585024048] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fe72c069590 con 0x7fe71807c770 2024-05-11T08:07:20.907 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.907+0000 7fe728ff9640 1 -- 172.21.15.167:0/1760842058 <== osd.1 v2:172.21.15.167:6816/1585024048 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fe72c069590 con 0x7fe71807c770 2024-05-11T08:07:20.907 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-05-11T08:07:20.907 INFO:teuthology.orchestra.run.smithi167.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-11T08:07:20.908 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-05-11T08:07:20.909 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.907+0000 7fe7322bd640 1 -- 172.21.15.167:0/1760842058 >> [v2:172.21.15.167:6816/1585024048,v1:172.21.15.167:6817/1585024048] conn(0x7fe71807c770 msgr2=0x7fe71807ebb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:20.909 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.907+0000 7fe7322bd640 1 --2- 172.21.15.167:0/1760842058 >> [v2:172.21.15.167:6816/1585024048,v1:172.21.15.167:6817/1585024048] conn(0x7fe71807c770 0x7fe71807ebb0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:20.909 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.907+0000 7fe7322bd640 1 -- 172.21.15.167:0/1760842058 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fe7180762d0 msgr2=0x7fe718078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:20.909 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.908+0000 7fe7322bd640 1 --2- 172.21.15.167:0/1760842058 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fe7180762d0 0x7fe718078790 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fe71c034000 tx=0x7fe71c02e000 comp rx=0 tx=0).stop 2024-05-11T08:07:20.909 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.908+0000 7fe7322bd640 1 -- 172.21.15.167:0/1760842058 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe72c163580 msgr2=0x7fe72c1527a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:20.909 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.908+0000 7fe7322bd640 1 --2- 172.21.15.167:0/1760842058 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe72c163580 0x7fe72c1527a0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fe72400c6b0 tx=0x7fe72400cb70 comp rx=0 tx=0).stop 2024-05-11T08:07:20.909 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.908+0000 7fe73085a640 1 -- 172.21.15.167:0/1760842058 reap_dead start 2024-05-11T08:07:20.909 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.908+0000 7fe7322bd640 1 -- 172.21.15.167:0/1760842058 shutdown_connections 2024-05-11T08:07:20.909 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.908+0000 7fe7322bd640 1 -- 172.21.15.167:0/1760842058 >> 172.21.15.167:0/1760842058 conn(0x7fe72c0b4ae0 msgr2=0x7fe72c0b3360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:07:20.909 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.908+0000 7fe7322bd640 1 -- 172.21.15.167:0/1760842058 shutdown_connections 2024-05-11T08:07:20.909 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:20.908+0000 7fe7322bd640 1 -- 172.21.15.167:0/1760842058 wait complete. 2024-05-11T08:07:20.918 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-05-11T08:07:21.021 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.020+0000 7fafc1d14640 1 Processor -- start 2024-05-11T08:07:21.021 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.020+0000 7fafc1d14640 1 -- start start 2024-05-11T08:07:21.024 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.021+0000 7fafc1d14640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fafbc13f290 0x7fafbc0735c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:21.024 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.021+0000 7fafc1d14640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fafbc0b50a0 0x7fafbc073b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:21.024 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.021+0000 7fafc1d14640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fafbc15fb90 con 0x7fafbc13eeb0 2024-05-11T08:07:21.024 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.021+0000 7fafc1d14640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7fafbc15d5b0 con 0x7fafbc13f290 2024-05-11T08:07:21.024 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.021+0000 7fafc1d14640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fafbc15fe70 con 0x7fafbc0b50a0 2024-05-11T08:07:21.024 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.021+0000 7fafbaffd640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fafbc13f290 0x7fafbc0735c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:21.024 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.021+0000 7fafbb7fe640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7fafbc13eeb0 0x7fafbc06f480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:59766/0 (socket says 172.21.15.167:59766) 2024-05-11T08:07:21.024 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.021+0000 7fafbb7fe640 1 -- 172.21.15.167:0/4028863953 learned_addr learned my addr 172.21.15.167:0/4028863953 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T08:07:21.024 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.021+0000 7fafbbfff640 1 --2- 172.21.15.167:0/4028863953 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fafbc0b50a0 0x7fafbc073b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:21.024 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.022+0000 7fafbaffd640 1 -- 172.21.15.167:0/4028863953 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fafbc0b50a0 msgr2=0x7fafbc073b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:07:21.024 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.022+0000 7fafbaffd640 1 --2- 172.21.15.167:0/4028863953 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fafbc0b50a0 0x7fafbc073b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:21.024 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.022+0000 7fafbaffd640 1 -- 172.21.15.167:0/4028863953 >> v1:172.21.15.167:6789/0 conn(0x7fafbc13eeb0 legacy=0x7fafbc06f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:21.024 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.022+0000 7fafbaffd640 1 -- 172.21.15.167:0/4028863953 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafbc160150 con 0x7fafbc13f290 2024-05-11T08:07:21.024 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.022+0000 7fafbaffd640 1 --2- 172.21.15.167:0/4028863953 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fafbc13f290 0x7fafbc0735c0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fafb00097a0 tx=0x7fafb002c310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b80042db5268639 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:21.025 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.022+0000 7fafba7fc640 1 -- 172.21.15.167:0/4028863953 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb0039070 con 0x7fafbc13f290 2024-05-11T08:07:21.025 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.022+0000 7fafba7fc640 1 -- 172.21.15.167:0/4028863953 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafb0034640 con 0x7fafbc13f290 2024-05-11T08:07:21.025 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.022+0000 7fafba7fc640 1 -- 172.21.15.167:0/4028863953 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb002ce60 con 0x7fafbc13f290 2024-05-11T08:07:21.025 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.023+0000 7fafc1d14640 1 -- 172.21.15.167:0/4028863953 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fafbc13f290 msgr2=0x7fafbc0735c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:21.025 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.023+0000 7fafc1d14640 1 --2- 172.21.15.167:0/4028863953 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fafbc13f290 0x7fafbc0735c0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fafb00097a0 tx=0x7fafb002c310 comp rx=0 tx=0).stop 2024-05-11T08:07:21.025 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.023+0000 7fafc1d14640 1 -- 172.21.15.167:0/4028863953 shutdown_connections 2024-05-11T08:07:21.025 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.023+0000 7fafc1d14640 1 --2- 172.21.15.167:0/4028863953 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fafbc0b50a0 0x7fafbc073b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:21.025 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.023+0000 7fafc1d14640 1 --2- 172.21.15.167:0/4028863953 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fafbc13f290 0x7fafbc0735c0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:21.025 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.023+0000 7fafc1d14640 1 -- 172.21.15.167:0/4028863953 >> 172.21.15.167:0/4028863953 conn(0x7fafbc0cd8f0 msgr2=0x7fafbc0cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:07:21.025 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.023+0000 7fafc1d14640 1 -- 172.21.15.167:0/4028863953 shutdown_connections 2024-05-11T08:07:21.025 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.023+0000 7fafc1d14640 1 -- 172.21.15.167:0/4028863953 wait complete. 2024-05-11T08:07:21.027 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.024+0000 7fafc1d14640 1 Processor -- start 2024-05-11T08:07:21.027 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.024+0000 7fafc1d14640 1 -- start start 2024-05-11T08:07:21.027 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.025+0000 7fafc1d14640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fafbc0b50a0 0x7fafbc137fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:21.028 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.025+0000 7fafc1d14640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fafbc13eeb0 0x7fafbc13ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:21.028 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.025+0000 7fafc1d14640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fafbc13f290 0x7fafbc139200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:21.028 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.025+0000 7fafc1d14640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fafbc074040 con 0x7fafbc13eeb0 2024-05-11T08:07:21.028 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.025+0000 7fafc1d14640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fafbc160710 con 0x7fafbc13f290 2024-05-11T08:07:21.028 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.025+0000 7fafc1d14640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fafbc06fb90 con 0x7fafbc0b50a0 2024-05-11T08:07:21.028 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.025+0000 7fafbaffd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fafbc13eeb0 0x7fafbc13ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:21.028 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.025+0000 7fafbaffd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fafbc13eeb0 0x7fafbc13ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:56724/0 (socket says 172.21.15.167:56724) 2024-05-11T08:07:21.028 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.025+0000 7fafbaffd640 1 -- 172.21.15.167:0/2045350134 learned_addr learned my addr 172.21.15.167:0/2045350134 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:07:21.028 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.026+0000 7fafbaffd640 1 -- 172.21.15.167:0/2045350134 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fafbc0b50a0 msgr2=0x7fafbc137fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:21.028 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.026+0000 7fafbbfff640 1 --2- 172.21.15.167:0/2045350134 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fafbc13f290 0x7fafbc139200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:21.028 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.026+0000 7fafbaffd640 1 --2- 172.21.15.167:0/2045350134 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fafbc0b50a0 0x7fafbc137fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:21.028 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.026+0000 7fafbaffd640 1 -- 172.21.15.167:0/2045350134 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fafbc13f290 msgr2=0x7fafbc139200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:21.028 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.026+0000 7fafbaffd640 1 --2- 172.21.15.167:0/2045350134 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fafbc13f290 0x7fafbc139200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:21.028 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.026+0000 7fafbaffd640 1 -- 172.21.15.167:0/2045350134 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafbc0cecf0 con 0x7fafbc13eeb0 2024-05-11T08:07:21.029 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.026+0000 7fafbaffd640 1 --2- 172.21.15.167:0/2045350134 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fafbc13eeb0 0x7fafbc13ceb0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fafb0037040 tx=0x7fafb00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:21.029 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.026+0000 7fafa3fff640 1 -- 172.21.15.167:0/2045350134 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb0043030 con 0x7fafbc13eeb0 2024-05-11T08:07:21.029 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.026+0000 7fafa3fff640 1 -- 172.21.15.167:0/2045350134 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafb0034440 con 0x7fafbc13eeb0 2024-05-11T08:07:21.029 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.026+0000 7fafc1d14640 1 -- 172.21.15.167:0/2045350134 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafbc0cefd0 con 0x7fafbc13eeb0 2024-05-11T08:07:21.029 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.026+0000 7fafa3fff640 1 -- 172.21.15.167:0/2045350134 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb0039040 con 0x7fafbc13eeb0 2024-05-11T08:07:21.029 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.026+0000 7fafc1d14640 1 -- 172.21.15.167:0/2045350134 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafbc0cf870 con 0x7fafbc13eeb0 2024-05-11T08:07:21.029 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.027+0000 7fafc1d14640 1 -- 172.21.15.167:0/2045350134 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafbc0cf3c0 con 0x7fafbc13eeb0 2024-05-11T08:07:21.029 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.028+0000 7fafa3fff640 1 -- 172.21.15.167:0/2045350134 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fafb0046070 con 0x7fafbc13eeb0 2024-05-11T08:07:21.030 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.028+0000 7fafa3fff640 1 --2- 172.21.15.167:0/2045350134 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fafa8076300 0x7fafa80787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:21.030 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.028+0000 7fafa3fff640 1 -- 172.21.15.167:0/2045350134 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5335+0+0 (secure 0 0 0) 0x7fafb00b99f0 con 0x7fafbc13eeb0 2024-05-11T08:07:21.030 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.028+0000 7fafbb7fe640 1 --2- 172.21.15.167:0/2045350134 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fafa8076300 0x7fafa80787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:21.030 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.029+0000 7fafbb7fe640 1 --2- 172.21.15.167:0/2045350134 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fafa8076300 0x7fafa80787c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fafac002060 tx=0x7fafac001650 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:21.032 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.032+0000 7fafa3fff640 1 -- 172.21.15.167:0/2045350134 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fafb00834c0 con 0x7fafbc13eeb0 2024-05-11T08:07:21.214 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.214+0000 7fafc1d14640 1 -- 172.21.15.167:0/2045350134 --> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7fafbc069590 con 0x7fafa8076300 2024-05-11T08:07:21.214 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.214+0000 7fafa3fff640 1 -- 172.21.15.167:0/2045350134 <== mgr.4113 v2:172.21.15.167:6832/253569496 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fafbc069590 con 0x7fafa8076300 2024-05-11T08:07:21.215 INFO:teuthology.orchestra.run.smithi167.stderr:instructed osd(s) 1 to deep-scrub 2024-05-11T08:07:21.217 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.216+0000 7fafc1d14640 1 -- 172.21.15.167:0/2045350134 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fafa8076300 msgr2=0x7fafa80787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:21.217 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.216+0000 7fafc1d14640 1 --2- 172.21.15.167:0/2045350134 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fafa8076300 0x7fafa80787c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fafac002060 tx=0x7fafac001650 comp rx=0 tx=0).stop 2024-05-11T08:07:21.217 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.216+0000 7fafc1d14640 1 -- 172.21.15.167:0/2045350134 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fafbc13eeb0 msgr2=0x7fafbc13ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:21.217 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.216+0000 7fafc1d14640 1 --2- 172.21.15.167:0/2045350134 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fafbc13eeb0 0x7fafbc13ceb0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fafb0037040 tx=0x7fafb00046c0 comp rx=0 tx=0).stop 2024-05-11T08:07:21.217 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.216+0000 7fafc1d14640 1 -- 172.21.15.167:0/2045350134 shutdown_connections 2024-05-11T08:07:21.217 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.216+0000 7fafc1d14640 1 --2- 172.21.15.167:0/2045350134 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fafa8076300 0x7fafa80787c0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:21.217 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.216+0000 7fafc1d14640 1 --2- 172.21.15.167:0/2045350134 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fafbc13f290 0x7fafbc139200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:21.217 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.216+0000 7fafc1d14640 1 --2- 172.21.15.167:0/2045350134 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fafbc13eeb0 0x7fafbc13ceb0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:21.217 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.216+0000 7fafc1d14640 1 --2- 172.21.15.167:0/2045350134 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fafbc0b50a0 0x7fafbc137fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:21.217 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.216+0000 7fafc1d14640 1 -- 172.21.15.167:0/2045350134 >> 172.21.15.167:0/2045350134 conn(0x7fafbc0cd8f0 msgr2=0x7fafbc155a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:07:21.217 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.217+0000 7fafc1d14640 1 -- 172.21.15.167:0/2045350134 shutdown_connections 2024-05-11T08:07:21.217 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.217+0000 7fafc1d14640 1 -- 172.21.15.167:0/2045350134 wait complete. 2024-05-11T08:07:21.228 INFO:tasks.ceph:Scrubbing osd.2 2024-05-11T08:07:21.228 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-05-11T08:07:21.332 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.332+0000 7fc5407b6640 1 Processor -- start 2024-05-11T08:07:21.332 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.332+0000 7fc5407b6640 1 -- start start 2024-05-11T08:07:21.333 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.333+0000 7fc5407b6640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fc5380d44b0 0x7fc53806cb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:21.334 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.333+0000 7fc5407b6640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc5380b4460 0x7fc53806d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:21.334 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.333+0000 7fc5407b6640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc538163420 con 0x7fc5381628c0 2024-05-11T08:07:21.334 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.333+0000 7fc5407b6640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7fc538164030 con 0x7fc5380d44b0 2024-05-11T08:07:21.334 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.333+0000 7fc5407b6640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fc538163700 con 0x7fc5380b4460 2024-05-11T08:07:21.334 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.333+0000 7fc53dd51640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fc5380d44b0 0x7fc53806cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:21.334 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.333+0000 7fc53dd51640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fc5380d44b0 0x7fc53806cb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:56740/0 (socket says 172.21.15.167:56740) 2024-05-11T08:07:21.334 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.333+0000 7fc53dd51640 1 -- 172.21.15.167:0/3491141121 learned_addr learned my addr 172.21.15.167:0/3491141121 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:07:21.334 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.334+0000 7fc53ed53640 1 --2- 172.21.15.167:0/3491141121 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc5380b4460 0x7fc53806d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:21.334 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.334+0000 7fc53dd51640 1 -- 172.21.15.167:0/3491141121 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc5380b4460 msgr2=0x7fc53806d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:07:21.334 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.334+0000 7fc53dd51640 1 --2- 172.21.15.167:0/3491141121 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc5380b4460 0x7fc53806d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:21.334 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.334+0000 7fc53dd51640 1 -- 172.21.15.167:0/3491141121 >> v1:172.21.15.204:6789/0 conn(0x7fc5381628c0 legacy=0x7fc5380d3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:07:21.334 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.334+0000 7fc53dd51640 1 -- 172.21.15.167:0/3491141121 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5381639e0 con 0x7fc5380d44b0 2024-05-11T08:07:21.334 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.334+0000 7fc53ed53640 1 --2- 172.21.15.167:0/3491141121 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc5380b4460 0x7fc53806d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T08:07:21.336 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.334+0000 7fc53dd51640 1 --2- 172.21.15.167:0/3491141121 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc5380d44b0 0x7fc53806cb60 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fc528009870 tx=0x7fc52802a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a8e0e429e5d27bb server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:21.336 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.334+0000 7fc53d550640 1 -- 172.21.15.167:0/3491141121 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc528037070 con 0x7fc5380d44b0 2024-05-11T08:07:21.336 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.335+0000 7fc53d550640 1 -- 172.21.15.167:0/3491141121 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc528032440 con 0x7fc5380d44b0 2024-05-11T08:07:21.336 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.335+0000 7fc53d550640 1 -- 172.21.15.167:0/3491141121 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc528032810 con 0x7fc5380d44b0 2024-05-11T08:07:21.336 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.335+0000 7fc5407b6640 1 -- 172.21.15.167:0/3491141121 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc5380d44b0 msgr2=0x7fc53806cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:21.336 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.335+0000 7fc5407b6640 1 --2- 172.21.15.167:0/3491141121 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc5380d44b0 0x7fc53806cb60 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fc528009870 tx=0x7fc52802a250 comp rx=0 tx=0).stop 2024-05-11T08:07:21.336 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.335+0000 7fc5407b6640 1 -- 172.21.15.167:0/3491141121 shutdown_connections 2024-05-11T08:07:21.336 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.335+0000 7fc5407b6640 1 --2- 172.21.15.167:0/3491141121 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc5380b4460 0x7fc53806d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:21.336 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.335+0000 7fc5407b6640 1 --2- 172.21.15.167:0/3491141121 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc5380d44b0 0x7fc53806cb60 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:21.336 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.335+0000 7fc5407b6640 1 -- 172.21.15.167:0/3491141121 >> 172.21.15.167:0/3491141121 conn(0x7fc5380b6cb0 msgr2=0x7fc5380b7090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:07:21.336 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.335+0000 7fc5407b6640 1 -- 172.21.15.167:0/3491141121 shutdown_connections 2024-05-11T08:07:21.337 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.335+0000 7fc5407b6640 1 -- 172.21.15.167:0/3491141121 wait complete. 2024-05-11T08:07:21.339 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.336+0000 7fc5407b6640 1 Processor -- start 2024-05-11T08:07:21.339 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.336+0000 7fc5407b6640 1 -- start start 2024-05-11T08:07:21.339 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.337+0000 7fc5407b6640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc5380b4460 0x7fc53815f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:21.339 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.337+0000 7fc5407b6640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc5380d44b0 0x7fc5381e9890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:21.339 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.337+0000 7fc5407b6640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc5381628c0 0x7fc5381f4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:21.339 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.337+0000 7fc5407b6640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc538162ca0 con 0x7fc5380b4460 2024-05-11T08:07:21.339 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.337+0000 7fc5407b6640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc5381630d0 con 0x7fc5381628c0 2024-05-11T08:07:21.339 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.337+0000 7fc5407b6640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fc538162e20 con 0x7fc5380d44b0 2024-05-11T08:07:21.339 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.337+0000 7fc53e552640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc5380b4460 0x7fc53815f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:21.339 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.337+0000 7fc53e552640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc5380b4460 0x7fc53815f480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:56752/0 (socket says 172.21.15.167:56752) 2024-05-11T08:07:21.339 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.337+0000 7fc53e552640 1 -- 172.21.15.167:0/536604492 learned_addr learned my addr 172.21.15.167:0/536604492 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:07:21.339 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.337+0000 7fc53ed53640 1 --2- 172.21.15.167:0/536604492 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc5381628c0 0x7fc5381f4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:21.339 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.337+0000 7fc53dd51640 1 --2- 172.21.15.167:0/536604492 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc5380d44b0 0x7fc5381e9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:21.340 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.337+0000 7fc53e552640 1 -- 172.21.15.167:0/536604492 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc5380d44b0 msgr2=0x7fc5381e9890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:21.340 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.337+0000 7fc53e552640 1 --2- 172.21.15.167:0/536604492 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc5380d44b0 0x7fc5381e9890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:21.340 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.337+0000 7fc53e552640 1 -- 172.21.15.167:0/536604492 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc5381628c0 msgr2=0x7fc5381f4cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:21.340 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.337+0000 7fc53e552640 1 --2- 172.21.15.167:0/536604492 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc5381628c0 0x7fc5381f4cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:21.340 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.337+0000 7fc53e552640 1 -- 172.21.15.167:0/536604492 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5381ea920 con 0x7fc5380b4460 2024-05-11T08:07:21.340 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.338+0000 7fc53e552640 1 --2- 172.21.15.167:0/536604492 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc5380b4460 0x7fc53815f480 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fc53400c3a0 tx=0x7fc53400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:21.340 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.338+0000 7fc52effd640 1 -- 172.21.15.167:0/536604492 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc534017070 con 0x7fc5380b4460 2024-05-11T08:07:21.340 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.338+0000 7fc52effd640 1 -- 172.21.15.167:0/536604492 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5340052d0 con 0x7fc5380b4460 2024-05-11T08:07:21.340 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.338+0000 7fc5407b6640 1 -- 172.21.15.167:0/536604492 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5381e9dd0 con 0x7fc5380b4460 2024-05-11T08:07:21.340 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.338+0000 7fc52effd640 1 -- 172.21.15.167:0/536604492 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc534003830 con 0x7fc5380b4460 2024-05-11T08:07:21.340 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.338+0000 7fc5407b6640 1 -- 172.21.15.167:0/536604492 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5381ea360 con 0x7fc5380b4460 2024-05-11T08:07:21.340 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.339+0000 7fc5407b6640 1 -- 172.21.15.167:0/536604492 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc538069590 con 0x7fc5380b4460 2024-05-11T08:07:21.341 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.340+0000 7fc52effd640 1 -- 172.21.15.167:0/536604492 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fc53401a070 con 0x7fc5380b4460 2024-05-11T08:07:21.342 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.340+0000 7fc52effd640 1 --2- 172.21.15.167:0/536604492 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fc51c0762d0 0x7fc51c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:21.342 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.340+0000 7fc52effd640 1 -- 172.21.15.167:0/536604492 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5335+0+0 (secure 0 0 0) 0x7fc534097020 con 0x7fc5380b4460 2024-05-11T08:07:21.342 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.340+0000 7fc53dd51640 1 --2- 172.21.15.167:0/536604492 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fc51c0762d0 0x7fc51c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:21.342 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.340+0000 7fc52effd640 1 --2- 172.21.15.167:0/536604492 >> [v2:172.21.15.167:6800/3242166900,v1:172.21.15.167:6801/3242166900] conn(0x7fc51c07c770 0x7fc51c07ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:21.342 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.340+0000 7fc52effd640 1 -- 172.21.15.167:0/536604492 --> [v2:172.21.15.167:6800/3242166900,v1:172.21.15.167:6801/3242166900] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc53401dec0 con 0x7fc51c07c770 2024-05-11T08:07:21.342 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.340+0000 7fc52effd640 1 -- 172.21.15.167:0/536604492 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fc5340974b0 con 0x7fc5380b4460 2024-05-11T08:07:21.342 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.340+0000 7fc53ed53640 1 --2- 172.21.15.167:0/536604492 >> [v2:172.21.15.167:6800/3242166900,v1:172.21.15.167:6801/3242166900] conn(0x7fc51c07c770 0x7fc51c07ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:21.342 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.340+0000 7fc53dd51640 1 --2- 172.21.15.167:0/536604492 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fc51c0762d0 0x7fc51c078790 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fc528034000 tx=0x7fc52802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:21.342 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.341+0000 7fc53ed53640 1 --2- 172.21.15.167:0/536604492 >> [v2:172.21.15.167:6800/3242166900,v1:172.21.15.167:6801/3242166900] conn(0x7fc51c07c770 0x7fc51c07ebb0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:21.342 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.341+0000 7fc52effd640 1 -- 172.21.15.167:0/536604492 <== osd.2 v2:172.21.15.167:6800/3242166900 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc53401dec0 con 0x7fc51c07c770 2024-05-11T08:07:21.354 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.354+0000 7fc5407b6640 1 -- 172.21.15.167:0/536604492 --> [v2:172.21.15.167:6800/3242166900,v1:172.21.15.167:6801/3242166900] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fc5380d4a80 con 0x7fc51c07c770 2024-05-11T08:07:21.354 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.354+0000 7fc52effd640 1 -- 172.21.15.167:0/536604492 <== osd.2 v2:172.21.15.167:6800/3242166900 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fc5380d4a80 con 0x7fc51c07c770 2024-05-11T08:07:21.355 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-05-11T08:07:21.355 INFO:teuthology.orchestra.run.smithi167.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-11T08:07:21.355 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-05-11T08:07:21.356 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.355+0000 7fc5407b6640 1 -- 172.21.15.167:0/536604492 >> [v2:172.21.15.167:6800/3242166900,v1:172.21.15.167:6801/3242166900] conn(0x7fc51c07c770 msgr2=0x7fc51c07ebb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:21.356 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.355+0000 7fc5407b6640 1 --2- 172.21.15.167:0/536604492 >> [v2:172.21.15.167:6800/3242166900,v1:172.21.15.167:6801/3242166900] conn(0x7fc51c07c770 0x7fc51c07ebb0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:21.356 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.355+0000 7fc5407b6640 1 -- 172.21.15.167:0/536604492 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fc51c0762d0 msgr2=0x7fc51c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:21.356 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.355+0000 7fc5407b6640 1 --2- 172.21.15.167:0/536604492 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fc51c0762d0 0x7fc51c078790 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fc528034000 tx=0x7fc52802e000 comp rx=0 tx=0).stop 2024-05-11T08:07:21.356 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.355+0000 7fc5407b6640 1 -- 172.21.15.167:0/536604492 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc5380b4460 msgr2=0x7fc53815f480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:21.356 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.355+0000 7fc5407b6640 1 --2- 172.21.15.167:0/536604492 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc5380b4460 0x7fc53815f480 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fc53400c3a0 tx=0x7fc53400c860 comp rx=0 tx=0).stop 2024-05-11T08:07:21.356 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.355+0000 7fc53ed53640 1 -- 172.21.15.167:0/536604492 reap_dead start 2024-05-11T08:07:21.356 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.355+0000 7fc5407b6640 1 -- 172.21.15.167:0/536604492 shutdown_connections 2024-05-11T08:07:21.356 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.355+0000 7fc5407b6640 1 -- 172.21.15.167:0/536604492 >> 172.21.15.167:0/536604492 conn(0x7fc5380b6cb0 msgr2=0x7fc5380d1440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:07:21.356 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.355+0000 7fc5407b6640 1 -- 172.21.15.167:0/536604492 shutdown_connections 2024-05-11T08:07:21.357 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.355+0000 7fc5407b6640 1 -- 172.21.15.167:0/536604492 wait complete. 2024-05-11T08:07:21.365 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-05-11T08:07:21.467 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.466+0000 7fb65a1f9640 1 Processor -- start 2024-05-11T08:07:21.467 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.466+0000 7fb65a1f9640 1 -- start start 2024-05-11T08:07:21.467 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.467+0000 7fb65a1f9640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fb65415f330 0x7fb6540b6860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:21.467 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.467+0000 7fb65a1f9640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb65406c930 con 0x7fb65406c450 2024-05-11T08:07:21.468 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.467+0000 7fb65a1f9640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb65406cdd0 con 0x7fb6540737a0 2024-05-11T08:07:21.468 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.467+0000 7fb65a1f9640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7fb65406d0b0 con 0x7fb65415f330 2024-05-11T08:07:21.468 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.467+0000 7fb652ffd640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7fb65406c450 0x7fb654073090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:59772/0 (socket says 172.21.15.167:59772) 2024-05-11T08:07:21.468 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.467+0000 7fb652ffd640 1 -- 172.21.15.167:0/2380969344 learned_addr learned my addr 172.21.15.167:0/2380969344 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T08:07:21.468 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.467+0000 7fb6537fe640 1 --2- 172.21.15.167:0/2380969344 >> v2:172.21.15.204:3300/0 conn(0x7fb65415f330 0x7fb6540b6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:21.468 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.467+0000 7fb6527fc640 1 -- 172.21.15.167:0/2380969344 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1602643802 0 0) 0x7fb65406c930 con 0x7fb65406c450 2024-05-11T08:07:21.468 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.467+0000 7fb6527fc640 1 -- 172.21.15.167:0/2380969344 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb63c003520 con 0x7fb65406c450 2024-05-11T08:07:21.471 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.468+0000 7fb6527fc640 1 -- 172.21.15.167:0/2380969344 <== mon.0 v1:172.21.15.167:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3729783611 0 0) 0x7fb63c003520 con 0x7fb65406c450 2024-05-11T08:07:21.471 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.468+0000 7fb6527fc640 1 -- 172.21.15.167:0/2380969344 >> v2:172.21.15.204:3300/0 conn(0x7fb65415f330 msgr2=0x7fb6540b6860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:07:21.471 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.468+0000 7fb6527fc640 1 --2- 172.21.15.167:0/2380969344 >> v2:172.21.15.204:3300/0 conn(0x7fb65415f330 0x7fb6540b6860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:21.471 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.468+0000 7fb6527fc640 1 -- 172.21.15.167:0/2380969344 >> v1:172.21.15.204:6789/0 conn(0x7fb6540737a0 legacy=0x7fb6540d0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:21.471 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.468+0000 7fb6527fc640 1 -- 172.21.15.167:0/2380969344 --> v1:172.21.15.167:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb65406d390 con 0x7fb65406c450 2024-05-11T08:07:21.471 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.468+0000 7fb6527fc640 1 -- 172.21.15.167:0/2380969344 <== mon.0 v1:172.21.15.167:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3602277846 0 0) 0x7fb648001a70 con 0x7fb65406c450 2024-05-11T08:07:21.471 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.468+0000 7fb6527fc640 1 -- 172.21.15.167:0/2380969344 >> v1:172.21.15.167:6789/0 conn(0x7fb65406c450 legacy=0x7fb654073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:21.471 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.468+0000 7fb6527fc640 1 --2- 172.21.15.167:0/2380969344 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb63c003a50 0x7fb63c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:21.471 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.468+0000 7fb6527fc640 1 --2- 172.21.15.167:0/2380969344 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb63c004800 0x7fb63c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:21.472 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.468+0000 7fb6527fc640 1 --2- 172.21.15.167:0/2380969344 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb63c0051c0 0x7fb63c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:21.472 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.468+0000 7fb6527fc640 1 -- 172.21.15.167:0/2380969344 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb65406d0b0 con 0x7fb63c003a50 2024-05-11T08:07:21.472 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.468+0000 7fb6527fc640 1 -- 172.21.15.167:0/2380969344 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb63c00c160 con 0x7fb63c0051c0 2024-05-11T08:07:21.472 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.468+0000 7fb6527fc640 1 -- 172.21.15.167:0/2380969344 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fb63c00c360 con 0x7fb63c004800 2024-05-11T08:07:21.472 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.468+0000 7fb6537fe640 1 --2- 172.21.15.167:0/2380969344 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb63c003a50 0x7fb63c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:21.472 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.468+0000 7fb653fff640 1 --2- 172.21.15.167:0/2380969344 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb63c0051c0 0x7fb63c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:21.472 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.468+0000 7fb652ffd640 1 --2- 172.21.15.167:0/2380969344 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb63c004800 0x7fb63c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:21.472 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.468+0000 7fb6537fe640 1 -- 172.21.15.167:0/2380969344 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb63c004800 msgr2=0x7fb63c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:07:21.472 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.468+0000 7fb6537fe640 1 --2- 172.21.15.167:0/2380969344 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb63c004800 0x7fb63c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:21.472 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.468+0000 7fb6537fe640 1 -- 172.21.15.167:0/2380969344 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb63c0051c0 msgr2=0x7fb63c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:07:21.472 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.468+0000 7fb6537fe640 1 --2- 172.21.15.167:0/2380969344 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb63c0051c0 0x7fb63c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:21.472 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.468+0000 7fb653fff640 1 -- 172.21.15.167:0/2380969344 reap_dead start 2024-05-11T08:07:21.472 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.468+0000 7fb6537fe640 1 -- 172.21.15.167:0/2380969344 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb63c003520 con 0x7fb63c003a50 2024-05-11T08:07:21.472 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.469+0000 7fb6537fe640 1 --2- 172.21.15.167:0/2380969344 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb63c003a50 0x7fb63c003e70 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fb644009ac0 tx=0x7fb63c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4b83458a00bc55b server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:21.473 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.469+0000 7fb6527fc640 1 -- 172.21.15.167:0/2380969344 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb644036070 con 0x7fb63c003a50 2024-05-11T08:07:21.473 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.469+0000 7fb6527fc640 1 -- 172.21.15.167:0/2380969344 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb644031400 con 0x7fb63c003a50 2024-05-11T08:07:21.473 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.469+0000 7fb6527fc640 1 -- 172.21.15.167:0/2380969344 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb644009690 con 0x7fb63c003a50 2024-05-11T08:07:21.473 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.469+0000 7fb65a1f9640 1 -- 172.21.15.167:0/2380969344 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb63c003a50 msgr2=0x7fb63c003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:21.473 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.469+0000 7fb65a1f9640 1 --2- 172.21.15.167:0/2380969344 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb63c003a50 0x7fb63c003e70 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fb644009ac0 tx=0x7fb63c005830 comp rx=0 tx=0).stop 2024-05-11T08:07:21.473 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.469+0000 7fb65a1f9640 1 -- 172.21.15.167:0/2380969344 shutdown_connections 2024-05-11T08:07:21.473 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.469+0000 7fb65a1f9640 1 --2- 172.21.15.167:0/2380969344 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb63c003a50 0x7fb63c003e70 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:21.473 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.469+0000 7fb65a1f9640 1 -- 172.21.15.167:0/2380969344 >> 172.21.15.167:0/2380969344 conn(0x7fb654070290 msgr2=0x7fb654070670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:07:21.473 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.470+0000 7fb65a1f9640 1 -- 172.21.15.167:0/2380969344 shutdown_connections 2024-05-11T08:07:21.473 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.470+0000 7fb65a1f9640 1 -- 172.21.15.167:0/2380969344 wait complete. 2024-05-11T08:07:21.473 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.471+0000 7fb65a1f9640 1 Processor -- start 2024-05-11T08:07:21.473 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.471+0000 7fb65a1f9640 1 -- start start 2024-05-11T08:07:21.476 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.472+0000 7fb65a1f9640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb63c003a50 0x7fb6541568f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:21.476 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.472+0000 7fb65a1f9640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb654156120 0x7fb65414fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:21.476 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.472+0000 7fb65a1f9640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb65415fc70 0x7fb6541600f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:21.476 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.472+0000 7fb65a1f9640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb654073f00 con 0x7fb65415fc70 2024-05-11T08:07:21.476 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.472+0000 7fb65a1f9640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb654073d80 con 0x7fb63c003a50 2024-05-11T08:07:21.476 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.472+0000 7fb65a1f9640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fb654074740 con 0x7fb654156120 2024-05-11T08:07:21.477 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.472+0000 7fb653fff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb65415fc70 0x7fb6541600f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:21.477 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.472+0000 7fb653fff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb65415fc70 0x7fb6541600f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:56774/0 (socket says 172.21.15.167:56774) 2024-05-11T08:07:21.477 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.472+0000 7fb653fff640 1 -- 172.21.15.167:0/1570116334 learned_addr learned my addr 172.21.15.167:0/1570116334 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:07:21.477 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.472+0000 7fb6537fe640 1 --2- 172.21.15.167:0/1570116334 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb63c003a50 0x7fb6541568f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:21.477 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.472+0000 7fb652ffd640 1 --2- 172.21.15.167:0/1570116334 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb654156120 0x7fb65414fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:21.477 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.472+0000 7fb653fff640 1 -- 172.21.15.167:0/1570116334 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb654156120 msgr2=0x7fb65414fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:21.477 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.472+0000 7fb653fff640 1 --2- 172.21.15.167:0/1570116334 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb654156120 0x7fb65414fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:21.477 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.472+0000 7fb653fff640 1 -- 172.21.15.167:0/1570116334 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb63c003a50 msgr2=0x7fb6541568f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:21.477 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.472+0000 7fb653fff640 1 --2- 172.21.15.167:0/1570116334 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb63c003a50 0x7fb6541568f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:21.477 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.472+0000 7fb653fff640 1 -- 172.21.15.167:0/1570116334 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb654156f10 con 0x7fb65415fc70 2024-05-11T08:07:21.477 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.472+0000 7fb6537fe640 1 --2- 172.21.15.167:0/1570116334 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb63c003a50 0x7fb6541568f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T08:07:21.477 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.472+0000 7fb653fff640 1 --2- 172.21.15.167:0/1570116334 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb65415fc70 0x7fb6541600f0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fb65406e850 tx=0x7fb64c00e8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:21.477 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.472+0000 7fb63bfff640 1 -- 172.21.15.167:0/1570116334 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb64c019070 con 0x7fb65415fc70 2024-05-11T08:07:21.478 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.472+0000 7fb63bfff640 1 -- 172.21.15.167:0/1570116334 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb64c0052e0 con 0x7fb65415fc70 2024-05-11T08:07:21.478 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.472+0000 7fb65a1f9640 1 -- 172.21.15.167:0/1570116334 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6541571f0 con 0x7fb65415fc70 2024-05-11T08:07:21.478 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.472+0000 7fb63bfff640 1 -- 172.21.15.167:0/1570116334 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb64c003810 con 0x7fb65415fc70 2024-05-11T08:07:21.478 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.472+0000 7fb65a1f9640 1 -- 172.21.15.167:0/1570116334 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb65415aa10 con 0x7fb65415fc70 2024-05-11T08:07:21.478 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.473+0000 7fb65a1f9640 1 -- 172.21.15.167:0/1570116334 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb654069590 con 0x7fb65415fc70 2024-05-11T08:07:21.478 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.474+0000 7fb63bfff640 1 -- 172.21.15.167:0/1570116334 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fb64c0039b0 con 0x7fb65415fc70 2024-05-11T08:07:21.478 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.474+0000 7fb63bfff640 1 --2- 172.21.15.167:0/1570116334 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fb640076300 0x7fb6400787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:21.478 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.475+0000 7fb6537fe640 1 --2- 172.21.15.167:0/1570116334 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fb640076300 0x7fb6400787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:21.478 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.475+0000 7fb63bfff640 1 -- 172.21.15.167:0/1570116334 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5335+0+0 (secure 0 0 0) 0x7fb64c096720 con 0x7fb65415fc70 2024-05-11T08:07:21.478 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.475+0000 7fb6537fe640 1 --2- 172.21.15.167:0/1570116334 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fb640076300 0x7fb6400787c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fb644033000 tx=0x7fb644008030 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:21.479 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.479+0000 7fb63bfff640 1 -- 172.21.15.167:0/1570116334 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fb64c05fc30 con 0x7fb65415fc70 2024-05-11T08:07:21.661 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.660+0000 7fb65a1f9640 1 -- 172.21.15.167:0/1570116334 --> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7fb65415c6c0 con 0x7fb640076300 2024-05-11T08:07:21.661 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.661+0000 7fb63bfff640 1 -- 172.21.15.167:0/1570116334 <== mgr.4113 v2:172.21.15.167:6832/253569496 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fb65415c6c0 con 0x7fb640076300 2024-05-11T08:07:21.661 INFO:teuthology.orchestra.run.smithi167.stderr:instructed osd(s) 2 to deep-scrub 2024-05-11T08:07:21.663 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.663+0000 7fb65a1f9640 1 -- 172.21.15.167:0/1570116334 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fb640076300 msgr2=0x7fb6400787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:21.663 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.663+0000 7fb65a1f9640 1 --2- 172.21.15.167:0/1570116334 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fb640076300 0x7fb6400787c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fb644033000 tx=0x7fb644008030 comp rx=0 tx=0).stop 2024-05-11T08:07:21.663 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.663+0000 7fb65a1f9640 1 -- 172.21.15.167:0/1570116334 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb65415fc70 msgr2=0x7fb6541600f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:21.663 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.663+0000 7fb65a1f9640 1 --2- 172.21.15.167:0/1570116334 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb65415fc70 0x7fb6541600f0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fb65406e850 tx=0x7fb64c00e8a0 comp rx=0 tx=0).stop 2024-05-11T08:07:21.664 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.663+0000 7fb65a1f9640 1 -- 172.21.15.167:0/1570116334 shutdown_connections 2024-05-11T08:07:21.664 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.663+0000 7fb65a1f9640 1 --2- 172.21.15.167:0/1570116334 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fb640076300 0x7fb6400787c0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:21.664 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.663+0000 7fb65a1f9640 1 --2- 172.21.15.167:0/1570116334 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb65415fc70 0x7fb6541600f0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:21.664 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.663+0000 7fb65a1f9640 1 --2- 172.21.15.167:0/1570116334 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb654156120 0x7fb65414fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:21.664 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.663+0000 7fb65a1f9640 1 --2- 172.21.15.167:0/1570116334 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb63c003a50 0x7fb6541568f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:21.664 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.663+0000 7fb65a1f9640 1 -- 172.21.15.167:0/1570116334 >> 172.21.15.167:0/1570116334 conn(0x7fb654070290 msgr2=0x7fb654073090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:07:21.664 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.663+0000 7fb65a1f9640 1 -- 172.21.15.167:0/1570116334 shutdown_connections 2024-05-11T08:07:21.664 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.663+0000 7fb65a1f9640 1 -- 172.21.15.167:0/1570116334 wait complete. 2024-05-11T08:07:21.674 INFO:tasks.ceph:Scrubbing osd.3 2024-05-11T08:07:21.674 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-05-11T08:07:21.779 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.778+0000 7f96db5f7640 1 Processor -- start 2024-05-11T08:07:21.779 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.779+0000 7f96db5f7640 1 -- start start 2024-05-11T08:07:21.781 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.780+0000 7f96db5f7640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f96d40cd430 0x7f96d406e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:21.781 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.780+0000 7f96db5f7640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f96d40aed70 0x7f96d40af1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:21.781 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.780+0000 7f96db5f7640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96d415d9e0 con 0x7f96d40adea0 2024-05-11T08:07:21.781 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.780+0000 7f96db5f7640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f96d415dcc0 con 0x7f96d40aed70 2024-05-11T08:07:21.781 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.780+0000 7f96db5f7640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f96d415dfa0 con 0x7f96d40cd430 2024-05-11T08:07:21.781 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.780+0000 7f96d9b94640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f96d40adea0 0x7f96d4160f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:59776/0 (socket says 172.21.15.167:59776) 2024-05-11T08:07:21.781 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.780+0000 7f96d9b94640 1 -- 172.21.15.167:0/2186680429 learned_addr learned my addr 172.21.15.167:0/2186680429 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T08:07:21.781 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.780+0000 7f96d8b92640 1 --2- 172.21.15.167:0/2186680429 >> v2:172.21.15.204:3300/0 conn(0x7f96d40aed70 0x7f96d40af1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:21.781 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.780+0000 7f96d9393640 1 --2- 172.21.15.167:0/2186680429 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f96d40cd430 0x7f96d406e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:21.781 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.780+0000 7f96c7fff640 1 -- 172.21.15.167:0/2186680429 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 873074195 0 0) 0x7f96d415d9e0 con 0x7f96d40adea0 2024-05-11T08:07:21.781 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.780+0000 7f96c7fff640 1 -- 172.21.15.167:0/2186680429 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f96c0003520 con 0x7f96d40adea0 2024-05-11T08:07:21.782 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.780+0000 7f96d9393640 1 -- 172.21.15.167:0/2186680429 >> v2:172.21.15.204:3300/0 conn(0x7f96d40aed70 msgr2=0x7f96d40af1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:07:21.782 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.780+0000 7f96d9393640 1 --2- 172.21.15.167:0/2186680429 >> v2:172.21.15.204:3300/0 conn(0x7f96d40aed70 0x7f96d40af1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:21.782 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.780+0000 7f96d9393640 1 -- 172.21.15.167:0/2186680429 >> v1:172.21.15.167:6789/0 conn(0x7f96d40adea0 legacy=0x7f96d4160f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:21.782 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.780+0000 7f96c7fff640 1 -- 172.21.15.167:0/2186680429 <== mon.0 v1:172.21.15.167:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1934677787 0 0) 0x7f96c0003520 con 0x7f96d40adea0 2024-05-11T08:07:21.782 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.780+0000 7f96d9393640 1 -- 172.21.15.167:0/2186680429 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96d415e280 con 0x7f96d40cd430 2024-05-11T08:07:21.782 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.780+0000 7f96d8b92640 1 --2- 172.21.15.167:0/2186680429 >> v2:172.21.15.204:3300/0 conn(0x7f96d40aed70 0x7f96d40af1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T08:07:21.782 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.781+0000 7f96d9393640 1 --2- 172.21.15.167:0/2186680429 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f96d40cd430 0x7f96d406e4b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f96c80097a0 tx=0x7f96c802a080 comp rx=0 tx=0).ready entity=mon.2 client_cookie=24142ecd41c91690 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:21.785 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.781+0000 7f96c7fff640 1 -- 172.21.15.167:0/2186680429 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96c8037070 con 0x7f96d40cd430 2024-05-11T08:07:21.785 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.781+0000 7f96c7fff640 1 -- 172.21.15.167:0/2186680429 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96c802ab10 con 0x7f96d40cd430 2024-05-11T08:07:21.785 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.781+0000 7f96c7fff640 1 -- 172.21.15.167:0/2186680429 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96cc001a70 con 0x7f96d40cd430 2024-05-11T08:07:21.785 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.782+0000 7f96db5f7640 1 -- 172.21.15.167:0/2186680429 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f96d40cd430 msgr2=0x7f96d406e4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:21.785 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.782+0000 7f96db5f7640 1 --2- 172.21.15.167:0/2186680429 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f96d40cd430 0x7f96d406e4b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f96c80097a0 tx=0x7f96c802a080 comp rx=0 tx=0).stop 2024-05-11T08:07:21.785 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.782+0000 7f96db5f7640 1 -- 172.21.15.167:0/2186680429 shutdown_connections 2024-05-11T08:07:21.785 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.782+0000 7f96db5f7640 1 --2- 172.21.15.167:0/2186680429 >> v2:172.21.15.204:3300/0 conn(0x7f96d40aed70 0x7f96d40af1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:21.785 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.782+0000 7f96db5f7640 1 --2- 172.21.15.167:0/2186680429 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f96d40cd430 0x7f96d406e4b0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:21.786 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.782+0000 7f96db5f7640 1 -- 172.21.15.167:0/2186680429 >> 172.21.15.167:0/2186680429 conn(0x7f96d40d10d0 msgr2=0x7f96d40d14e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:07:21.786 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.782+0000 7f96db5f7640 1 -- 172.21.15.167:0/2186680429 shutdown_connections 2024-05-11T08:07:21.786 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.782+0000 7f96db5f7640 1 -- 172.21.15.167:0/2186680429 wait complete. 2024-05-11T08:07:21.786 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.783+0000 7f96db5f7640 1 Processor -- start 2024-05-11T08:07:21.786 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.783+0000 7f96db5f7640 1 -- start start 2024-05-11T08:07:21.786 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.784+0000 7f96db5f7640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f96d40adea0 0x7f96d409e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:21.786 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.784+0000 7f96db5f7640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f96d40aed70 0x7f96d41f3620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:21.786 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.784+0000 7f96db5f7640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f96d40cd430 0x7f96d409c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:21.786 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.784+0000 7f96db5f7640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f96d415e560 con 0x7f96d40aed70 2024-05-11T08:07:21.786 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.784+0000 7f96db5f7640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f96d415d690 con 0x7f96d40cd430 2024-05-11T08:07:21.786 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.784+0000 7f96db5f7640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f96d40ae310 con 0x7f96d40adea0 2024-05-11T08:07:21.786 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.784+0000 7f96d8b92640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f96d40aed70 0x7f96d41f3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:21.787 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.784+0000 7f96d8b92640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f96d40aed70 0x7f96d41f3620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:56784/0 (socket says 172.21.15.167:56784) 2024-05-11T08:07:21.787 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.784+0000 7f96d8b92640 1 -- 172.21.15.167:0/3394973738 learned_addr learned my addr 172.21.15.167:0/3394973738 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:07:21.787 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.784+0000 7f96d9393640 1 --2- 172.21.15.167:0/3394973738 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f96d40adea0 0x7f96d409e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:21.787 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.784+0000 7f96d9b94640 1 --2- 172.21.15.167:0/3394973738 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f96d40cd430 0x7f96d409c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:21.787 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.784+0000 7f96d8b92640 1 -- 172.21.15.167:0/3394973738 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f96d40adea0 msgr2=0x7f96d409e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:21.787 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.784+0000 7f96d8b92640 1 --2- 172.21.15.167:0/3394973738 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f96d40adea0 0x7f96d409e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:21.787 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.784+0000 7f96d8b92640 1 -- 172.21.15.167:0/3394973738 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f96d40cd430 msgr2=0x7f96d409c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:21.787 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.784+0000 7f96d8b92640 1 --2- 172.21.15.167:0/3394973738 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f96d40cd430 0x7f96d409c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:21.787 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.784+0000 7f96d8b92640 1 -- 172.21.15.167:0/3394973738 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96d41f3b60 con 0x7f96d40aed70 2024-05-11T08:07:21.787 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.784+0000 7f96d8b92640 1 --2- 172.21.15.167:0/3394973738 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f96d40aed70 0x7f96d41f3620 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f96d000ebe0 tx=0x7f96d000c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:21.787 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.784+0000 7f96c5ffb640 1 -- 172.21.15.167:0/3394973738 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96d0017070 con 0x7f96d40aed70 2024-05-11T08:07:21.787 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.785+0000 7f96c5ffb640 1 -- 172.21.15.167:0/3394973738 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96d000cc30 con 0x7f96d40aed70 2024-05-11T08:07:21.787 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.785+0000 7f96db5f7640 1 -- 172.21.15.167:0/3394973738 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96d409e900 con 0x7f96d40aed70 2024-05-11T08:07:21.788 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.785+0000 7f96c5ffb640 1 -- 172.21.15.167:0/3394973738 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96d0013470 con 0x7f96d40aed70 2024-05-11T08:07:21.788 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.785+0000 7f96db5f7640 1 -- 172.21.15.167:0/3394973738 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96d40c5f60 con 0x7f96d40aed70 2024-05-11T08:07:21.789 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.785+0000 7f96db5f7640 1 -- 172.21.15.167:0/3394973738 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f96d40c62a0 con 0x7f96d40aed70 2024-05-11T08:07:21.789 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.786+0000 7f96c5ffb640 1 -- 172.21.15.167:0/3394973738 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f96d001a070 con 0x7f96d40aed70 2024-05-11T08:07:21.789 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.787+0000 7f96c5ffb640 1 --2- 172.21.15.167:0/3394973738 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f96b80762d0 0x7f96b8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:21.789 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.787+0000 7f96c5ffb640 1 -- 172.21.15.167:0/3394973738 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5335+0+0 (secure 0 0 0) 0x7f96d0097750 con 0x7f96d40aed70 2024-05-11T08:07:21.789 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.787+0000 7f96d9393640 1 --2- 172.21.15.167:0/3394973738 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f96b80762d0 0x7f96b8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:21.789 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.787+0000 7f96c5ffb640 1 --2- 172.21.15.167:0/3394973738 >> [v2:172.21.15.167:6824/453689293,v1:172.21.15.167:6825/453689293] conn(0x7f96b807c770 0x7f96b807ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:21.789 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.787+0000 7f96c5ffb640 1 -- 172.21.15.167:0/3394973738 --> [v2:172.21.15.167:6824/453689293,v1:172.21.15.167:6825/453689293] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f96d0016ec0 con 0x7f96b807c770 2024-05-11T08:07:21.789 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.787+0000 7f96c5ffb640 1 -- 172.21.15.167:0/3394973738 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f96d0097be0 con 0x7f96d40aed70 2024-05-11T08:07:21.790 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.787+0000 7f96d9b94640 1 --2- 172.21.15.167:0/3394973738 >> [v2:172.21.15.167:6824/453689293,v1:172.21.15.167:6825/453689293] conn(0x7f96b807c770 0x7f96b807ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:21.790 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.787+0000 7f96d9393640 1 --2- 172.21.15.167:0/3394973738 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f96b80762d0 0x7f96b8078790 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f96c8034000 tx=0x7f96c802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:21.790 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.788+0000 7f96d9b94640 1 --2- 172.21.15.167:0/3394973738 >> [v2:172.21.15.167:6824/453689293,v1:172.21.15.167:6825/453689293] conn(0x7f96b807c770 0x7f96b807ebb0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:21.790 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.788+0000 7f96c5ffb640 1 -- 172.21.15.167:0/3394973738 <== osd.3 v2:172.21.15.167:6824/453689293 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f96d0016ec0 con 0x7f96b807c770 2024-05-11T08:07:21.801 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.801+0000 7f96db5f7640 1 -- 172.21.15.167:0/3394973738 --> [v2:172.21.15.167:6824/453689293,v1:172.21.15.167:6825/453689293] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f96d4069590 con 0x7f96b807c770 2024-05-11T08:07:21.801 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.801+0000 7f96c5ffb640 1 -- 172.21.15.167:0/3394973738 <== osd.3 v2:172.21.15.167:6824/453689293 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f96d4069590 con 0x7f96b807c770 2024-05-11T08:07:21.802 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-05-11T08:07:21.802 INFO:teuthology.orchestra.run.smithi167.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-11T08:07:21.802 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-05-11T08:07:21.803 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.802+0000 7f96db5f7640 1 -- 172.21.15.167:0/3394973738 >> [v2:172.21.15.167:6824/453689293,v1:172.21.15.167:6825/453689293] conn(0x7f96b807c770 msgr2=0x7f96b807ebb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:21.803 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.802+0000 7f96db5f7640 1 --2- 172.21.15.167:0/3394973738 >> [v2:172.21.15.167:6824/453689293,v1:172.21.15.167:6825/453689293] conn(0x7f96b807c770 0x7f96b807ebb0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:21.803 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.802+0000 7f96db5f7640 1 -- 172.21.15.167:0/3394973738 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f96b80762d0 msgr2=0x7f96b8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:21.803 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.802+0000 7f96db5f7640 1 --2- 172.21.15.167:0/3394973738 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f96b80762d0 0x7f96b8078790 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f96c8034000 tx=0x7f96c802e000 comp rx=0 tx=0).stop 2024-05-11T08:07:21.803 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.802+0000 7f96db5f7640 1 -- 172.21.15.167:0/3394973738 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f96d40aed70 msgr2=0x7f96d41f3620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:21.803 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.802+0000 7f96db5f7640 1 --2- 172.21.15.167:0/3394973738 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f96d40aed70 0x7f96d41f3620 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f96d000ebe0 tx=0x7f96d000c520 comp rx=0 tx=0).stop 2024-05-11T08:07:21.803 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.802+0000 7f96d9b94640 1 -- 172.21.15.167:0/3394973738 reap_dead start 2024-05-11T08:07:21.803 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.802+0000 7f96db5f7640 1 -- 172.21.15.167:0/3394973738 shutdown_connections 2024-05-11T08:07:21.803 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.802+0000 7f96db5f7640 1 -- 172.21.15.167:0/3394973738 >> 172.21.15.167:0/3394973738 conn(0x7f96d40d10d0 msgr2=0x7f96d406fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:07:21.804 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.802+0000 7f96db5f7640 1 -- 172.21.15.167:0/3394973738 shutdown_connections 2024-05-11T08:07:21.804 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.802+0000 7f96db5f7640 1 -- 172.21.15.167:0/3394973738 wait complete. 2024-05-11T08:07:21.812 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-05-11T08:07:21.913 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.913+0000 7f7ccc714640 1 Processor -- start 2024-05-11T08:07:21.913 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.913+0000 7f7ccc714640 1 -- start start 2024-05-11T08:07:21.915 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.914+0000 7f7ccc714640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f7cc40cd610 0x7f7cc4163be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:21.915 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.914+0000 7f7ccc714640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7cc40cd9f0 0x7f7cc4160230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:21.915 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.914+0000 7f7ccc714640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7cc40b6300 con 0x7f7cc4160770 2024-05-11T08:07:21.915 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.914+0000 7f7ccc714640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f7cc40b65e0 con 0x7f7cc40cd610 2024-05-11T08:07:21.915 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.914+0000 7f7ccc714640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f7cc4163100 con 0x7f7cc40cd9f0 2024-05-11T08:07:21.915 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.914+0000 7f7cca4b0640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f7cc40cd610 0x7f7cc4163be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:21.915 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.914+0000 7f7cc9caf640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7cc40cd9f0 0x7f7cc4160230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:21.916 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.914+0000 7f7ccacb1640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f7cc4160770 0x7f7cc40d0ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.167:58200/0 (socket says 172.21.15.167:58200) 2024-05-11T08:07:21.916 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.914+0000 7f7ccacb1640 1 -- 172.21.15.167:0/237960701 learned_addr learned my addr 172.21.15.167:0/237960701 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T08:07:21.916 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.914+0000 7f7cca4b0640 1 -- 172.21.15.167:0/237960701 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7cc40cd9f0 msgr2=0x7f7cc4160230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:07:21.916 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.914+0000 7f7cca4b0640 1 --2- 172.21.15.167:0/237960701 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7cc40cd9f0 0x7f7cc4160230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:21.916 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.914+0000 7f7cca4b0640 1 -- 172.21.15.167:0/237960701 >> v1:172.21.15.204:6789/0 conn(0x7f7cc4160770 legacy=0x7f7cc40d0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:21.916 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.914+0000 7f7cca4b0640 1 -- 172.21.15.167:0/237960701 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cc41633e0 con 0x7f7cc40cd610 2024-05-11T08:07:21.916 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.915+0000 7f7cca4b0640 1 --2- 172.21.15.167:0/237960701 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7cc40cd610 0x7f7cc4163be0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f7cc0009d90 tx=0x7f7cc002a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fcc76c7d1e2d6a34 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:21.919 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.915+0000 7f7cc94ae640 1 -- 172.21.15.167:0/237960701 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cc0007740 con 0x7f7cc40cd610 2024-05-11T08:07:21.919 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.915+0000 7f7cc94ae640 1 -- 172.21.15.167:0/237960701 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7cc00078e0 con 0x7f7cc40cd610 2024-05-11T08:07:21.920 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.915+0000 7f7cc94ae640 1 -- 172.21.15.167:0/237960701 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cc0007cb0 con 0x7f7cc40cd610 2024-05-11T08:07:21.920 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.916+0000 7f7ccc714640 1 -- 172.21.15.167:0/237960701 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7cc40cd610 msgr2=0x7f7cc4163be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:21.920 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.916+0000 7f7ccc714640 1 --2- 172.21.15.167:0/237960701 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7cc40cd610 0x7f7cc4163be0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f7cc0009d90 tx=0x7f7cc002a660 comp rx=0 tx=0).stop 2024-05-11T08:07:21.920 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.916+0000 7f7ccc714640 1 -- 172.21.15.167:0/237960701 shutdown_connections 2024-05-11T08:07:21.920 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.916+0000 7f7ccc714640 1 --2- 172.21.15.167:0/237960701 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7cc40cd9f0 0x7f7cc4160230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:21.920 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.916+0000 7f7ccc714640 1 --2- 172.21.15.167:0/237960701 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7cc40cd610 0x7f7cc4163be0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:21.920 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.916+0000 7f7ccc714640 1 -- 172.21.15.167:0/237960701 >> 172.21.15.167:0/237960701 conn(0x7f7cc4072730 msgr2=0x7f7cc40d2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:07:21.920 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.916+0000 7f7ccc714640 1 -- 172.21.15.167:0/237960701 shutdown_connections 2024-05-11T08:07:21.920 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.916+0000 7f7ccc714640 1 -- 172.21.15.167:0/237960701 wait complete. 2024-05-11T08:07:21.920 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.917+0000 7f7ccc714640 1 Processor -- start 2024-05-11T08:07:21.920 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.917+0000 7f7ccc714640 1 -- start start 2024-05-11T08:07:21.921 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.918+0000 7f7ccc714640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7cc40cd610 0x7f7cc415d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:21.921 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.918+0000 7f7ccc714640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7cc40cd9f0 0x7f7cc40cac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:21.921 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.918+0000 7f7ccc714640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7cc4160770 0x7f7cc41550e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:21.921 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.918+0000 7f7ccc714640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7cc40cde50 con 0x7f7cc4160770 2024-05-11T08:07:21.921 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.918+0000 7f7ccc714640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7cc40d2a80 con 0x7f7cc40cd610 2024-05-11T08:07:21.921 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.918+0000 7f7ccc714640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f7cc40b5e90 con 0x7f7cc40cd9f0 2024-05-11T08:07:21.921 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.918+0000 7f7ccacb1640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7cc4160770 0x7f7cc41550e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:21.921 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.918+0000 7f7ccacb1640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7cc4160770 0x7f7cc41550e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:56788/0 (socket says 172.21.15.167:56788) 2024-05-11T08:07:21.921 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.918+0000 7f7ccacb1640 1 -- 172.21.15.167:0/3897212816 learned_addr learned my addr 172.21.15.167:0/3897212816 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:07:21.921 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.918+0000 7f7cc9caf640 1 --2- 172.21.15.167:0/3897212816 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7cc40cd9f0 0x7f7cc40cac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:21.922 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.918+0000 7f7cca4b0640 1 --2- 172.21.15.167:0/3897212816 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7cc40cd610 0x7f7cc415d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:21.922 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.918+0000 7f7ccacb1640 1 -- 172.21.15.167:0/3897212816 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7cc40cd9f0 msgr2=0x7f7cc40cac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:21.922 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.918+0000 7f7ccacb1640 1 --2- 172.21.15.167:0/3897212816 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7cc40cd9f0 0x7f7cc40cac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:21.922 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.918+0000 7f7ccacb1640 1 -- 172.21.15.167:0/3897212816 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7cc40cd610 msgr2=0x7f7cc415d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:21.922 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.918+0000 7f7ccacb1640 1 --2- 172.21.15.167:0/3897212816 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7cc40cd610 0x7f7cc415d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:21.922 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.918+0000 7f7ccacb1640 1 -- 172.21.15.167:0/3897212816 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cc40c8130 con 0x7f7cc4160770 2024-05-11T08:07:21.922 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.919+0000 7f7ccacb1640 1 --2- 172.21.15.167:0/3897212816 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7cc4160770 0x7f7cc41550e0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f7cbc00c450 tx=0x7f7cbc00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:21.922 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.919+0000 7f7cbaffd640 1 -- 172.21.15.167:0/3897212816 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cbc017070 con 0x7f7cc4160770 2024-05-11T08:07:21.922 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.919+0000 7f7cbaffd640 1 -- 172.21.15.167:0/3897212816 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7cbc005ce0 con 0x7f7cc4160770 2024-05-11T08:07:21.923 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.919+0000 7f7ccc714640 1 -- 172.21.15.167:0/3897212816 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cc40c8410 con 0x7f7cc4160770 2024-05-11T08:07:21.923 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.919+0000 7f7cbaffd640 1 -- 172.21.15.167:0/3897212816 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cbc007270 con 0x7f7cc4160770 2024-05-11T08:07:21.923 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.919+0000 7f7ccc714640 1 -- 172.21.15.167:0/3897212816 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cc40c89d0 con 0x7f7cc4160770 2024-05-11T08:07:21.923 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.920+0000 7f7ccc714640 1 -- 172.21.15.167:0/3897212816 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7cc40c8ec0 con 0x7f7cc4160770 2024-05-11T08:07:21.924 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.920+0000 7f7cbaffd640 1 -- 172.21.15.167:0/3897212816 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f7cbc01a070 con 0x7f7cc4160770 2024-05-11T08:07:21.924 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.921+0000 7f7cbaffd640 1 --2- 172.21.15.167:0/3897212816 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f7ca80762d0 0x7f7ca8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:21.924 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.921+0000 7f7cbaffd640 1 -- 172.21.15.167:0/3897212816 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5335+0+0 (secure 0 0 0) 0x7f7cbc096700 con 0x7f7cc4160770 2024-05-11T08:07:21.924 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.921+0000 7f7cca4b0640 1 --2- 172.21.15.167:0/3897212816 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f7ca80762d0 0x7f7ca8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:21.924 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.921+0000 7f7cca4b0640 1 --2- 172.21.15.167:0/3897212816 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f7ca80762d0 0x7f7ca8078790 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f7cc0034000 tx=0x7f7cc002f000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:21.925 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:21.925+0000 7f7cbaffd640 1 -- 172.21.15.167:0/3897212816 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f7cbc05fb80 con 0x7f7cc4160770 2024-05-11T08:07:22.106 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.106+0000 7f7ccc714640 1 -- 172.21.15.167:0/3897212816 --> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7f7cc4069590 con 0x7f7ca80762d0 2024-05-11T08:07:22.107 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.107+0000 7f7cbaffd640 1 -- 172.21.15.167:0/3897212816 <== mgr.4113 v2:172.21.15.167:6832/253569496 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f7cc4069590 con 0x7f7ca80762d0 2024-05-11T08:07:22.107 INFO:teuthology.orchestra.run.smithi167.stderr:instructed osd(s) 3 to deep-scrub 2024-05-11T08:07:22.109 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.109+0000 7f7ccc714640 1 -- 172.21.15.167:0/3897212816 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f7ca80762d0 msgr2=0x7f7ca8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:22.109 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.109+0000 7f7ccc714640 1 --2- 172.21.15.167:0/3897212816 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f7ca80762d0 0x7f7ca8078790 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f7cc0034000 tx=0x7f7cc002f000 comp rx=0 tx=0).stop 2024-05-11T08:07:22.109 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.109+0000 7f7ccc714640 1 -- 172.21.15.167:0/3897212816 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7cc4160770 msgr2=0x7f7cc41550e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:22.109 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.109+0000 7f7ccc714640 1 --2- 172.21.15.167:0/3897212816 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7cc4160770 0x7f7cc41550e0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f7cbc00c450 tx=0x7f7cbc00c910 comp rx=0 tx=0).stop 2024-05-11T08:07:22.109 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.109+0000 7f7ccc714640 1 -- 172.21.15.167:0/3897212816 shutdown_connections 2024-05-11T08:07:22.110 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.109+0000 7f7ccc714640 1 --2- 172.21.15.167:0/3897212816 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f7ca80762d0 0x7f7ca8078790 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:22.110 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.109+0000 7f7ccc714640 1 --2- 172.21.15.167:0/3897212816 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7cc4160770 0x7f7cc41550e0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:22.110 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.109+0000 7f7ccc714640 1 --2- 172.21.15.167:0/3897212816 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7cc40cd9f0 0x7f7cc40cac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:22.110 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.109+0000 7f7ccc714640 1 --2- 172.21.15.167:0/3897212816 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7cc40cd610 0x7f7cc415d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:22.110 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.109+0000 7f7ccc714640 1 -- 172.21.15.167:0/3897212816 >> 172.21.15.167:0/3897212816 conn(0x7f7cc4072730 msgr2=0x7f7cc4158bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:07:22.110 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.109+0000 7f7ccc714640 1 -- 172.21.15.167:0/3897212816 shutdown_connections 2024-05-11T08:07:22.110 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.109+0000 7f7ccc714640 1 -- 172.21.15.167:0/3897212816 wait complete. 2024-05-11T08:07:22.120 INFO:tasks.ceph:Scrubbing osd.4 2024-05-11T08:07:22.120 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-05-11T08:07:22.226 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.226+0000 7fd4afd89640 1 Processor -- start 2024-05-11T08:07:22.226 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.226+0000 7fd4afd89640 1 -- start start 2024-05-11T08:07:22.229 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.227+0000 7fd4afd89640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fd4a8148d50 0x7fd4a80cd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:22.229 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.227+0000 7fd4afd89640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd4a8163580 0x7fd4a80d0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:22.229 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.227+0000 7fd4afd89640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4a80cfad0 con 0x7fd4a80d2630 2024-05-11T08:07:22.230 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.227+0000 7fd4afd89640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7fd4a80cda40 con 0x7fd4a8148d50 2024-05-11T08:07:22.230 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.227+0000 7fd4afd89640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fd4a80cfdb0 con 0x7fd4a8163580 2024-05-11T08:07:22.230 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.227+0000 7fd4adb25640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fd4a8148d50 0x7fd4a80cd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:22.230 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.227+0000 7fd4adb25640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fd4a8148d50 0x7fd4a80cd500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:56792/0 (socket says 172.21.15.167:56792) 2024-05-11T08:07:22.230 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.227+0000 7fd4adb25640 1 -- 172.21.15.167:0/3792990502 learned_addr learned my addr 172.21.15.167:0/3792990502 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:07:22.230 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.227+0000 7fd4ae326640 1 --2- 172.21.15.167:0/3792990502 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd4a8163580 0x7fd4a80d0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:22.230 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.227+0000 7fd4adb25640 1 -- 172.21.15.167:0/3792990502 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd4a8163580 msgr2=0x7fd4a80d0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:07:22.230 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.227+0000 7fd4adb25640 1 --2- 172.21.15.167:0/3792990502 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd4a8163580 0x7fd4a80d0e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:22.231 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.227+0000 7fd4adb25640 1 -- 172.21.15.167:0/3792990502 >> v1:172.21.15.204:6789/0 conn(0x7fd4a80d2630 legacy=0x7fd4a8162e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:22.231 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.227+0000 7fd4adb25640 1 -- 172.21.15.167:0/3792990502 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4a80d0090 con 0x7fd4a8148d50 2024-05-11T08:07:22.231 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.228+0000 7fd4adb25640 1 --2- 172.21.15.167:0/3792990502 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd4a8148d50 0x7fd4a80cd500 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fd4a4009870 tx=0x7fd4a402a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b047032f4b4e1e2 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:22.231 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.228+0000 7fd4acb23640 1 -- 172.21.15.167:0/3792990502 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4a4037070 con 0x7fd4a8148d50 2024-05-11T08:07:22.231 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.228+0000 7fd4acb23640 1 -- 172.21.15.167:0/3792990502 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4a4032440 con 0x7fd4a8148d50 2024-05-11T08:07:22.231 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.228+0000 7fd4acb23640 1 -- 172.21.15.167:0/3792990502 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4a4032810 con 0x7fd4a8148d50 2024-05-11T08:07:22.231 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.229+0000 7fd4afd89640 1 -- 172.21.15.167:0/3792990502 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd4a8148d50 msgr2=0x7fd4a80cd500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:22.231 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.229+0000 7fd4afd89640 1 --2- 172.21.15.167:0/3792990502 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd4a8148d50 0x7fd4a80cd500 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fd4a4009870 tx=0x7fd4a402a130 comp rx=0 tx=0).stop 2024-05-11T08:07:22.231 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.229+0000 7fd4afd89640 1 -- 172.21.15.167:0/3792990502 shutdown_connections 2024-05-11T08:07:22.231 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.229+0000 7fd4afd89640 1 --2- 172.21.15.167:0/3792990502 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd4a8163580 0x7fd4a80d0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:22.231 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.229+0000 7fd4afd89640 1 --2- 172.21.15.167:0/3792990502 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd4a8148d50 0x7fd4a80cd500 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:22.231 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.229+0000 7fd4afd89640 1 -- 172.21.15.167:0/3792990502 >> 172.21.15.167:0/3792990502 conn(0x7fd4a80b4ae0 msgr2=0x7fd4a80b4ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:07:22.231 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.229+0000 7fd4afd89640 1 -- 172.21.15.167:0/3792990502 shutdown_connections 2024-05-11T08:07:22.232 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.229+0000 7fd4afd89640 1 -- 172.21.15.167:0/3792990502 wait complete. 2024-05-11T08:07:22.236 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.230+0000 7fd4afd89640 1 Processor -- start 2024-05-11T08:07:22.236 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.230+0000 7fd4afd89640 1 -- start start 2024-05-11T08:07:22.236 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.231+0000 7fd4afd89640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd4a80d2630 0x7fd4a81476e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:22.236 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.231+0000 7fd4afd89640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd4a8148d50 0x7fd4a81620c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:22.236 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.231+0000 7fd4afd89640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd4a8163580 0x7fd4a8162600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:22.236 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.231+0000 7fd4afd89640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd4a80d1380 con 0x7fd4a8148d50 2024-05-11T08:07:22.236 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.231+0000 7fd4afd89640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd4a80b5de0 con 0x7fd4a80d2630 2024-05-11T08:07:22.236 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.231+0000 7fd4afd89640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fd4a80d1500 con 0x7fd4a8163580 2024-05-11T08:07:22.236 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.231+0000 7fd4ad324640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd4a8148d50 0x7fd4a81620c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:22.237 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.231+0000 7fd4ad324640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd4a8148d50 0x7fd4a81620c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:56802/0 (socket says 172.21.15.167:56802) 2024-05-11T08:07:22.237 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.231+0000 7fd4ad324640 1 -- 172.21.15.167:0/3332392655 learned_addr learned my addr 172.21.15.167:0/3332392655 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:07:22.237 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.231+0000 7fd4adb25640 1 --2- 172.21.15.167:0/3332392655 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd4a80d2630 0x7fd4a81476e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:22.237 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.231+0000 7fd4ae326640 1 --2- 172.21.15.167:0/3332392655 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd4a8163580 0x7fd4a8162600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:22.237 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.231+0000 7fd4ad324640 1 -- 172.21.15.167:0/3332392655 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd4a8163580 msgr2=0x7fd4a8162600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:22.237 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.231+0000 7fd4ad324640 1 --2- 172.21.15.167:0/3332392655 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd4a8163580 0x7fd4a8162600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:22.237 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.231+0000 7fd4ad324640 1 -- 172.21.15.167:0/3332392655 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd4a80d2630 msgr2=0x7fd4a81476e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:22.237 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.231+0000 7fd4ad324640 1 --2- 172.21.15.167:0/3332392655 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd4a80d2630 0x7fd4a81476e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:22.237 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.231+0000 7fd4ad324640 1 -- 172.21.15.167:0/3332392655 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4a8147c20 con 0x7fd4a8148d50 2024-05-11T08:07:22.237 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.231+0000 7fd4ad324640 1 --2- 172.21.15.167:0/3332392655 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd4a8148d50 0x7fd4a81620c0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fd49800c450 tx=0x7fd49800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:22.237 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.231+0000 7fd49e7fc640 1 -- 172.21.15.167:0/3332392655 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4980052d0 con 0x7fd4a8148d50 2024-05-11T08:07:22.237 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.231+0000 7fd49e7fc640 1 -- 172.21.15.167:0/3332392655 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd498005470 con 0x7fd4a8148d50 2024-05-11T08:07:22.237 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.231+0000 7fd4afd89640 1 -- 172.21.15.167:0/3332392655 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4a81e9b60 con 0x7fd4a8148d50 2024-05-11T08:07:22.238 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.231+0000 7fd49e7fc640 1 -- 172.21.15.167:0/3332392655 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd498012870 con 0x7fd4a8148d50 2024-05-11T08:07:22.238 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.231+0000 7fd4afd89640 1 -- 172.21.15.167:0/3332392655 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4a81ea120 con 0x7fd4a8148d50 2024-05-11T08:07:22.238 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.232+0000 7fd4afd89640 1 -- 172.21.15.167:0/3332392655 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd4a8069590 con 0x7fd4a8148d50 2024-05-11T08:07:22.238 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.233+0000 7fd49e7fc640 1 -- 172.21.15.167:0/3332392655 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fd498012a10 con 0x7fd4a8148d50 2024-05-11T08:07:22.238 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.234+0000 7fd49e7fc640 1 --2- 172.21.15.167:0/3332392655 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fd48c0762d0 0x7fd48c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:22.238 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.234+0000 7fd4adb25640 1 --2- 172.21.15.167:0/3332392655 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fd48c0762d0 0x7fd48c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:22.238 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.234+0000 7fd49e7fc640 1 -- 172.21.15.167:0/3332392655 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5335+0+0 (secure 0 0 0) 0x7fd498097930 con 0x7fd4a8148d50 2024-05-11T08:07:22.238 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.234+0000 7fd49e7fc640 1 --2- 172.21.15.167:0/3332392655 >> [v2:172.21.15.204:6800/1311831500,v1:172.21.15.204:6801/1311831500] conn(0x7fd48c07c770 0x7fd48c07ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:22.238 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.234+0000 7fd49e7fc640 1 -- 172.21.15.167:0/3332392655 --> [v2:172.21.15.204:6800/1311831500,v1:172.21.15.204:6801/1311831500] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd48c07f280 con 0x7fd48c07c770 2024-05-11T08:07:22.238 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.234+0000 7fd49e7fc640 1 -- 172.21.15.167:0/3332392655 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fd498097dc0 con 0x7fd4a8148d50 2024-05-11T08:07:22.238 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.234+0000 7fd4adb25640 1 --2- 172.21.15.167:0/3332392655 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fd48c0762d0 0x7fd48c078790 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fd4a4034000 tx=0x7fd4a402e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:22.238 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.234+0000 7fd4ae326640 1 --2- 172.21.15.167:0/3332392655 >> [v2:172.21.15.204:6800/1311831500,v1:172.21.15.204:6801/1311831500] conn(0x7fd48c07c770 0x7fd48c07ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:22.238 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.235+0000 7fd4ae326640 1 --2- 172.21.15.167:0/3332392655 >> [v2:172.21.15.204:6800/1311831500,v1:172.21.15.204:6801/1311831500] conn(0x7fd48c07c770 0x7fd48c07ebb0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:22.239 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.235+0000 7fd49e7fc640 1 -- 172.21.15.167:0/3332392655 <== osd.4 v2:172.21.15.204:6800/1311831500 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd48c07f280 con 0x7fd48c07c770 2024-05-11T08:07:22.248 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.248+0000 7fd4afd89640 1 -- 172.21.15.167:0/3332392655 --> [v2:172.21.15.204:6800/1311831500,v1:172.21.15.204:6801/1311831500] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fd4a80b8e60 con 0x7fd48c07c770 2024-05-11T08:07:22.249 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.249+0000 7fd49e7fc640 1 -- 172.21.15.167:0/3332392655 <== osd.4 v2:172.21.15.204:6800/1311831500 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fd4a80b8e60 con 0x7fd48c07c770 2024-05-11T08:07:22.249 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-05-11T08:07:22.249 INFO:teuthology.orchestra.run.smithi167.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-11T08:07:22.249 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-05-11T08:07:22.250 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.250+0000 7fd4afd89640 1 -- 172.21.15.167:0/3332392655 >> [v2:172.21.15.204:6800/1311831500,v1:172.21.15.204:6801/1311831500] conn(0x7fd48c07c770 msgr2=0x7fd48c07ebb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:22.250 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.250+0000 7fd4afd89640 1 --2- 172.21.15.167:0/3332392655 >> [v2:172.21.15.204:6800/1311831500,v1:172.21.15.204:6801/1311831500] conn(0x7fd48c07c770 0x7fd48c07ebb0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:22.250 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.250+0000 7fd4afd89640 1 -- 172.21.15.167:0/3332392655 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fd48c0762d0 msgr2=0x7fd48c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:22.251 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.250+0000 7fd4afd89640 1 --2- 172.21.15.167:0/3332392655 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fd48c0762d0 0x7fd48c078790 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fd4a4034000 tx=0x7fd4a402e000 comp rx=0 tx=0).stop 2024-05-11T08:07:22.251 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.250+0000 7fd4afd89640 1 -- 172.21.15.167:0/3332392655 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd4a8148d50 msgr2=0x7fd4a81620c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:22.251 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.250+0000 7fd4afd89640 1 --2- 172.21.15.167:0/3332392655 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd4a8148d50 0x7fd4a81620c0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fd49800c450 tx=0x7fd49800c910 comp rx=0 tx=0).stop 2024-05-11T08:07:22.251 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.250+0000 7fd4ae326640 1 -- 172.21.15.167:0/3332392655 reap_dead start 2024-05-11T08:07:22.251 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.250+0000 7fd4afd89640 1 -- 172.21.15.167:0/3332392655 shutdown_connections 2024-05-11T08:07:22.251 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.250+0000 7fd4afd89640 1 -- 172.21.15.167:0/3332392655 >> 172.21.15.167:0/3332392655 conn(0x7fd4a80b4ae0 msgr2=0x7fd4a80cee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:07:22.251 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.250+0000 7fd4afd89640 1 -- 172.21.15.167:0/3332392655 shutdown_connections 2024-05-11T08:07:22.251 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.250+0000 7fd4afd89640 1 -- 172.21.15.167:0/3332392655 wait complete. 2024-05-11T08:07:22.260 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-05-11T08:07:22.361 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.361+0000 7f8ee4ab0640 1 Processor -- start 2024-05-11T08:07:22.361 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.361+0000 7f8ee4ab0640 1 -- start start 2024-05-11T08:07:22.362 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.362+0000 7f8ee4ab0640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f8ee00b5a10 0x7f8ee00b5e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:22.363 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.362+0000 7f8ee4ab0640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8ee00710f0 0x7f8ee0071580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:22.363 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.362+0000 7f8ee4ab0640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ee0163e60 con 0x7f8ee01488a0 2024-05-11T08:07:22.363 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.362+0000 7f8ee4ab0640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f8ee00b63b0 con 0x7f8ee00b5a10 2024-05-11T08:07:22.363 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.362+0000 7f8ee4ab0640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f8ee00b5580 con 0x7f8ee00710f0 2024-05-11T08:07:22.363 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.362+0000 7f8eded9d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8ee00710f0 0x7f8ee0071580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:22.363 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.362+0000 7f8ed7fff640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f8ee00b5a10 0x7f8ee00b5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:22.363 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.362+0000 7f8ede59c640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f8ee01488a0 0x7f8ee015f910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.167:58224/0 (socket says 172.21.15.167:58224) 2024-05-11T08:07:22.363 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.362+0000 7f8ede59c640 1 -- 172.21.15.167:0/3799732183 learned_addr learned my addr 172.21.15.167:0/3799732183 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T08:07:22.363 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.362+0000 7f8eded9d640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8ee00710f0 0x7f8ee0071580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.167:48730/0 (socket says 172.21.15.167:48730) 2024-05-11T08:07:22.363 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.362+0000 7f8eded9d640 1 -- 172.21.15.167:0/3799732183 >> v2:172.21.15.204:3300/0 conn(0x7f8ee00b5a10 msgr2=0x7f8ee00b5e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:07:22.363 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.362+0000 7f8eded9d640 1 --2- 172.21.15.167:0/3799732183 >> v2:172.21.15.204:3300/0 conn(0x7f8ee00b5a10 0x7f8ee00b5e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:22.363 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.362+0000 7f8eded9d640 1 -- 172.21.15.167:0/3799732183 >> v1:172.21.15.204:6789/0 conn(0x7f8ee01488a0 legacy=0x7f8ee015f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:22.363 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.362+0000 7f8eded9d640 1 -- 172.21.15.167:0/3799732183 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ee00b4f70 con 0x7f8ee00710f0 2024-05-11T08:07:22.364 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.362+0000 7f8eddd9b640 1 -- 172.21.15.167:0/3799732183 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3597688633 0 0) 0x7f8ee0163e60 con 0x7f8ee01488a0 2024-05-11T08:07:22.364 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.362+0000 7f8ed7fff640 1 --2- 172.21.15.167:0/3799732183 >> v2:172.21.15.204:3300/0 conn(0x7f8ee00b5a10 0x7f8ee00b5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T08:07:22.365 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.363+0000 7f8eded9d640 1 --2- 172.21.15.167:0/3799732183 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8ee00710f0 0x7f8ee0071580 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f8ed8009d90 tx=0x7f8ed802a5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d80f37023d8f9c56 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:22.365 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.363+0000 7f8eddd9b640 1 -- 172.21.15.167:0/3799732183 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ed8031600 con 0x7f8ee00710f0 2024-05-11T08:07:22.365 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.363+0000 7f8eddd9b640 1 -- 172.21.15.167:0/3799732183 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ed80317a0 con 0x7f8ee00710f0 2024-05-11T08:07:22.365 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.363+0000 7f8eddd9b640 1 -- 172.21.15.167:0/3799732183 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ed8031b70 con 0x7f8ee00710f0 2024-05-11T08:07:22.365 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.364+0000 7f8ee4ab0640 1 -- 172.21.15.167:0/3799732183 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8ee00710f0 msgr2=0x7f8ee0071580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:22.365 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.364+0000 7f8ee4ab0640 1 --2- 172.21.15.167:0/3799732183 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8ee00710f0 0x7f8ee0071580 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f8ed8009d90 tx=0x7f8ed802a5d0 comp rx=0 tx=0).stop 2024-05-11T08:07:22.365 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.364+0000 7f8ee4ab0640 1 -- 172.21.15.167:0/3799732183 shutdown_connections 2024-05-11T08:07:22.366 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.364+0000 7f8ee4ab0640 1 --2- 172.21.15.167:0/3799732183 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8ee00710f0 0x7f8ee0071580 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:22.366 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.364+0000 7f8ee4ab0640 1 --2- 172.21.15.167:0/3799732183 >> v2:172.21.15.204:3300/0 conn(0x7f8ee00b5a10 0x7f8ee00b5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:22.366 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.364+0000 7f8ee4ab0640 1 -- 172.21.15.167:0/3799732183 >> 172.21.15.167:0/3799732183 conn(0x7f8ee00d2550 msgr2=0x7f8ee00ce890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:07:22.366 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.364+0000 7f8ee4ab0640 1 -- 172.21.15.167:0/3799732183 shutdown_connections 2024-05-11T08:07:22.366 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.364+0000 7f8ee4ab0640 1 -- 172.21.15.167:0/3799732183 wait complete. 2024-05-11T08:07:22.366 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.365+0000 7f8ee4ab0640 1 Processor -- start 2024-05-11T08:07:22.366 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.365+0000 7f8ee4ab0640 1 -- start start 2024-05-11T08:07:22.368 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.366+0000 7f8ee4ab0640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8ee00710f0 0x7f8ee013d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:22.368 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.366+0000 7f8ee4ab0640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8ee00b5a10 0x7f8ee013eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:22.368 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.366+0000 7f8ee4ab0640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8ee01488a0 0x7f8ee0141d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:22.368 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.366+0000 7f8ee4ab0640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8ee0160020 con 0x7f8ee00710f0 2024-05-11T08:07:22.368 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.366+0000 7f8ee4ab0640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8ee00b5210 con 0x7f8ee01488a0 2024-05-11T08:07:22.368 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.366+0000 7f8ee4ab0640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f8ee0148c80 con 0x7f8ee00b5a10 2024-05-11T08:07:22.368 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.366+0000 7f8ede59c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8ee00710f0 0x7f8ee013d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:22.368 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.366+0000 7f8ede59c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8ee00710f0 0x7f8ee013d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:56818/0 (socket says 172.21.15.167:56818) 2024-05-11T08:07:22.369 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.366+0000 7f8ede59c640 1 -- 172.21.15.167:0/121272962 learned_addr learned my addr 172.21.15.167:0/121272962 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:07:22.369 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.366+0000 7f8eded9d640 1 --2- 172.21.15.167:0/121272962 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8ee01488a0 0x7f8ee0141d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:22.369 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.366+0000 7f8ed7fff640 1 --2- 172.21.15.167:0/121272962 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8ee00b5a10 0x7f8ee013eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:22.369 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.366+0000 7f8ede59c640 1 -- 172.21.15.167:0/121272962 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8ee00b5a10 msgr2=0x7f8ee013eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:22.369 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.366+0000 7f8ede59c640 1 --2- 172.21.15.167:0/121272962 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8ee00b5a10 0x7f8ee013eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:22.369 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.366+0000 7f8ede59c640 1 -- 172.21.15.167:0/121272962 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8ee01488a0 msgr2=0x7f8ee0141d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:22.369 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.366+0000 7f8ede59c640 1 --2- 172.21.15.167:0/121272962 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8ee01488a0 0x7f8ee0141d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:22.369 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.366+0000 7f8ede59c640 1 -- 172.21.15.167:0/121272962 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ee0142260 con 0x7f8ee00710f0 2024-05-11T08:07:22.369 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.366+0000 7f8ed7fff640 1 --2- 172.21.15.167:0/121272962 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8ee00b5a10 0x7f8ee013eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T08:07:22.369 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.367+0000 7f8ede59c640 1 --2- 172.21.15.167:0/121272962 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8ee00710f0 0x7f8ee013d670 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f8ed000c4c0 tx=0x7f8ed000c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:22.369 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.367+0000 7f8ed6ffd640 1 -- 172.21.15.167:0/121272962 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ed0018070 con 0x7f8ee00710f0 2024-05-11T08:07:22.369 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.367+0000 7f8ed6ffd640 1 -- 172.21.15.167:0/121272962 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ed0007500 con 0x7f8ee00710f0 2024-05-11T08:07:22.369 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.367+0000 7f8ee4ab0640 1 -- 172.21.15.167:0/121272962 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ee013dd00 con 0x7f8ee00710f0 2024-05-11T08:07:22.369 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.367+0000 7f8ed6ffd640 1 -- 172.21.15.167:0/121272962 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ed0004a20 con 0x7f8ee00710f0 2024-05-11T08:07:22.370 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.367+0000 7f8ee4ab0640 1 -- 172.21.15.167:0/121272962 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ee013e2c0 con 0x7f8ee00710f0 2024-05-11T08:07:22.370 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.368+0000 7f8ee4ab0640 1 -- 172.21.15.167:0/121272962 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ee0069590 con 0x7f8ee00710f0 2024-05-11T08:07:22.370 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.369+0000 7f8ed6ffd640 1 -- 172.21.15.167:0/121272962 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f8ed001b070 con 0x7f8ee00710f0 2024-05-11T08:07:22.370 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.369+0000 7f8ed6ffd640 1 --2- 172.21.15.167:0/121272962 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f8ec00762d0 0x7f8ec0078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:22.371 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.369+0000 7f8ed7fff640 1 --2- 172.21.15.167:0/121272962 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f8ec00762d0 0x7f8ec0078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:22.371 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.369+0000 7f8ed6ffd640 1 -- 172.21.15.167:0/121272962 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5335+0+0 (secure 0 0 0) 0x7f8ed0096740 con 0x7f8ee00710f0 2024-05-11T08:07:22.371 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.370+0000 7f8ed7fff640 1 --2- 172.21.15.167:0/121272962 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f8ec00762d0 0x7f8ec0078790 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f8ee0140790 tx=0x7f8ec8008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:22.373 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.374+0000 7f8ed6ffd640 1 -- 172.21.15.167:0/121272962 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f8ed00602a0 con 0x7f8ee00710f0 2024-05-11T08:07:22.553 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.552+0000 7f8ee4ab0640 1 -- 172.21.15.167:0/121272962 --> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7f8ee00738c0 con 0x7f8ec00762d0 2024-05-11T08:07:22.553 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.553+0000 7f8ed6ffd640 1 -- 172.21.15.167:0/121272962 <== mgr.4113 v2:172.21.15.167:6832/253569496 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f8ee00738c0 con 0x7f8ec00762d0 2024-05-11T08:07:22.553 INFO:teuthology.orchestra.run.smithi167.stderr:instructed osd(s) 4 to deep-scrub 2024-05-11T08:07:22.555 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.555+0000 7f8ee4ab0640 1 -- 172.21.15.167:0/121272962 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f8ec00762d0 msgr2=0x7f8ec0078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:22.555 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.555+0000 7f8ee4ab0640 1 --2- 172.21.15.167:0/121272962 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f8ec00762d0 0x7f8ec0078790 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f8ee0140790 tx=0x7f8ec8008040 comp rx=0 tx=0).stop 2024-05-11T08:07:22.555 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.555+0000 7f8ee4ab0640 1 -- 172.21.15.167:0/121272962 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8ee00710f0 msgr2=0x7f8ee013d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:22.555 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.555+0000 7f8ee4ab0640 1 --2- 172.21.15.167:0/121272962 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8ee00710f0 0x7f8ee013d670 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f8ed000c4c0 tx=0x7f8ed000c980 comp rx=0 tx=0).stop 2024-05-11T08:07:22.555 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.555+0000 7f8ee4ab0640 1 -- 172.21.15.167:0/121272962 shutdown_connections 2024-05-11T08:07:22.555 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.555+0000 7f8ee4ab0640 1 --2- 172.21.15.167:0/121272962 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f8ec00762d0 0x7f8ec0078790 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:22.556 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.555+0000 7f8ee4ab0640 1 --2- 172.21.15.167:0/121272962 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8ee01488a0 0x7f8ee0141d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:22.556 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.555+0000 7f8ee4ab0640 1 --2- 172.21.15.167:0/121272962 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8ee00b5a10 0x7f8ee013eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:22.556 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.555+0000 7f8ee4ab0640 1 --2- 172.21.15.167:0/121272962 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8ee00710f0 0x7f8ee013d670 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:22.556 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.555+0000 7f8ee4ab0640 1 -- 172.21.15.167:0/121272962 >> 172.21.15.167:0/121272962 conn(0x7f8ee00d2550 msgr2=0x7f8ee00b6530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:07:22.556 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.555+0000 7f8ee4ab0640 1 -- 172.21.15.167:0/121272962 shutdown_connections 2024-05-11T08:07:22.556 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.555+0000 7f8ee4ab0640 1 -- 172.21.15.167:0/121272962 wait complete. 2024-05-11T08:07:22.566 INFO:tasks.ceph:Scrubbing osd.5 2024-05-11T08:07:22.566 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-05-11T08:07:22.671 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.671+0000 7f5ccef6a640 1 Processor -- start 2024-05-11T08:07:22.672 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.671+0000 7f5ccef6a640 1 -- start start 2024-05-11T08:07:22.673 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.672+0000 7f5ccef6a640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f5cc8142740 0x7f5cc80b4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:22.673 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.672+0000 7f5ccef6a640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5cc8072150 0x7f5cc8072590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:22.673 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.672+0000 7f5ccef6a640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5cc806eb90 con 0x7f5cc80b4ca0 2024-05-11T08:07:22.673 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.672+0000 7f5ccef6a640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f5cc806f7a0 con 0x7f5cc8142740 2024-05-11T08:07:22.673 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.672+0000 7f5ccef6a640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f5cc806ee70 con 0x7f5cc8072150 2024-05-11T08:07:22.673 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.673+0000 7f5ccd507640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5cc8072150 0x7f5cc8072590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:22.673 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.673+0000 7f5cccd06640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f5cc8142740 0x7f5cc80b4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:22.673 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.673+0000 7f5ccd507640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5cc8072150 0x7f5cc8072590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.167:48750/0 (socket says 172.21.15.167:48750) 2024-05-11T08:07:22.673 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.673+0000 7f5ccd507640 1 -- 172.21.15.167:0/521188716 learned_addr learned my addr 172.21.15.167:0/521188716 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:07:22.674 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.673+0000 7f5ccd507640 1 -- 172.21.15.167:0/521188716 >> v2:172.21.15.204:3300/0 conn(0x7f5cc8142740 msgr2=0x7f5cc80b4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:07:22.674 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.673+0000 7f5ccd507640 1 --2- 172.21.15.167:0/521188716 >> v2:172.21.15.204:3300/0 conn(0x7f5cc8142740 0x7f5cc80b4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:22.674 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.673+0000 7f5ccd507640 1 -- 172.21.15.167:0/521188716 >> v1:172.21.15.204:6789/0 conn(0x7f5cc80b4ca0 legacy=0x7f5cc806fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:22.674 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.673+0000 7f5ccd507640 1 -- 172.21.15.167:0/521188716 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cc806f150 con 0x7f5cc8072150 2024-05-11T08:07:22.675 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.674+0000 7f5ccd507640 1 --2- 172.21.15.167:0/521188716 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5cc8072150 0x7f5cc8072590 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5cc0009e50 tx=0x7f5cc002a680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=55c435a7322d977e server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:22.675 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.674+0000 7f5cc77fe640 1 -- 172.21.15.167:0/521188716 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cc0036070 con 0x7f5cc8072150 2024-05-11T08:07:22.675 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.674+0000 7f5cc77fe640 1 -- 172.21.15.167:0/521188716 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5cc0031600 con 0x7f5cc8072150 2024-05-11T08:07:22.675 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.674+0000 7f5cc77fe640 1 -- 172.21.15.167:0/521188716 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cc00319d0 con 0x7f5cc8072150 2024-05-11T08:07:22.675 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.674+0000 7f5ccef6a640 1 -- 172.21.15.167:0/521188716 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5cc8072150 msgr2=0x7f5cc8072590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:22.675 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.674+0000 7f5ccef6a640 1 --2- 172.21.15.167:0/521188716 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5cc8072150 0x7f5cc8072590 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5cc0009e50 tx=0x7f5cc002a680 comp rx=0 tx=0).stop 2024-05-11T08:07:22.675 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.674+0000 7f5ccef6a640 1 -- 172.21.15.167:0/521188716 shutdown_connections 2024-05-11T08:07:22.675 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.674+0000 7f5ccef6a640 1 --2- 172.21.15.167:0/521188716 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5cc8072150 0x7f5cc8072590 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:22.675 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.674+0000 7f5ccef6a640 1 --2- 172.21.15.167:0/521188716 >> v2:172.21.15.204:3300/0 conn(0x7f5cc8142740 0x7f5cc80b4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:22.676 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.674+0000 7f5ccef6a640 1 -- 172.21.15.167:0/521188716 >> 172.21.15.167:0/521188716 conn(0x7f5cc80d0c70 msgr2=0x7f5cc80d1080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:07:22.676 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.674+0000 7f5ccef6a640 1 -- 172.21.15.167:0/521188716 shutdown_connections 2024-05-11T08:07:22.676 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.674+0000 7f5ccef6a640 1 -- 172.21.15.167:0/521188716 wait complete. 2024-05-11T08:07:22.678 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.676+0000 7f5ccef6a640 1 Processor -- start 2024-05-11T08:07:22.678 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.676+0000 7f5ccef6a640 1 -- start start 2024-05-11T08:07:22.678 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.676+0000 7f5ccef6a640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5cc8072150 0x7f5cc8151b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:22.678 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.677+0000 7f5ccef6a640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5cc80b4ca0 0x7f5cc81f6980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:22.678 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.677+0000 7f5ccef6a640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5cc8142740 0x7f5cc815e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:22.678 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.677+0000 7f5ccef6a640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5cc81609a0 con 0x7f5cc8142740 2024-05-11T08:07:22.678 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.677+0000 7f5ccef6a640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5cc8142b20 con 0x7f5cc80b4ca0 2024-05-11T08:07:22.679 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.677+0000 7f5ccef6a640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f5cc8160b20 con 0x7f5cc8072150 2024-05-11T08:07:22.679 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.677+0000 7f5ccd507640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5cc8142740 0x7f5cc815e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:22.679 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.677+0000 7f5ccd507640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5cc8142740 0x7f5cc815e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:56824/0 (socket says 172.21.15.167:56824) 2024-05-11T08:07:22.679 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.677+0000 7f5ccd507640 1 -- 172.21.15.167:0/4031144145 learned_addr learned my addr 172.21.15.167:0/4031144145 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:07:22.679 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.677+0000 7f5cccd06640 1 --2- 172.21.15.167:0/4031144145 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5cc8072150 0x7f5cc8151b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:22.679 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.677+0000 7f5cc7fff640 1 --2- 172.21.15.167:0/4031144145 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5cc80b4ca0 0x7f5cc81f6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:22.679 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.677+0000 7f5ccd507640 1 -- 172.21.15.167:0/4031144145 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5cc8072150 msgr2=0x7f5cc8151b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:22.679 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.677+0000 7f5ccd507640 1 --2- 172.21.15.167:0/4031144145 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5cc8072150 0x7f5cc8151b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:22.679 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.677+0000 7f5ccd507640 1 -- 172.21.15.167:0/4031144145 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5cc80b4ca0 msgr2=0x7f5cc81f6980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:22.679 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.677+0000 7f5ccd507640 1 --2- 172.21.15.167:0/4031144145 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5cc80b4ca0 0x7f5cc81f6980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:22.679 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.677+0000 7f5ccd507640 1 -- 172.21.15.167:0/4031144145 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cc81f8b70 con 0x7f5cc8142740 2024-05-11T08:07:22.680 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.677+0000 7f5cc7fff640 1 --2- 172.21.15.167:0/4031144145 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5cc80b4ca0 0x7f5cc81f6980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T08:07:22.680 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.677+0000 7f5ccd507640 1 --2- 172.21.15.167:0/4031144145 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5cc8142740 0x7f5cc815e810 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f5cc0034040 tx=0x7f5cc0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:22.680 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.677+0000 7f5cc57fa640 1 -- 172.21.15.167:0/4031144145 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cc0036050 con 0x7f5cc8142740 2024-05-11T08:07:22.680 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.677+0000 7f5ccef6a640 1 -- 172.21.15.167:0/4031144145 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cc815da50 con 0x7f5cc8142740 2024-05-11T08:07:22.680 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.677+0000 7f5cc57fa640 1 -- 172.21.15.167:0/4031144145 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5cc0042070 con 0x7f5cc8142740 2024-05-11T08:07:22.680 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.677+0000 7f5cc57fa640 1 -- 172.21.15.167:0/4031144145 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cc0009510 con 0x7f5cc8142740 2024-05-11T08:07:22.680 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.677+0000 7f5ccef6a640 1 -- 172.21.15.167:0/4031144145 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cc815e010 con 0x7f5cc8142740 2024-05-11T08:07:22.680 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.678+0000 7f5ccef6a640 1 -- 172.21.15.167:0/4031144145 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5cc815d800 con 0x7f5cc8142740 2024-05-11T08:07:22.681 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.679+0000 7f5cc57fa640 1 -- 172.21.15.167:0/4031144145 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f5cc0007bf0 con 0x7f5cc8142740 2024-05-11T08:07:22.681 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.679+0000 7f5cc57fa640 1 --2- 172.21.15.167:0/4031144145 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f5cb40762d0 0x7f5cb4078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:22.681 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.679+0000 7f5cc57fa640 1 -- 172.21.15.167:0/4031144145 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5335+0+0 (secure 0 0 0) 0x7f5cc00b8cd0 con 0x7f5cc8142740 2024-05-11T08:07:22.682 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.679+0000 7f5cccd06640 1 --2- 172.21.15.167:0/4031144145 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f5cb40762d0 0x7f5cb4078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:22.682 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.680+0000 7f5cc57fa640 1 --2- 172.21.15.167:0/4031144145 >> [v2:172.21.15.204:6824/2921847521,v1:172.21.15.204:6825/2921847521] conn(0x7f5cb407c770 0x7f5cb407ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:22.682 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.680+0000 7f5cc57fa640 1 -- 172.21.15.167:0/4031144145 --> [v2:172.21.15.204:6824/2921847521,v1:172.21.15.204:6825/2921847521] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5cb407f280 con 0x7f5cb407c770 2024-05-11T08:07:22.682 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.680+0000 7f5cc57fa640 1 -- 172.21.15.167:0/4031144145 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f5cc00b9160 con 0x7f5cc8142740 2024-05-11T08:07:22.682 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.680+0000 7f5cc7fff640 1 --2- 172.21.15.167:0/4031144145 >> [v2:172.21.15.204:6824/2921847521,v1:172.21.15.204:6825/2921847521] conn(0x7f5cb407c770 0x7f5cb407ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:22.682 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.680+0000 7f5cccd06640 1 --2- 172.21.15.167:0/4031144145 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f5cb40762d0 0x7f5cb4078790 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f5cb800b240 tx=0x7f5cb800b3a0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:22.682 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.680+0000 7f5cc7fff640 1 --2- 172.21.15.167:0/4031144145 >> [v2:172.21.15.204:6824/2921847521,v1:172.21.15.204:6825/2921847521] conn(0x7f5cb407c770 0x7f5cb407ebb0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:22.682 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.681+0000 7f5cc57fa640 1 -- 172.21.15.167:0/4031144145 <== osd.5 v2:172.21.15.204:6824/2921847521 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5cb407f280 con 0x7f5cb407c770 2024-05-11T08:07:22.694 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.694+0000 7f5ccef6a640 1 -- 172.21.15.167:0/4031144145 --> [v2:172.21.15.204:6824/2921847521,v1:172.21.15.204:6825/2921847521] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f5cc8069590 con 0x7f5cb407c770 2024-05-11T08:07:22.695 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.695+0000 7f5cc57fa640 1 -- 172.21.15.167:0/4031144145 <== osd.5 v2:172.21.15.204:6824/2921847521 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f5cc8069590 con 0x7f5cb407c770 2024-05-11T08:07:22.695 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-05-11T08:07:22.695 INFO:teuthology.orchestra.run.smithi167.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-11T08:07:22.695 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-05-11T08:07:22.696 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.695+0000 7f5ccef6a640 1 -- 172.21.15.167:0/4031144145 >> [v2:172.21.15.204:6824/2921847521,v1:172.21.15.204:6825/2921847521] conn(0x7f5cb407c770 msgr2=0x7f5cb407ebb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:22.696 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.695+0000 7f5ccef6a640 1 --2- 172.21.15.167:0/4031144145 >> [v2:172.21.15.204:6824/2921847521,v1:172.21.15.204:6825/2921847521] conn(0x7f5cb407c770 0x7f5cb407ebb0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:22.696 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.695+0000 7f5ccef6a640 1 -- 172.21.15.167:0/4031144145 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f5cb40762d0 msgr2=0x7f5cb4078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:22.696 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.695+0000 7f5ccef6a640 1 --2- 172.21.15.167:0/4031144145 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f5cb40762d0 0x7f5cb4078790 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f5cb800b240 tx=0x7f5cb800b3a0 comp rx=0 tx=0).stop 2024-05-11T08:07:22.696 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.695+0000 7f5ccef6a640 1 -- 172.21.15.167:0/4031144145 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5cc8142740 msgr2=0x7f5cc815e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:22.696 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.695+0000 7f5ccef6a640 1 --2- 172.21.15.167:0/4031144145 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5cc8142740 0x7f5cc815e810 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f5cc0034040 tx=0x7f5cc0004a40 comp rx=0 tx=0).stop 2024-05-11T08:07:22.696 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.695+0000 7f5ccd507640 1 -- 172.21.15.167:0/4031144145 reap_dead start 2024-05-11T08:07:22.697 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.695+0000 7f5ccef6a640 1 -- 172.21.15.167:0/4031144145 shutdown_connections 2024-05-11T08:07:22.697 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.695+0000 7f5ccef6a640 1 -- 172.21.15.167:0/4031144145 >> 172.21.15.167:0/4031144145 conn(0x7f5cc80d0c70 msgr2=0x7f5cc8073070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:07:22.697 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.695+0000 7f5ccef6a640 1 -- 172.21.15.167:0/4031144145 shutdown_connections 2024-05-11T08:07:22.697 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.695+0000 7f5ccef6a640 1 -- 172.21.15.167:0/4031144145 wait complete. 2024-05-11T08:07:22.706 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-05-11T08:07:22.807 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.806+0000 7ff156958640 1 Processor -- start 2024-05-11T08:07:22.807 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.806+0000 7ff156958640 1 -- start start 2024-05-11T08:07:22.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.807+0000 7ff156958640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff1500718e0 0x7ff150160e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:22.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.807+0000 7ff156958640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7ff150071cc0 0x7ff1500cd610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:22.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.807+0000 7ff156958640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7ff1500cdb50 0x7ff1501573a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:22.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.807+0000 7ff156958640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7ff1500d1d20 con 0x7ff150071cc0 2024-05-11T08:07:22.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.807+0000 7ff156958640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7ff150161390 con 0x7ff1500cdb50 2024-05-11T08:07:22.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.807+0000 7ff156958640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7ff1500d0ec0 con 0x7ff1500718e0 2024-05-11T08:07:22.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.807+0000 7ff147fff640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7ff150071cc0 0x7ff1500cd610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:22.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.808+0000 7ff147fff640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7ff150071cc0 0x7ff1500cd610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:56840/0 (socket says 172.21.15.167:56840) 2024-05-11T08:07:22.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.808+0000 7ff147fff640 1 -- 172.21.15.167:0/4243250758 learned_addr learned my addr 172.21.15.167:0/4243250758 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:07:22.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.808+0000 7ff154ef5640 1 --2- 172.21.15.167:0/4243250758 >> v2:172.21.15.204:3300/0 conn(0x7ff1500cdb50 0x7ff1501573a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:22.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.808+0000 7ff14ffff640 1 --2- 172.21.15.167:0/4243250758 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff1500718e0 0x7ff150160e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:22.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.808+0000 7ff147fff640 1 -- 172.21.15.167:0/4243250758 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff1500718e0 msgr2=0x7ff150160e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:07:22.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.808+0000 7ff147fff640 1 --2- 172.21.15.167:0/4243250758 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff1500718e0 0x7ff150160e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:22.810 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.808+0000 7ff147fff640 1 -- 172.21.15.167:0/4243250758 >> v2:172.21.15.204:3300/0 conn(0x7ff1500cdb50 msgr2=0x7ff1501573a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:07:22.810 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.808+0000 7ff147fff640 1 --2- 172.21.15.167:0/4243250758 >> v2:172.21.15.204:3300/0 conn(0x7ff1500cdb50 0x7ff1501573a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:22.810 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.808+0000 7ff147fff640 1 -- 172.21.15.167:0/4243250758 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1500d11a0 con 0x7ff150071cc0 2024-05-11T08:07:22.810 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.808+0000 7ff154ef5640 1 --2- 172.21.15.167:0/4243250758 >> v2:172.21.15.204:3300/0 conn(0x7ff1500cdb50 0x7ff1501573a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T08:07:22.810 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.808+0000 7ff14ffff640 1 --2- 172.21.15.167:0/4243250758 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff1500718e0 0x7ff150160e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T08:07:22.810 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.808+0000 7ff147fff640 1 --2- 172.21.15.167:0/4243250758 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff150071cc0 0x7ff1500cd610 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7ff1500ceb70 tx=0x7ff13802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1f0ee6d79aa5bfd server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:22.810 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.809+0000 7ff14f7fe640 1 -- 172.21.15.167:0/4243250758 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff138037070 con 0x7ff150071cc0 2024-05-11T08:07:22.810 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.809+0000 7ff14f7fe640 1 -- 172.21.15.167:0/4243250758 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff138032440 con 0x7ff150071cc0 2024-05-11T08:07:22.810 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.809+0000 7ff14f7fe640 1 -- 172.21.15.167:0/4243250758 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff138032810 con 0x7ff150071cc0 2024-05-11T08:07:22.813 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.809+0000 7ff156958640 1 -- 172.21.15.167:0/4243250758 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff150071cc0 msgr2=0x7ff1500cd610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:22.813 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.809+0000 7ff156958640 1 --2- 172.21.15.167:0/4243250758 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff150071cc0 0x7ff1500cd610 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7ff1500ceb70 tx=0x7ff13802a1e0 comp rx=0 tx=0).stop 2024-05-11T08:07:22.813 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.809+0000 7ff156958640 1 -- 172.21.15.167:0/4243250758 shutdown_connections 2024-05-11T08:07:22.813 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.809+0000 7ff156958640 1 --2- 172.21.15.167:0/4243250758 >> v2:172.21.15.204:3300/0 conn(0x7ff1500cdb50 0x7ff1501573a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:22.813 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.809+0000 7ff156958640 1 --2- 172.21.15.167:0/4243250758 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff150071cc0 0x7ff1500cd610 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:22.813 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.809+0000 7ff156958640 1 --2- 172.21.15.167:0/4243250758 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff1500718e0 0x7ff150160e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:22.814 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.809+0000 7ff156958640 1 -- 172.21.15.167:0/4243250758 >> 172.21.15.167:0/4243250758 conn(0x7ff150072730 msgr2=0x7ff1500d2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:07:22.814 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.809+0000 7ff156958640 1 -- 172.21.15.167:0/4243250758 shutdown_connections 2024-05-11T08:07:22.814 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.809+0000 7ff156958640 1 -- 172.21.15.167:0/4243250758 wait complete. 2024-05-11T08:07:22.814 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.810+0000 7ff156958640 1 Processor -- start 2024-05-11T08:07:22.814 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.810+0000 7ff156958640 1 -- start start 2024-05-11T08:07:22.814 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.811+0000 7ff156958640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff1500718e0 0x7ff1500c6af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:22.814 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.811+0000 7ff156958640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff150071cc0 0x7ff150152000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:22.814 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.811+0000 7ff156958640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff1500cdb50 0x7ff1500cc470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:22.814 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.811+0000 7ff156958640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff1500d1a40 con 0x7ff1500718e0 2024-05-11T08:07:22.814 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.811+0000 7ff156958640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff1500d1760 con 0x7ff150071cc0 2024-05-11T08:07:22.814 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.811+0000 7ff156958640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7ff1501578e0 con 0x7ff1500cdb50 2024-05-11T08:07:22.814 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.811+0000 7ff14ffff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff1500718e0 0x7ff1500c6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:22.814 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.811+0000 7ff14ffff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff1500718e0 0x7ff1500c6af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:56846/0 (socket says 172.21.15.167:56846) 2024-05-11T08:07:22.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.811+0000 7ff14ffff640 1 -- 172.21.15.167:0/1902870226 learned_addr learned my addr 172.21.15.167:0/1902870226 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:07:22.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.811+0000 7ff147fff640 1 --2- 172.21.15.167:0/1902870226 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff150071cc0 0x7ff150152000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:22.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.811+0000 7ff154ef5640 1 --2- 172.21.15.167:0/1902870226 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff1500cdb50 0x7ff1500cc470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:22.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.812+0000 7ff14ffff640 1 -- 172.21.15.167:0/1902870226 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff1500cdb50 msgr2=0x7ff1500cc470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:22.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.812+0000 7ff14ffff640 1 --2- 172.21.15.167:0/1902870226 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff1500cdb50 0x7ff1500cc470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:22.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.812+0000 7ff14ffff640 1 -- 172.21.15.167:0/1902870226 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff150071cc0 msgr2=0x7ff150152000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:22.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.812+0000 7ff14ffff640 1 --2- 172.21.15.167:0/1902870226 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff150071cc0 0x7ff150152000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:22.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.812+0000 7ff14ffff640 1 -- 172.21.15.167:0/1902870226 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1500c7030 con 0x7ff1500718e0 2024-05-11T08:07:22.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.812+0000 7ff154ef5640 1 --2- 172.21.15.167:0/1902870226 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff1500cdb50 0x7ff1500cc470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T08:07:22.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.812+0000 7ff14ffff640 1 --2- 172.21.15.167:0/1902870226 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff1500718e0 0x7ff1500c6af0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7ff14000c7f0 tx=0x7ff14000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:22.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.812+0000 7ff14d7fa640 1 -- 172.21.15.167:0/1902870226 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff140015070 con 0x7ff1500718e0 2024-05-11T08:07:22.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.812+0000 7ff14d7fa640 1 -- 172.21.15.167:0/1902870226 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1400079d0 con 0x7ff1500718e0 2024-05-11T08:07:22.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.812+0000 7ff156958640 1 -- 172.21.15.167:0/1902870226 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff150152540 con 0x7ff1500718e0 2024-05-11T08:07:22.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.812+0000 7ff14d7fa640 1 -- 172.21.15.167:0/1902870226 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff140007da0 con 0x7ff1500718e0 2024-05-11T08:07:22.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.812+0000 7ff156958640 1 -- 172.21.15.167:0/1902870226 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1500cb180 con 0x7ff1500718e0 2024-05-11T08:07:22.816 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.813+0000 7ff156958640 1 -- 172.21.15.167:0/1902870226 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff150069590 con 0x7ff1500718e0 2024-05-11T08:07:22.816 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.813+0000 7ff14d7fa640 1 -- 172.21.15.167:0/1902870226 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7ff140018070 con 0x7ff1500718e0 2024-05-11T08:07:22.816 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.814+0000 7ff14d7fa640 1 --2- 172.21.15.167:0/1902870226 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7ff1300762d0 0x7ff130078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:22.816 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.814+0000 7ff14d7fa640 1 -- 172.21.15.167:0/1902870226 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5335+0+0 (secure 0 0 0) 0x7ff140097350 con 0x7ff1500718e0 2024-05-11T08:07:22.817 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.814+0000 7ff147fff640 1 --2- 172.21.15.167:0/1902870226 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7ff1300762d0 0x7ff130078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:22.817 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.814+0000 7ff147fff640 1 --2- 172.21.15.167:0/1902870226 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7ff1300762d0 0x7ff130078790 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff138034000 tx=0x7ff13802e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:22.818 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.818+0000 7ff14d7fa640 1 -- 172.21.15.167:0/1902870226 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7ff140060e20 con 0x7ff1500718e0 2024-05-11T08:07:22.999 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.998+0000 7ff156958640 1 -- 172.21.15.167:0/1902870226 --> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7ff1500d25e0 con 0x7ff1300762d0 2024-05-11T08:07:22.999 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:22.999+0000 7ff14d7fa640 1 -- 172.21.15.167:0/1902870226 <== mgr.4113 v2:172.21.15.167:6832/253569496 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7ff1500d25e0 con 0x7ff1300762d0 2024-05-11T08:07:22.999 INFO:teuthology.orchestra.run.smithi167.stderr:instructed osd(s) 5 to deep-scrub 2024-05-11T08:07:23.001 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.001+0000 7ff156958640 1 -- 172.21.15.167:0/1902870226 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7ff1300762d0 msgr2=0x7ff130078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:23.001 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.001+0000 7ff156958640 1 --2- 172.21.15.167:0/1902870226 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7ff1300762d0 0x7ff130078790 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff138034000 tx=0x7ff13802e000 comp rx=0 tx=0).stop 2024-05-11T08:07:23.001 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.001+0000 7ff156958640 1 -- 172.21.15.167:0/1902870226 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff1500718e0 msgr2=0x7ff1500c6af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:23.001 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.001+0000 7ff156958640 1 --2- 172.21.15.167:0/1902870226 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff1500718e0 0x7ff1500c6af0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7ff14000c7f0 tx=0x7ff14000ccb0 comp rx=0 tx=0).stop 2024-05-11T08:07:23.002 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.001+0000 7ff156958640 1 -- 172.21.15.167:0/1902870226 shutdown_connections 2024-05-11T08:07:23.002 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.001+0000 7ff156958640 1 --2- 172.21.15.167:0/1902870226 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7ff1300762d0 0x7ff130078790 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:23.002 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.001+0000 7ff156958640 1 --2- 172.21.15.167:0/1902870226 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff1500cdb50 0x7ff1500cc470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:23.002 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.001+0000 7ff156958640 1 --2- 172.21.15.167:0/1902870226 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff150071cc0 0x7ff150152000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:23.002 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.001+0000 7ff156958640 1 --2- 172.21.15.167:0/1902870226 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff1500718e0 0x7ff1500c6af0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:23.002 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.001+0000 7ff156958640 1 -- 172.21.15.167:0/1902870226 >> 172.21.15.167:0/1902870226 conn(0x7ff150072730 msgr2=0x7ff1500b29e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:07:23.002 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.001+0000 7ff156958640 1 -- 172.21.15.167:0/1902870226 shutdown_connections 2024-05-11T08:07:23.002 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.001+0000 7ff156958640 1 -- 172.21.15.167:0/1902870226 wait complete. 2024-05-11T08:07:23.012 INFO:tasks.ceph:Scrubbing osd.6 2024-05-11T08:07:23.012 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-05-11T08:07:23.116 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.116+0000 7fecfcc83640 1 Processor -- start 2024-05-11T08:07:23.117 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.116+0000 7fecfcc83640 1 -- start start 2024-05-11T08:07:23.118 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.117+0000 7fecfcc83640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fecf80d44b0 0x7fecf806cb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:23.118 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.117+0000 7fecfcc83640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fecf8163420 con 0x7fecf80b4460 2024-05-11T08:07:23.118 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.117+0000 7fecfcc83640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fecf8163700 con 0x7fecf81628c0 2024-05-11T08:07:23.118 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.117+0000 7fecfcc83640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7fecf8164030 con 0x7fecf80d44b0 2024-05-11T08:07:23.118 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.118+0000 7fecf5d9b640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fecf80d44b0 0x7fecf806cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:23.118 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.118+0000 7fecf6d9d640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7fecf80b4460 0x7fecf806d0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:59786/0 (socket says 172.21.15.167:59786) 2024-05-11T08:07:23.118 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.118+0000 7fecf6d9d640 1 -- 172.21.15.167:0/2064776791 learned_addr learned my addr 172.21.15.167:0/2064776791 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T08:07:23.118 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.118+0000 7fecf5d9b640 1 -- 172.21.15.167:0/2064776791 >> v1:172.21.15.204:6789/0 conn(0x7fecf81628c0 legacy=0x7fecf80d3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:07:23.119 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.118+0000 7fecf5d9b640 1 -- 172.21.15.167:0/2064776791 >> v1:172.21.15.167:6789/0 conn(0x7fecf80b4460 legacy=0x7fecf806d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:23.119 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.118+0000 7fecf5d9b640 1 -- 172.21.15.167:0/2064776791 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecf81639e0 con 0x7fecf80d44b0 2024-05-11T08:07:23.119 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.118+0000 7fecf559a640 1 -- 172.21.15.167:0/2064776791 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2738854095 0 0) 0x7fecf8163420 con 0x7fecf80b4460 2024-05-11T08:07:23.120 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.119+0000 7fecf5d9b640 1 --2- 172.21.15.167:0/2064776791 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fecf80d44b0 0x7fecf806cb60 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fecec0097a0 tx=0x7fecec02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6898c44be89d5581 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:23.120 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.119+0000 7fecf559a640 1 -- 172.21.15.167:0/2064776791 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecec037070 con 0x7fecf80d44b0 2024-05-11T08:07:23.120 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.119+0000 7fecf559a640 1 -- 172.21.15.167:0/2064776791 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fecec032440 con 0x7fecf80d44b0 2024-05-11T08:07:23.120 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.119+0000 7fecf559a640 1 -- 172.21.15.167:0/2064776791 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecec032810 con 0x7fecf80d44b0 2024-05-11T08:07:23.120 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.119+0000 7fecfcc83640 1 -- 172.21.15.167:0/2064776791 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fecf80d44b0 msgr2=0x7fecf806cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:23.120 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.119+0000 7fecfcc83640 1 --2- 172.21.15.167:0/2064776791 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fecf80d44b0 0x7fecf806cb60 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fecec0097a0 tx=0x7fecec02a180 comp rx=0 tx=0).stop 2024-05-11T08:07:23.120 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.119+0000 7fecfcc83640 1 -- 172.21.15.167:0/2064776791 shutdown_connections 2024-05-11T08:07:23.120 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.119+0000 7fecfcc83640 1 --2- 172.21.15.167:0/2064776791 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fecf80d44b0 0x7fecf806cb60 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:23.120 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.119+0000 7fecfcc83640 1 -- 172.21.15.167:0/2064776791 >> 172.21.15.167:0/2064776791 conn(0x7fecf80b6cb0 msgr2=0x7fecf80b7090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:07:23.120 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.119+0000 7fecfcc83640 1 -- 172.21.15.167:0/2064776791 shutdown_connections 2024-05-11T08:07:23.121 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.119+0000 7fecfcc83640 1 -- 172.21.15.167:0/2064776791 wait complete. 2024-05-11T08:07:23.123 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.121+0000 7fecfcc83640 1 Processor -- start 2024-05-11T08:07:23.123 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.121+0000 7fecfcc83640 1 -- start start 2024-05-11T08:07:23.123 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.121+0000 7fecfcc83640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fecf80b4460 0x7fecf80ce050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:23.123 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.121+0000 7fecfcc83640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fecf80d44b0 0x7fecf81f4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:23.123 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.121+0000 7fecfcc83640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fecf81628c0 0x7fecf8151180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:23.123 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.121+0000 7fecfcc83640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fecf8162ca0 con 0x7fecf80d44b0 2024-05-11T08:07:23.123 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.121+0000 7fecfcc83640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fecf81630d0 con 0x7fecf81628c0 2024-05-11T08:07:23.123 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.121+0000 7fecfcc83640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fecf8162e20 con 0x7fecf80b4460 2024-05-11T08:07:23.123 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.122+0000 7fecf5d9b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fecf80d44b0 0x7fecf81f4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:23.123 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.122+0000 7fecf5d9b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fecf80d44b0 0x7fecf81f4ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:56870/0 (socket says 172.21.15.167:56870) 2024-05-11T08:07:23.123 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.122+0000 7fecf5d9b640 1 -- 172.21.15.167:0/2485853667 learned_addr learned my addr 172.21.15.167:0/2485853667 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:07:23.124 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.122+0000 7fecf6d9d640 1 --2- 172.21.15.167:0/2485853667 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fecf81628c0 0x7fecf8151180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:23.124 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.122+0000 7fecf659c640 1 --2- 172.21.15.167:0/2485853667 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fecf80b4460 0x7fecf80ce050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:23.124 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.122+0000 7fecf5d9b640 1 -- 172.21.15.167:0/2485853667 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fecf80b4460 msgr2=0x7fecf80ce050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:23.124 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.122+0000 7fecf5d9b640 1 --2- 172.21.15.167:0/2485853667 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fecf80b4460 0x7fecf80ce050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:23.124 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.122+0000 7fecf5d9b640 1 -- 172.21.15.167:0/2485853667 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fecf81628c0 msgr2=0x7fecf8151180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:23.124 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.122+0000 7fecf5d9b640 1 --2- 172.21.15.167:0/2485853667 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fecf81628c0 0x7fecf8151180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:23.124 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.122+0000 7fecf5d9b640 1 -- 172.21.15.167:0/2485853667 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecf81f5930 con 0x7fecf80d44b0 2024-05-11T08:07:23.124 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.122+0000 7fecf659c640 1 --2- 172.21.15.167:0/2485853667 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fecf80b4460 0x7fecf80ce050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T08:07:23.124 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.122+0000 7fecf5d9b640 1 --2- 172.21.15.167:0/2485853667 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fecf80d44b0 0x7fecf81f4ee0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fecec0097a0 tx=0x7fecec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:23.124 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.122+0000 7fece6ffd640 1 -- 172.21.15.167:0/2485853667 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecec041070 con 0x7fecf80d44b0 2024-05-11T08:07:23.124 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.122+0000 7fece6ffd640 1 -- 172.21.15.167:0/2485853667 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fecec03c870 con 0x7fecf80d44b0 2024-05-11T08:07:23.124 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.122+0000 7fecfcc83640 1 -- 172.21.15.167:0/2485853667 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecf81f5c10 con 0x7fecf80d44b0 2024-05-11T08:07:23.124 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.122+0000 7fece6ffd640 1 -- 172.21.15.167:0/2485853667 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecec037040 con 0x7fecf80d44b0 2024-05-11T08:07:23.124 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.122+0000 7fecfcc83640 1 -- 172.21.15.167:0/2485853667 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecf81f61d0 con 0x7fecf80d44b0 2024-05-11T08:07:23.126 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.123+0000 7fecfcc83640 1 -- 172.21.15.167:0/2485853667 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fecf8069590 con 0x7fecf80d44b0 2024-05-11T08:07:23.126 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.124+0000 7fece6ffd640 1 -- 172.21.15.167:0/2485853667 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fecec044070 con 0x7fecf80d44b0 2024-05-11T08:07:23.126 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.124+0000 7fece6ffd640 1 --2- 172.21.15.167:0/2485853667 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fecd8076300 0x7fecd80787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:23.126 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.124+0000 7fecf659c640 1 --2- 172.21.15.167:0/2485853667 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fecd8076300 0x7fecd80787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:23.126 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.124+0000 7fece6ffd640 1 -- 172.21.15.167:0/2485853667 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5335+0+0 (secure 0 0 0) 0x7fecec0b80b0 con 0x7fecf80d44b0 2024-05-11T08:07:23.126 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.125+0000 7fece6ffd640 1 --2- 172.21.15.167:0/2485853667 >> [v2:172.21.15.204:6816/2889851077,v1:172.21.15.204:6817/2889851077] conn(0x7fecd807c7a0 0x7fecd807ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:23.126 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.125+0000 7fece6ffd640 1 -- 172.21.15.167:0/2485853667 --> [v2:172.21.15.204:6816/2889851077,v1:172.21.15.204:6817/2889851077] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fecec041e70 con 0x7fecd807c7a0 2024-05-11T08:07:23.126 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.125+0000 7fece6ffd640 1 -- 172.21.15.167:0/2485853667 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fecec0bf050 con 0x7fecf80d44b0 2024-05-11T08:07:23.126 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.125+0000 7fecf659c640 1 --2- 172.21.15.167:0/2485853667 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fecd8076300 0x7fecd80787c0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fece800a830 tx=0x7fece8008040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:23.126 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.125+0000 7fecf6d9d640 1 --2- 172.21.15.167:0/2485853667 >> [v2:172.21.15.204:6816/2889851077,v1:172.21.15.204:6817/2889851077] conn(0x7fecd807c7a0 0x7fecd807ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:23.126 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.125+0000 7fecf6d9d640 1 --2- 172.21.15.167:0/2485853667 >> [v2:172.21.15.204:6816/2889851077,v1:172.21.15.204:6817/2889851077] conn(0x7fecd807c7a0 0x7fecd807ebe0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:23.127 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.126+0000 7fece6ffd640 1 -- 172.21.15.167:0/2485853667 <== osd.6 v2:172.21.15.204:6816/2889851077 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fecec041e70 con 0x7fecd807c7a0 2024-05-11T08:07:23.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.139+0000 7fecfcc83640 1 -- 172.21.15.167:0/2485853667 --> [v2:172.21.15.204:6816/2889851077,v1:172.21.15.204:6817/2889851077] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fecf80d4a80 con 0x7fecd807c7a0 2024-05-11T08:07:23.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.139+0000 7fece6ffd640 1 -- 172.21.15.167:0/2485853667 <== osd.6 v2:172.21.15.204:6816/2889851077 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fecf80d4a80 con 0x7fecd807c7a0 2024-05-11T08:07:23.139 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-05-11T08:07:23.139 INFO:teuthology.orchestra.run.smithi167.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-11T08:07:23.140 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-05-11T08:07:23.140 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.139+0000 7fecfcc83640 1 -- 172.21.15.167:0/2485853667 >> [v2:172.21.15.204:6816/2889851077,v1:172.21.15.204:6817/2889851077] conn(0x7fecd807c7a0 msgr2=0x7fecd807ebe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:23.140 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.139+0000 7fecfcc83640 1 --2- 172.21.15.167:0/2485853667 >> [v2:172.21.15.204:6816/2889851077,v1:172.21.15.204:6817/2889851077] conn(0x7fecd807c7a0 0x7fecd807ebe0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:23.140 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.140+0000 7fecfcc83640 1 -- 172.21.15.167:0/2485853667 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fecd8076300 msgr2=0x7fecd80787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:23.141 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.140+0000 7fecfcc83640 1 --2- 172.21.15.167:0/2485853667 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fecd8076300 0x7fecd80787c0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fece800a830 tx=0x7fece8008040 comp rx=0 tx=0).stop 2024-05-11T08:07:23.141 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.140+0000 7fecfcc83640 1 -- 172.21.15.167:0/2485853667 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fecf80d44b0 msgr2=0x7fecf81f4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:23.141 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.140+0000 7fecfcc83640 1 --2- 172.21.15.167:0/2485853667 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fecf80d44b0 0x7fecf81f4ee0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fecec0097a0 tx=0x7fecec0046c0 comp rx=0 tx=0).stop 2024-05-11T08:07:23.141 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.140+0000 7fecf6d9d640 1 -- 172.21.15.167:0/2485853667 reap_dead start 2024-05-11T08:07:23.141 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.140+0000 7fecfcc83640 1 -- 172.21.15.167:0/2485853667 shutdown_connections 2024-05-11T08:07:23.141 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.140+0000 7fecfcc83640 1 -- 172.21.15.167:0/2485853667 >> 172.21.15.167:0/2485853667 conn(0x7fecf80b6cb0 msgr2=0x7fecf80b6190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:07:23.141 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.140+0000 7fecfcc83640 1 -- 172.21.15.167:0/2485853667 shutdown_connections 2024-05-11T08:07:23.141 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.140+0000 7fecfcc83640 1 -- 172.21.15.167:0/2485853667 wait complete. 2024-05-11T08:07:23.150 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-05-11T08:07:23.252 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.251+0000 7f6f8b4e8640 1 Processor -- start 2024-05-11T08:07:23.252 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.252+0000 7f6f8b4e8640 1 -- start start 2024-05-11T08:07:23.252 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.252+0000 7f6f8b4e8640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f6f840b74c0 0x7f6f840b7900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:23.252 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.252+0000 7f6f8b4e8640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6f840cd580 0x7f6f840cda10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:23.252 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.252+0000 7f6f8b4e8640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6f840d27c0 con 0x7f6f8415f220 2024-05-11T08:07:23.252 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.252+0000 7f6f8b4e8640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f6f8415f600 con 0x7f6f840b74c0 2024-05-11T08:07:23.253 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.252+0000 7f6f8b4e8640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f6f840cf450 con 0x7f6f840cd580 2024-05-11T08:07:23.254 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.253+0000 7f6f89a85640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6f840cd580 0x7f6f840cda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:23.254 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.253+0000 7f6f88a83640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f6f840b74c0 0x7f6f840b7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:23.254 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.253+0000 7f6f89a85640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6f840cd580 0x7f6f840cda10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.167:48800/0 (socket says 172.21.15.167:48800) 2024-05-11T08:07:23.254 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.253+0000 7f6f89284640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f6f8415f220 0x7f6f840d1510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.167:58244/0 (socket says 172.21.15.167:58244) 2024-05-11T08:07:23.254 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.253+0000 7f6f89a85640 1 -- 172.21.15.167:0/411127593 learned_addr learned my addr 172.21.15.167:0/411127593 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:07:23.254 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.253+0000 7f6f89a85640 1 -- 172.21.15.167:0/411127593 >> v2:172.21.15.204:3300/0 conn(0x7f6f840b74c0 msgr2=0x7f6f840b7900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:07:23.254 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.253+0000 7f6f89a85640 1 --2- 172.21.15.167:0/411127593 >> v2:172.21.15.204:3300/0 conn(0x7f6f840b74c0 0x7f6f840b7900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:23.255 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.253+0000 7f6f89a85640 1 -- 172.21.15.167:0/411127593 >> v1:172.21.15.204:6789/0 conn(0x7f6f8415f220 legacy=0x7f6f840d1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:23.258 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.253+0000 7f6f89a85640 1 -- 172.21.15.167:0/411127593 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f840cee00 con 0x7f6f840cd580 2024-05-11T08:07:23.258 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.253+0000 7f6f88a83640 1 --2- 172.21.15.167:0/411127593 >> v2:172.21.15.204:3300/0 conn(0x7f6f840b74c0 0x7f6f840b7900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T08:07:23.258 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.254+0000 7f6f89a85640 1 --2- 172.21.15.167:0/411127593 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6f840cd580 0x7f6f840cda10 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6f7c009e50 tx=0x7f6f7c02a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=672eb3d639882ff1 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:23.258 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.254+0000 7f6f77fff640 1 -- 172.21.15.167:0/411127593 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f7c036070 con 0x7f6f840cd580 2024-05-11T08:07:23.258 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.254+0000 7f6f77fff640 1 -- 172.21.15.167:0/411127593 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f7c031600 con 0x7f6f840cd580 2024-05-11T08:07:23.258 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.254+0000 7f6f77fff640 1 -- 172.21.15.167:0/411127593 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f7c0319d0 con 0x7f6f840cd580 2024-05-11T08:07:23.261 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.255+0000 7f6f8b4e8640 1 -- 172.21.15.167:0/411127593 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6f840cd580 msgr2=0x7f6f840cda10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:23.261 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.255+0000 7f6f8b4e8640 1 --2- 172.21.15.167:0/411127593 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6f840cd580 0x7f6f840cda10 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6f7c009e50 tx=0x7f6f7c02a710 comp rx=0 tx=0).stop 2024-05-11T08:07:23.261 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.255+0000 7f6f8b4e8640 1 -- 172.21.15.167:0/411127593 shutdown_connections 2024-05-11T08:07:23.262 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.255+0000 7f6f8b4e8640 1 --2- 172.21.15.167:0/411127593 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6f840cd580 0x7f6f840cda10 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:23.262 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.255+0000 7f6f8b4e8640 1 --2- 172.21.15.167:0/411127593 >> v2:172.21.15.204:3300/0 conn(0x7f6f840b74c0 0x7f6f840b7900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:23.262 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.255+0000 7f6f8b4e8640 1 -- 172.21.15.167:0/411127593 >> 172.21.15.167:0/411127593 conn(0x7f6f840b4ae0 msgr2=0x7f6f840b4ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:07:23.262 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.255+0000 7f6f8b4e8640 1 -- 172.21.15.167:0/411127593 shutdown_connections 2024-05-11T08:07:23.262 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.255+0000 7f6f8b4e8640 1 -- 172.21.15.167:0/411127593 wait complete. 2024-05-11T08:07:23.262 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.256+0000 7f6f8b4e8640 1 Processor -- start 2024-05-11T08:07:23.262 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.256+0000 7f6f8b4e8640 1 -- start start 2024-05-11T08:07:23.262 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.257+0000 7f6f8b4e8640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6f840b74c0 0x7f6f840cb510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:23.262 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.257+0000 7f6f8b4e8640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6f840cd580 0x7f6f840cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:23.262 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.257+0000 7f6f8b4e8640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6f8415f220 0x7f6f84150f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:23.262 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.257+0000 7f6f8b4e8640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6f840d2400 con 0x7f6f8415f220 2024-05-11T08:07:23.262 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.257+0000 7f6f8b4e8640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6f840ce2e0 con 0x7f6f840cd580 2024-05-11T08:07:23.262 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.257+0000 7f6f8b4e8640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f6f840d2580 con 0x7f6f840b74c0 2024-05-11T08:07:23.263 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.257+0000 7f6f89a85640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6f8415f220 0x7f6f84150f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:23.263 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.257+0000 7f6f89a85640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6f8415f220 0x7f6f84150f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:56876/0 (socket says 172.21.15.167:56876) 2024-05-11T08:07:23.263 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.257+0000 7f6f89a85640 1 -- 172.21.15.167:0/3326805006 learned_addr learned my addr 172.21.15.167:0/3326805006 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:07:23.263 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.257+0000 7f6f89284640 1 --2- 172.21.15.167:0/3326805006 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6f840b74c0 0x7f6f840cb510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:23.263 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.257+0000 7f6f88a83640 1 --2- 172.21.15.167:0/3326805006 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6f840cd580 0x7f6f840cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:23.263 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.257+0000 7f6f89a85640 1 -- 172.21.15.167:0/3326805006 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6f840b74c0 msgr2=0x7f6f840cb510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:23.263 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.257+0000 7f6f89a85640 1 --2- 172.21.15.167:0/3326805006 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6f840b74c0 0x7f6f840cb510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:23.263 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.257+0000 7f6f89a85640 1 -- 172.21.15.167:0/3326805006 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6f840cd580 msgr2=0x7f6f840cc8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:23.263 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.257+0000 7f6f89a85640 1 --2- 172.21.15.167:0/3326805006 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6f840cd580 0x7f6f840cc8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:23.263 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.257+0000 7f6f89a85640 1 -- 172.21.15.167:0/3326805006 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f840c8240 con 0x7f6f8415f220 2024-05-11T08:07:23.263 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.257+0000 7f6f89a85640 1 --2- 172.21.15.167:0/3326805006 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6f8415f220 0x7f6f84150f80 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f6f7c034040 tx=0x7f6f7c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:23.263 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.257+0000 7f6f75ffb640 1 -- 172.21.15.167:0/3326805006 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f7c036050 con 0x7f6f8415f220 2024-05-11T08:07:23.263 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.257+0000 7f6f75ffb640 1 -- 172.21.15.167:0/3326805006 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f7c042070 con 0x7f6f8415f220 2024-05-11T08:07:23.263 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.257+0000 7f6f8b4e8640 1 -- 172.21.15.167:0/3326805006 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f840c84b0 con 0x7f6f8415f220 2024-05-11T08:07:23.264 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.257+0000 7f6f75ffb640 1 -- 172.21.15.167:0/3326805006 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f7c009510 con 0x7f6f8415f220 2024-05-11T08:07:23.264 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.258+0000 7f6f8b4e8640 1 -- 172.21.15.167:0/3326805006 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f840c8d50 con 0x7f6f8415f220 2024-05-11T08:07:23.264 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.259+0000 7f6f8b4e8640 1 -- 172.21.15.167:0/3326805006 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f840c88a0 con 0x7f6f8415f220 2024-05-11T08:07:23.264 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.259+0000 7f6f75ffb640 1 -- 172.21.15.167:0/3326805006 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f6f7c007bf0 con 0x7f6f8415f220 2024-05-11T08:07:23.264 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.260+0000 7f6f75ffb640 1 --2- 172.21.15.167:0/3326805006 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f6f680762d0 0x7f6f68078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:23.264 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.260+0000 7f6f75ffb640 1 -- 172.21.15.167:0/3326805006 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5335+0+0 (secure 0 0 0) 0x7f6f7c0b80b0 con 0x7f6f8415f220 2024-05-11T08:07:23.264 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.260+0000 7f6f89284640 1 --2- 172.21.15.167:0/3326805006 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f6f680762d0 0x7f6f68078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:23.264 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.260+0000 7f6f89284640 1 --2- 172.21.15.167:0/3326805006 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f6f680762d0 0x7f6f68078790 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f6f78000f60 tx=0x7f6f78000bc0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:23.264 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.264+0000 7f6f75ffb640 1 -- 172.21.15.167:0/3326805006 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f6f7c0815c0 con 0x7f6f8415f220 2024-05-11T08:07:23.448 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.448+0000 7f6f8b4e8640 1 -- 172.21.15.167:0/3326805006 --> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7f6f8415ea20 con 0x7f6f680762d0 2024-05-11T08:07:23.448 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.448+0000 7f6f75ffb640 1 -- 172.21.15.167:0/3326805006 <== mgr.4113 v2:172.21.15.167:6832/253569496 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f6f8415ea20 con 0x7f6f680762d0 2024-05-11T08:07:23.448 INFO:teuthology.orchestra.run.smithi167.stderr:instructed osd(s) 6 to deep-scrub 2024-05-11T08:07:23.451 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.450+0000 7f6f8b4e8640 1 -- 172.21.15.167:0/3326805006 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f6f680762d0 msgr2=0x7f6f68078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:23.451 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.450+0000 7f6f8b4e8640 1 --2- 172.21.15.167:0/3326805006 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f6f680762d0 0x7f6f68078790 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f6f78000f60 tx=0x7f6f78000bc0 comp rx=0 tx=0).stop 2024-05-11T08:07:23.451 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.450+0000 7f6f8b4e8640 1 -- 172.21.15.167:0/3326805006 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6f8415f220 msgr2=0x7f6f84150f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:23.451 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.450+0000 7f6f8b4e8640 1 --2- 172.21.15.167:0/3326805006 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6f8415f220 0x7f6f84150f80 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f6f7c034040 tx=0x7f6f7c004a40 comp rx=0 tx=0).stop 2024-05-11T08:07:23.451 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.450+0000 7f6f8b4e8640 1 -- 172.21.15.167:0/3326805006 shutdown_connections 2024-05-11T08:07:23.451 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.450+0000 7f6f8b4e8640 1 --2- 172.21.15.167:0/3326805006 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f6f680762d0 0x7f6f68078790 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:23.451 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.450+0000 7f6f8b4e8640 1 --2- 172.21.15.167:0/3326805006 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6f8415f220 0x7f6f84150f80 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:23.451 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.450+0000 7f6f8b4e8640 1 --2- 172.21.15.167:0/3326805006 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6f840cd580 0x7f6f840cc8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:23.451 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.450+0000 7f6f8b4e8640 1 --2- 172.21.15.167:0/3326805006 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6f840b74c0 0x7f6f840cb510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:23.451 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.450+0000 7f6f8b4e8640 1 -- 172.21.15.167:0/3326805006 >> 172.21.15.167:0/3326805006 conn(0x7f6f840b4ae0 msgr2=0x7f6f84154a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:07:23.451 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.450+0000 7f6f8b4e8640 1 -- 172.21.15.167:0/3326805006 shutdown_connections 2024-05-11T08:07:23.452 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.450+0000 7f6f8b4e8640 1 -- 172.21.15.167:0/3326805006 wait complete. 2024-05-11T08:07:23.462 INFO:tasks.ceph:Scrubbing osd.7 2024-05-11T08:07:23.462 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-05-11T08:07:23.566 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.566+0000 7f394fbc1640 1 Processor -- start 2024-05-11T08:07:23.567 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.567+0000 7f394fbc1640 1 -- start start 2024-05-11T08:07:23.568 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.567+0000 7f394fbc1640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f39480cd680 0x7f3948070a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:23.568 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.567+0000 7f394fbc1640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3948160e20 con 0x7f394815cc80 2024-05-11T08:07:23.568 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.567+0000 7f394fbc1640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3948161100 con 0x7f39480d2980 2024-05-11T08:07:23.568 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.567+0000 7f394fbc1640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f3948161a30 con 0x7f39480cd680 2024-05-11T08:07:23.568 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.567+0000 7f394e15e640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f39480cd680 0x7f3948070a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:23.568 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.567+0000 7f394d95d640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f394815cc80 0x7f39480d2270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:59800/0 (socket says 172.21.15.167:59800) 2024-05-11T08:07:23.568 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.568+0000 7f394d95d640 1 -- 172.21.15.167:0/1542031891 learned_addr learned my addr 172.21.15.167:0/1542031891 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T08:07:23.568 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.568+0000 7f394e15e640 1 -- 172.21.15.167:0/1542031891 >> v1:172.21.15.204:6789/0 conn(0x7f39480d2980 legacy=0x7f394815d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:23.568 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.568+0000 7f394e15e640 1 -- 172.21.15.167:0/1542031891 >> v1:172.21.15.167:6789/0 conn(0x7f394815cc80 legacy=0x7f39480d2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:23.568 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.568+0000 7f394e15e640 1 -- 172.21.15.167:0/1542031891 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39481613e0 con 0x7f39480cd680 2024-05-11T08:07:23.570 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.568+0000 7f394e15e640 1 --2- 172.21.15.167:0/1542031891 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f39480cd680 0x7f3948070a10 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f3948161bb0 tx=0x7f394002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c284a0f198b57253 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:23.570 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.569+0000 7f394c95b640 1 -- 172.21.15.167:0/1542031891 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3940037070 con 0x7f39480cd680 2024-05-11T08:07:23.570 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.569+0000 7f394c95b640 1 -- 172.21.15.167:0/1542031891 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3940032440 con 0x7f39480cd680 2024-05-11T08:07:23.570 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.569+0000 7f394c95b640 1 -- 172.21.15.167:0/1542031891 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3940032810 con 0x7f39480cd680 2024-05-11T08:07:23.570 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.569+0000 7f394fbc1640 1 -- 172.21.15.167:0/1542031891 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f39480cd680 msgr2=0x7f3948070a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:23.570 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.569+0000 7f394fbc1640 1 --2- 172.21.15.167:0/1542031891 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f39480cd680 0x7f3948070a10 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f3948161bb0 tx=0x7f394002a210 comp rx=0 tx=0).stop 2024-05-11T08:07:23.570 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.569+0000 7f394fbc1640 1 -- 172.21.15.167:0/1542031891 shutdown_connections 2024-05-11T08:07:23.570 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.569+0000 7f394fbc1640 1 --2- 172.21.15.167:0/1542031891 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f39480cd680 0x7f3948070a10 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:23.570 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.569+0000 7f394fbc1640 1 -- 172.21.15.167:0/1542031891 >> 172.21.15.167:0/1542031891 conn(0x7f394806ca80 msgr2=0x7f39480b4bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:07:23.570 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.569+0000 7f394fbc1640 1 -- 172.21.15.167:0/1542031891 shutdown_connections 2024-05-11T08:07:23.570 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.569+0000 7f394fbc1640 1 -- 172.21.15.167:0/1542031891 wait complete. 2024-05-11T08:07:23.572 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.570+0000 7f394fbc1640 1 Processor -- start 2024-05-11T08:07:23.572 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.570+0000 7f394fbc1640 1 -- start start 2024-05-11T08:07:23.572 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.571+0000 7f394fbc1640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39480cd680 0x7f394815c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:23.572 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.571+0000 7f394fbc1640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f39480d2980 0x7f394815b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:23.572 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.571+0000 7f394fbc1640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f394815cc80 0x7f39480b8120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:23.573 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.571+0000 7f394fbc1640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f394815d1e0 con 0x7f394815cc80 2024-05-11T08:07:23.573 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.571+0000 7f394fbc1640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f394815d060 con 0x7f39480cd680 2024-05-11T08:07:23.573 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.571+0000 7f394fbc1640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f394815d360 con 0x7f39480d2980 2024-05-11T08:07:23.573 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.571+0000 7f394e15e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f394815cc80 0x7f39480b8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:23.573 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.571+0000 7f394e15e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f394815cc80 0x7f39480b8120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:56894/0 (socket says 172.21.15.167:56894) 2024-05-11T08:07:23.573 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.571+0000 7f394e15e640 1 -- 172.21.15.167:0/69035464 learned_addr learned my addr 172.21.15.167:0/69035464 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:07:23.573 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.572+0000 7f394e15e640 1 -- 172.21.15.167:0/69035464 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f39480d2980 msgr2=0x7f394815b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:23.573 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.572+0000 7f394e15e640 1 --2- 172.21.15.167:0/69035464 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f39480d2980 0x7f394815b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:23.573 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.572+0000 7f394e15e640 1 -- 172.21.15.167:0/69035464 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39480cd680 msgr2=0x7f394815c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:23.573 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.572+0000 7f394e15e640 1 --2- 172.21.15.167:0/69035464 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39480cd680 0x7f394815c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:23.573 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.572+0000 7f394e15e640 1 -- 172.21.15.167:0/69035464 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f394815b8a0 con 0x7f394815cc80 2024-05-11T08:07:23.573 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.572+0000 7f394e15e640 1 --2- 172.21.15.167:0/69035464 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f394815cc80 0x7f39480b8120 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f3940035040 tx=0x7f39400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:23.573 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.572+0000 7f393e7fc640 1 -- 172.21.15.167:0/69035464 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3940037040 con 0x7f394815cc80 2024-05-11T08:07:23.573 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.572+0000 7f393e7fc640 1 -- 172.21.15.167:0/69035464 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3940042070 con 0x7f394815cc80 2024-05-11T08:07:23.574 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.572+0000 7f394fbc1640 1 -- 172.21.15.167:0/69035464 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f394815bb80 con 0x7f394815cc80 2024-05-11T08:07:23.574 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.572+0000 7f393e7fc640 1 -- 172.21.15.167:0/69035464 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3940030b30 con 0x7f394815cc80 2024-05-11T08:07:23.574 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.572+0000 7f394fbc1640 1 -- 172.21.15.167:0/69035464 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39480c89f0 con 0x7f394815cc80 2024-05-11T08:07:23.575 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.573+0000 7f394fbc1640 1 -- 172.21.15.167:0/69035464 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f39480b8b00 con 0x7f394815cc80 2024-05-11T08:07:23.575 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.574+0000 7f393e7fc640 1 -- 172.21.15.167:0/69035464 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f3940030cd0 con 0x7f394815cc80 2024-05-11T08:07:23.575 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.574+0000 7f393e7fc640 1 --2- 172.21.15.167:0/69035464 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f392c0762d0 0x7f392c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:23.575 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.574+0000 7f394d95d640 1 --2- 172.21.15.167:0/69035464 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f392c0762d0 0x7f392c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:23.575 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.574+0000 7f393e7fc640 1 -- 172.21.15.167:0/69035464 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5335+0+0 (secure 0 0 0) 0x7f39400b7fc0 con 0x7f394815cc80 2024-05-11T08:07:23.575 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.574+0000 7f393e7fc640 1 --2- 172.21.15.167:0/69035464 >> [v2:172.21.15.204:6808/1949534184,v1:172.21.15.204:6809/1949534184] conn(0x7f392c07c770 0x7f392c07ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:23.575 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.574+0000 7f393e7fc640 1 -- 172.21.15.167:0/69035464 --> [v2:172.21.15.204:6808/1949534184,v1:172.21.15.204:6809/1949534184] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f394004bec0 con 0x7f392c07c770 2024-05-11T08:07:23.575 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.574+0000 7f393e7fc640 1 -- 172.21.15.167:0/69035464 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f39400bf050 con 0x7f394815cc80 2024-05-11T08:07:23.576 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.575+0000 7f394d95d640 1 --2- 172.21.15.167:0/69035464 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f392c0762d0 0x7f392c078790 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f3944005c00 tx=0x7f39440016a0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:23.576 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.575+0000 7f394d15c640 1 --2- 172.21.15.167:0/69035464 >> [v2:172.21.15.204:6808/1949534184,v1:172.21.15.204:6809/1949534184] conn(0x7f392c07c770 0x7f392c07ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:23.576 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.575+0000 7f394d15c640 1 --2- 172.21.15.167:0/69035464 >> [v2:172.21.15.204:6808/1949534184,v1:172.21.15.204:6809/1949534184] conn(0x7f392c07c770 0x7f392c07ebb0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:23.576 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.576+0000 7f393e7fc640 1 -- 172.21.15.167:0/69035464 <== osd.7 v2:172.21.15.204:6808/1949534184 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f394004bec0 con 0x7f392c07c770 2024-05-11T08:07:23.588 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.588+0000 7f394fbc1640 1 -- 172.21.15.167:0/69035464 --> [v2:172.21.15.204:6808/1949534184,v1:172.21.15.204:6809/1949534184] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f394815b640 con 0x7f392c07c770 2024-05-11T08:07:23.589 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.589+0000 7f393e7fc640 1 -- 172.21.15.167:0/69035464 <== osd.7 v2:172.21.15.204:6808/1949534184 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f39380050b0 con 0x7f392c07c770 2024-05-11T08:07:23.589 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-05-11T08:07:23.589 INFO:teuthology.orchestra.run.smithi167.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-11T08:07:23.589 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-05-11T08:07:23.590 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.589+0000 7f394fbc1640 1 -- 172.21.15.167:0/69035464 >> [v2:172.21.15.204:6808/1949534184,v1:172.21.15.204:6809/1949534184] conn(0x7f392c07c770 msgr2=0x7f392c07ebb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:23.590 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.589+0000 7f394fbc1640 1 --2- 172.21.15.167:0/69035464 >> [v2:172.21.15.204:6808/1949534184,v1:172.21.15.204:6809/1949534184] conn(0x7f392c07c770 0x7f392c07ebb0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:23.590 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.589+0000 7f394fbc1640 1 -- 172.21.15.167:0/69035464 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f392c0762d0 msgr2=0x7f392c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:23.590 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.589+0000 7f394fbc1640 1 --2- 172.21.15.167:0/69035464 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f392c0762d0 0x7f392c078790 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f3944005c00 tx=0x7f39440016a0 comp rx=0 tx=0).stop 2024-05-11T08:07:23.591 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.589+0000 7f394fbc1640 1 -- 172.21.15.167:0/69035464 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f394815cc80 msgr2=0x7f39480b8120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:23.591 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.589+0000 7f394fbc1640 1 --2- 172.21.15.167:0/69035464 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f394815cc80 0x7f39480b8120 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f3940035040 tx=0x7f39400046c0 comp rx=0 tx=0).stop 2024-05-11T08:07:23.591 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.589+0000 7f394e15e640 1 -- 172.21.15.167:0/69035464 reap_dead start 2024-05-11T08:07:23.591 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.589+0000 7f394fbc1640 1 -- 172.21.15.167:0/69035464 shutdown_connections 2024-05-11T08:07:23.591 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.589+0000 7f394fbc1640 1 -- 172.21.15.167:0/69035464 >> 172.21.15.167:0/69035464 conn(0x7f394806ca80 msgr2=0x7f39480b4b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:07:23.591 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.589+0000 7f394fbc1640 1 -- 172.21.15.167:0/69035464 shutdown_connections 2024-05-11T08:07:23.591 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.590+0000 7f394fbc1640 1 -- 172.21.15.167:0/69035464 wait complete. 2024-05-11T08:07:23.600 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-05-11T08:07:23.702 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.702+0000 7f9dd7672640 1 Processor -- start 2024-05-11T08:07:23.702 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.702+0000 7f9dd7672640 1 -- start start 2024-05-11T08:07:23.704 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.703+0000 7f9dd7672640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9dd00718e0 0x7f9dd0160e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:23.704 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.703+0000 7f9dd7672640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f9dd01573a0 0x7f9dd0157810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:23.704 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.703+0000 7f9dd7672640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9dd0161390 con 0x7f9dd0071cc0 2024-05-11T08:07:23.704 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.703+0000 7f9dd7672640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f9dd00d1d20 con 0x7f9dd01573a0 2024-05-11T08:07:23.704 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.703+0000 7f9dd7672640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f9dd00d0ec0 con 0x7f9dd00718e0 2024-05-11T08:07:23.704 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.703+0000 7f9dd5c0f640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f9dd01573a0 0x7f9dd0157810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:23.705 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.703+0000 7f9dd5c0f640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f9dd01573a0 0x7f9dd0157810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:56908/0 (socket says 172.21.15.167:56908) 2024-05-11T08:07:23.705 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.703+0000 7f9dd5c0f640 1 -- 172.21.15.167:0/2988808913 learned_addr learned my addr 172.21.15.167:0/2988808913 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:07:23.705 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.703+0000 7f9dd540e640 1 --2- 172.21.15.167:0/2988808913 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9dd00718e0 0x7f9dd0160e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:23.705 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.703+0000 7f9dd5c0f640 1 -- 172.21.15.167:0/2988808913 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9dd00718e0 msgr2=0x7f9dd0160e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:07:23.705 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.703+0000 7f9dd5c0f640 1 --2- 172.21.15.167:0/2988808913 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9dd00718e0 0x7f9dd0160e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:23.705 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.703+0000 7f9dd5c0f640 1 -- 172.21.15.167:0/2988808913 >> v1:172.21.15.204:6789/0 conn(0x7f9dd0071cc0 legacy=0x7f9dd00cd610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:07:23.705 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.703+0000 7f9dd5c0f640 1 -- 172.21.15.167:0/2988808913 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dd0157d50 con 0x7f9dd01573a0 2024-05-11T08:07:23.705 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.703+0000 7f9dd540e640 1 --2- 172.21.15.167:0/2988808913 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9dd00718e0 0x7f9dd0160e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T08:07:23.705 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.704+0000 7f9dd5c0f640 1 --2- 172.21.15.167:0/2988808913 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9dd01573a0 0x7f9dd0157810 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f9dc8009e50 tx=0x7f9dc802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9404b66bb2b86298 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:23.705 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.704+0000 7f9dc7fff640 1 -- 172.21.15.167:0/2988808913 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dc8036070 con 0x7f9dd01573a0 2024-05-11T08:07:23.705 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.704+0000 7f9dc7fff640 1 -- 172.21.15.167:0/2988808913 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9dc8031600 con 0x7f9dd01573a0 2024-05-11T08:07:23.705 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.704+0000 7f9dc7fff640 1 -- 172.21.15.167:0/2988808913 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dc80319d0 con 0x7f9dd01573a0 2024-05-11T08:07:23.706 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.705+0000 7f9dd7672640 1 -- 172.21.15.167:0/2988808913 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9dd01573a0 msgr2=0x7f9dd0157810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:23.706 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.705+0000 7f9dd7672640 1 --2- 172.21.15.167:0/2988808913 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9dd01573a0 0x7f9dd0157810 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f9dc8009e50 tx=0x7f9dc802a680 comp rx=0 tx=0).stop 2024-05-11T08:07:23.707 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.705+0000 7f9dd7672640 1 -- 172.21.15.167:0/2988808913 shutdown_connections 2024-05-11T08:07:23.707 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.705+0000 7f9dd7672640 1 --2- 172.21.15.167:0/2988808913 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9dd01573a0 0x7f9dd0157810 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:23.707 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.705+0000 7f9dd7672640 1 --2- 172.21.15.167:0/2988808913 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9dd00718e0 0x7f9dd0160e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:23.707 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.705+0000 7f9dd7672640 1 -- 172.21.15.167:0/2988808913 >> 172.21.15.167:0/2988808913 conn(0x7f9dd0072730 msgr2=0x7f9dd00d2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:07:23.707 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.705+0000 7f9dd7672640 1 -- 172.21.15.167:0/2988808913 shutdown_connections 2024-05-11T08:07:23.707 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.705+0000 7f9dd7672640 1 -- 172.21.15.167:0/2988808913 wait complete. 2024-05-11T08:07:23.707 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.706+0000 7f9dd7672640 1 Processor -- start 2024-05-11T08:07:23.707 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.706+0000 7f9dd7672640 1 -- start start 2024-05-11T08:07:23.709 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.707+0000 7f9dd7672640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9dd00718e0 0x7f9dd00c56b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:23.709 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.707+0000 7f9dd7672640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9dd0071cc0 0x7f9dd015db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:23.709 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.707+0000 7f9dd7672640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9dd01573a0 0x7f9dd00b28b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:23.709 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.707+0000 7f9dd7672640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9dd00d1760 con 0x7f9dd00718e0 2024-05-11T08:07:23.709 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.707+0000 7f9dd7672640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f9dd00d1480 con 0x7f9dd0071cc0 2024-05-11T08:07:23.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.707+0000 7f9dd7672640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f9dd00d1a40 con 0x7f9dd01573a0 2024-05-11T08:07:23.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.707+0000 7f9dd540e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9dd00718e0 0x7f9dd00c56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:23.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.707+0000 7f9dd540e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9dd00718e0 0x7f9dd00c56b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:56918/0 (socket says 172.21.15.167:56918) 2024-05-11T08:07:23.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.707+0000 7f9dd540e640 1 -- 172.21.15.167:0/853656501 learned_addr learned my addr 172.21.15.167:0/853656501 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:07:23.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.707+0000 7f9dd4c0d640 1 --2- 172.21.15.167:0/853656501 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9dd0071cc0 0x7f9dd015db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:23.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.707+0000 7f9dd5c0f640 1 --2- 172.21.15.167:0/853656501 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9dd01573a0 0x7f9dd00b28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:23.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.707+0000 7f9dd540e640 1 -- 172.21.15.167:0/853656501 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9dd01573a0 msgr2=0x7f9dd00b28b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:23.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.707+0000 7f9dd540e640 1 --2- 172.21.15.167:0/853656501 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9dd01573a0 0x7f9dd00b28b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:23.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.707+0000 7f9dd540e640 1 -- 172.21.15.167:0/853656501 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9dd0071cc0 msgr2=0x7f9dd015db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:23.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.707+0000 7f9dd540e640 1 --2- 172.21.15.167:0/853656501 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9dd0071cc0 0x7f9dd015db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:23.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.707+0000 7f9dd540e640 1 -- 172.21.15.167:0/853656501 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dd015e080 con 0x7f9dd00718e0 2024-05-11T08:07:23.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.707+0000 7f9dd4c0d640 1 --2- 172.21.15.167:0/853656501 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9dd0071cc0 0x7f9dd015db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T08:07:23.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.708+0000 7f9dd540e640 1 --2- 172.21.15.167:0/853656501 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9dd00718e0 0x7f9dd00c56b0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc00c820 tx=0x7f9dcc00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:23.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.708+0000 7f9dc5ffb640 1 -- 172.21.15.167:0/853656501 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dcc015070 con 0x7f9dd00718e0 2024-05-11T08:07:23.711 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.708+0000 7f9dc5ffb640 1 -- 172.21.15.167:0/853656501 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9dcc007a30 con 0x7f9dd00718e0 2024-05-11T08:07:23.711 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.708+0000 7f9dd7672640 1 -- 172.21.15.167:0/853656501 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9dd0149050 con 0x7f9dd00718e0 2024-05-11T08:07:23.711 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.708+0000 7f9dc5ffb640 1 -- 172.21.15.167:0/853656501 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dcc007de0 con 0x7f9dd00718e0 2024-05-11T08:07:23.711 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.708+0000 7f9dd7672640 1 -- 172.21.15.167:0/853656501 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9dd015fac0 con 0x7f9dd00718e0 2024-05-11T08:07:23.711 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.709+0000 7f9dd7672640 1 -- 172.21.15.167:0/853656501 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9dd0069590 con 0x7f9dd00718e0 2024-05-11T08:07:23.711 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.709+0000 7f9dc5ffb640 1 -- 172.21.15.167:0/853656501 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f9dcc018070 con 0x7f9dd00718e0 2024-05-11T08:07:23.712 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.710+0000 7f9dc5ffb640 1 --2- 172.21.15.167:0/853656501 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f9db4076300 0x7f9db40787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:23.712 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.710+0000 7f9dc5ffb640 1 -- 172.21.15.167:0/853656501 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5335+0+0 (secure 0 0 0) 0x7f9dcc0972c0 con 0x7f9dd00718e0 2024-05-11T08:07:23.712 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.710+0000 7f9dd4c0d640 1 --2- 172.21.15.167:0/853656501 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f9db4076300 0x7f9db40787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:23.712 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.710+0000 7f9dd4c0d640 1 --2- 172.21.15.167:0/853656501 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f9db4076300 0x7f9db40787c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f9dd014c950 tx=0x7f9dc0009040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:23.714 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.714+0000 7f9dc5ffb640 1 -- 172.21.15.167:0/853656501 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f9dcc060e20 con 0x7f9dd00718e0 2024-05-11T08:07:23.896 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.896+0000 7f9dd7672640 1 -- 172.21.15.167:0/853656501 --> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7f9dd00d25e0 con 0x7f9db4076300 2024-05-11T08:07:23.897 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.896+0000 7f9dc5ffb640 1 -- 172.21.15.167:0/853656501 <== mgr.4113 v2:172.21.15.167:6832/253569496 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f9dd00d25e0 con 0x7f9db4076300 2024-05-11T08:07:23.897 INFO:teuthology.orchestra.run.smithi167.stderr:instructed osd(s) 7 to deep-scrub 2024-05-11T08:07:23.899 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.899+0000 7f9dd7672640 1 -- 172.21.15.167:0/853656501 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f9db4076300 msgr2=0x7f9db40787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:23.899 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.899+0000 7f9dd7672640 1 --2- 172.21.15.167:0/853656501 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f9db4076300 0x7f9db40787c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f9dd014c950 tx=0x7f9dc0009040 comp rx=0 tx=0).stop 2024-05-11T08:07:23.899 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.899+0000 7f9dd7672640 1 -- 172.21.15.167:0/853656501 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9dd00718e0 msgr2=0x7f9dd00c56b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:23.899 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.899+0000 7f9dd7672640 1 --2- 172.21.15.167:0/853656501 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9dd00718e0 0x7f9dd00c56b0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc00c820 tx=0x7f9dcc00cce0 comp rx=0 tx=0).stop 2024-05-11T08:07:23.899 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.899+0000 7f9dd7672640 1 -- 172.21.15.167:0/853656501 shutdown_connections 2024-05-11T08:07:23.899 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.899+0000 7f9dd7672640 1 --2- 172.21.15.167:0/853656501 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f9db4076300 0x7f9db40787c0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:23.900 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.899+0000 7f9dd7672640 1 --2- 172.21.15.167:0/853656501 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9dd01573a0 0x7f9dd00b28b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:23.900 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.899+0000 7f9dd7672640 1 --2- 172.21.15.167:0/853656501 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9dd0071cc0 0x7f9dd015db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:23.900 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.899+0000 7f9dd7672640 1 --2- 172.21.15.167:0/853656501 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9dd00718e0 0x7f9dd00c56b0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:23.900 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.899+0000 7f9dd7672640 1 -- 172.21.15.167:0/853656501 >> 172.21.15.167:0/853656501 conn(0x7f9dd0072730 msgr2=0x7f9dd00c8b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:07:23.900 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.899+0000 7f9dd7672640 1 -- 172.21.15.167:0/853656501 shutdown_connections 2024-05-11T08:07:23.900 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:23.899+0000 7f9dd7672640 1 -- 172.21.15.167:0/853656501 wait complete. 2024-05-11T08:07:23.910 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-11T08:07:24.010 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:24.009+0000 7f37e5ef8640 1 Processor -- start 2024-05-11T08:07:24.010 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:24.010+0000 7f37e5ef8640 1 -- start start 2024-05-11T08:07:24.011 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:24.010+0000 7f37e5ef8640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f37e0145ed0 0x7f37e006e600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:24.012 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:24.010+0000 7f37e5ef8640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f37e006eb40 0x7f37e0070730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:24.012 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:24.010+0000 7f37e5ef8640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37e00714f0 con 0x7f37e0070c70 2024-05-11T08:07:24.012 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:24.010+0000 7f37e5ef8640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7f37e01462b0 con 0x7f37e0145ed0 2024-05-11T08:07:24.012 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:24.010+0000 7f37e5ef8640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f37e006efa0 con 0x7f37e006eb40 2024-05-11T08:07:24.012 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:24.010+0000 7f37df7fe640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f37e0145ed0 0x7f37e006e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:24.012 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:24.010+0000 7f37df7fe640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f37e0145ed0 0x7f37e006e600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:56934/0 (socket says 172.21.15.167:56934) 2024-05-11T08:07:24.012 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:24.010+0000 7f37df7fe640 1 -- 172.21.15.167:0/2092778810 learned_addr learned my addr 172.21.15.167:0/2092778810 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:07:24.012 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:24.011+0000 7f37d6ffd640 1 --2- 172.21.15.167:0/2092778810 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f37e006eb40 0x7f37e0070730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:24.012 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:24.011+0000 7f37df7fe640 1 -- 172.21.15.167:0/2092778810 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f37e006eb40 msgr2=0x7f37e0070730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:07:24.012 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:24.011+0000 7f37df7fe640 1 --2- 172.21.15.167:0/2092778810 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f37e006eb40 0x7f37e0070730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:24.012 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:24.011+0000 7f37df7fe640 1 -- 172.21.15.167:0/2092778810 >> v1:172.21.15.204:6789/0 conn(0x7f37e0070c70 legacy=0x7f37e0160b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:07:24.012 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:24.011+0000 7f37df7fe640 1 -- 172.21.15.167:0/2092778810 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37e00717d0 con 0x7f37e0145ed0 2024-05-11T08:07:24.013 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:24.011+0000 7f37d6ffd640 1 --2- 172.21.15.167:0/2092778810 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f37e006eb40 0x7f37e0070730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T08:07:24.013 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:24.011+0000 7f37df7fe640 1 --2- 172.21.15.167:0/2092778810 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f37e0145ed0 0x7f37e006e600 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f37d0009870 tx=0x7f37d002a0a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c071eedbee2cb3d6 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:24.013 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:24.012+0000 7f37deffd640 1 -- 172.21.15.167:0/2092778810 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37d0037070 con 0x7f37e0145ed0 2024-05-11T08:07:24.013 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:24.012+0000 7f37deffd640 1 -- 172.21.15.167:0/2092778810 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37d002adf0 con 0x7f37e0145ed0 2024-05-11T08:07:24.013 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:24.012+0000 7f37deffd640 1 -- 172.21.15.167:0/2092778810 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37d0032660 con 0x7f37e0145ed0 2024-05-11T08:07:24.015 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:24.012+0000 7f37e5ef8640 1 -- 172.21.15.167:0/2092778810 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f37e0145ed0 msgr2=0x7f37e006e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:24.015 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:24.012+0000 7f37e5ef8640 1 --2- 172.21.15.167:0/2092778810 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f37e0145ed0 0x7f37e006e600 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f37d0009870 tx=0x7f37d002a0a0 comp rx=0 tx=0).stop 2024-05-11T08:07:24.015 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:24.012+0000 7f37e5ef8640 1 -- 172.21.15.167:0/2092778810 shutdown_connections 2024-05-11T08:07:24.016 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:24.012+0000 7f37e5ef8640 1 --2- 172.21.15.167:0/2092778810 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f37e006eb40 0x7f37e0070730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:24.016 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:24.012+0000 7f37e5ef8640 1 --2- 172.21.15.167:0/2092778810 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f37e0145ed0 0x7f37e006e600 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:24.016 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:24.012+0000 7f37e5ef8640 1 -- 172.21.15.167:0/2092778810 >> 172.21.15.167:0/2092778810 conn(0x7f37e00d1e40 msgr2=0x7f37e00d2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:07:24.016 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:24.012+0000 7f37e5ef8640 1 -- 172.21.15.167:0/2092778810 shutdown_connections 2024-05-11T08:07:24.016 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:24.012+0000 7f37e5ef8640 1 -- 172.21.15.167:0/2092778810 wait complete. 2024-05-11T08:07:24.016 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:24.013+0000 7f37e5ef8640 1 Processor -- start 2024-05-11T08:07:24.016 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:24.014+0000 7f37e5ef8640 1 -- start start 2024-05-11T08:07:24.016 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:24.014+0000 7f37e5ef8640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f37e006eb40 0x7f37e00b5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:24.016 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:24.014+0000 7f37e5ef8640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f37e0070c70 0x7f37e015c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:24.016 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:24.014+0000 7f37e5ef8640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f37e0145ed0 0x7f37e01e6980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:24.016 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:24.014+0000 7f37e5ef8640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f37e006cfc0 con 0x7f37e0070c70 2024-05-11T08:07:24.016 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:24.014+0000 7f37e5ef8640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f37e006cce0 con 0x7f37e0145ed0 2024-05-11T08:07:24.016 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:24.014+0000 7f37e5ef8640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f37e0071100 con 0x7f37e006eb40 2024-05-11T08:07:24.016 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:24.014+0000 7f37d6ffd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f37e0070c70 0x7f37e015c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:24.016 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:24.015+0000 7f37d6ffd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f37e0070c70 0x7f37e015c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:56944/0 (socket says 172.21.15.167:56944) 2024-05-11T08:07:24.017 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:24.015+0000 7f37d6ffd640 1 -- 172.21.15.167:0/2257130351 learned_addr learned my addr 172.21.15.167:0/2257130351 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:07:24.017 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:24.015+0000 7f37df7fe640 1 --2- 172.21.15.167:0/2257130351 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f37e006eb40 0x7f37e00b5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:24.017 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:24.015+0000 7f37dffff640 1 --2- 172.21.15.167:0/2257130351 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f37e0145ed0 0x7f37e01e6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:24.017 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:24.015+0000 7f37d6ffd640 1 -- 172.21.15.167:0/2257130351 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f37e006eb40 msgr2=0x7f37e00b5e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:24.017 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:24.015+0000 7f37d6ffd640 1 --2- 172.21.15.167:0/2257130351 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f37e006eb40 0x7f37e00b5e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:24.017 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:24.015+0000 7f37d6ffd640 1 -- 172.21.15.167:0/2257130351 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f37e0145ed0 msgr2=0x7f37e01e6980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:24.017 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:24.015+0000 7f37d6ffd640 1 --2- 172.21.15.167:0/2257130351 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f37e0145ed0 0x7f37e01e6980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:24.017 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:24.015+0000 7f37d6ffd640 1 -- 172.21.15.167:0/2257130351 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37e00b6480 con 0x7f37e0070c70 2024-05-11T08:07:24.017 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:24.015+0000 7f37df7fe640 1 --2- 172.21.15.167:0/2257130351 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f37e006eb40 0x7f37e00b5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T08:07:24.017 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:24.015+0000 7f37dffff640 1 --2- 172.21.15.167:0/2257130351 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f37e0145ed0 0x7f37e01e6980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T08:07:24.017 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:24.015+0000 7f37d6ffd640 1 --2- 172.21.15.167:0/2257130351 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f37e0070c70 0x7f37e015c800 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f37c800c7f0 tx=0x7f37c800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:24.017 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:24.015+0000 7f37dcff9640 1 -- 172.21.15.167:0/2257130351 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37c8015070 con 0x7f37e0070c70 2024-05-11T08:07:24.017 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:24.015+0000 7f37dcff9640 1 -- 172.21.15.167:0/2257130351 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37c8007a30 con 0x7f37e0070c70 2024-05-11T08:07:24.017 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:24.015+0000 7f37dcff9640 1 -- 172.21.15.167:0/2257130351 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37c8007e00 con 0x7f37e0070c70 2024-05-11T08:07:24.018 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:24.015+0000 7f37e5ef8640 1 -- 172.21.15.167:0/2257130351 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37e015e610 con 0x7f37e0070c70 2024-05-11T08:07:24.018 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:24.015+0000 7f37e5ef8640 1 -- 172.21.15.167:0/2257130351 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37e015ebd0 con 0x7f37e0070c70 2024-05-11T08:07:24.019 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:24.016+0000 7f37e5ef8640 1 -- 172.21.15.167:0/2257130351 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37e00b1920 con 0x7f37e0070c70 2024-05-11T08:07:24.019 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:24.017+0000 7f37dcff9640 1 -- 172.21.15.167:0/2257130351 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f37c8018070 con 0x7f37e0070c70 2024-05-11T08:07:24.019 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:24.017+0000 7f37dcff9640 1 --2- 172.21.15.167:0/2257130351 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f37c00762d0 0x7f37c0078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:24.019 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:24.018+0000 7f37df7fe640 1 --2- 172.21.15.167:0/2257130351 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f37c00762d0 0x7f37c0078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:24.019 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:24.018+0000 7f37dcff9640 1 -- 172.21.15.167:0/2257130351 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5335+0+0 (secure 0 0 0) 0x7f37c80973b0 con 0x7f37e0070c70 2024-05-11T08:07:24.019 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:24.018+0000 7f37df7fe640 1 --2- 172.21.15.167:0/2257130351 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f37c00762d0 0x7f37c0078790 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f37d0034000 tx=0x7f37d002e000 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:24.022 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:24.022+0000 7f37dcff9640 1 -- 172.21.15.167:0/2257130351 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f37e00b1920 con 0x7f37e0070c70 2024-05-11T08:07:24.204 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:24.204+0000 7f37e5ef8640 1 -- 172.21.15.167:0/2257130351 --> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f37e0069bc0 con 0x7f37c00762d0 2024-05-11T08:07:24.205 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:24.205+0000 7f37dcff9640 1 -- 172.21.15.167:0/2257130351 <== mgr.4113 v2:172.21.15.167:6832/253569496 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13344 (secure 0 0 0) 0x7f37e0069bc0 con 0x7f37c00762d0 2024-05-11T08:07:24.205 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T08:07:24.205 INFO:teuthology.orchestra.run.smithi167.stderr:dumped all 2024-05-11T08:07:24.207 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:24.207+0000 7f37e5ef8640 1 -- 172.21.15.167:0/2257130351 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f37c00762d0 msgr2=0x7f37c0078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:24.207 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:24.207+0000 7f37e5ef8640 1 --2- 172.21.15.167:0/2257130351 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f37c00762d0 0x7f37c0078790 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f37d0034000 tx=0x7f37d002e000 comp rx=0 tx=0).stop 2024-05-11T08:07:24.208 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:24.207+0000 7f37e5ef8640 1 -- 172.21.15.167:0/2257130351 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f37e0070c70 msgr2=0x7f37e015c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:24.208 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:24.207+0000 7f37e5ef8640 1 --2- 172.21.15.167:0/2257130351 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f37e0070c70 0x7f37e015c800 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f37c800c7f0 tx=0x7f37c800cbf0 comp rx=0 tx=0).stop 2024-05-11T08:07:24.208 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:24.207+0000 7f37e5ef8640 1 -- 172.21.15.167:0/2257130351 shutdown_connections 2024-05-11T08:07:24.208 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:24.207+0000 7f37e5ef8640 1 --2- 172.21.15.167:0/2257130351 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f37c00762d0 0x7f37c0078790 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:24.208 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:24.207+0000 7f37e5ef8640 1 --2- 172.21.15.167:0/2257130351 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f37e0145ed0 0x7f37e01e6980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:24.208 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:24.207+0000 7f37e5ef8640 1 --2- 172.21.15.167:0/2257130351 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f37e0070c70 0x7f37e015c800 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:24.208 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:24.207+0000 7f37e5ef8640 1 --2- 172.21.15.167:0/2257130351 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f37e006eb40 0x7f37e00b5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:24.208 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:24.207+0000 7f37e5ef8640 1 -- 172.21.15.167:0/2257130351 >> 172.21.15.167:0/2257130351 conn(0x7f37e00d1e40 msgr2=0x7f37e0160ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:07:24.208 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:24.207+0000 7f37e5ef8640 1 -- 172.21.15.167:0/2257130351 shutdown_connections 2024-05-11T08:07:24.208 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:24.207+0000 7f37e5ef8640 1 -- 172.21.15.167:0/2257130351 wait complete. 2024-05-11T08:07:24.218 INFO:teuthology.orchestra.run.smithi167.stdout:{"pg_ready":true,"pg_map":{"version":93,"stamp":"2024-05-11T08:07:23.933958+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":225916,"kb_used_data":5576,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754748804,"statfs":{"total":773094113280,"available":772862775296,"internally_reserved":0,"allocated":5709824,"data_stored":3160976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":225496625},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":-1,"num_object_clones":0,"num_object_copies":-2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-1,"num_whiteouts":0,"num_read":-4,"num_read_kb":-4,"num_write":-6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.784128"},"pg_stats":[{"pgid":"1.0","version":"18'72","reported_seq":160,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-11T08:07:16.358660+0000","last_change":"2024-05-11T08:05:45.772368+0000","last_active":"2024-05-11T08:07:16.358660+0000","last_peered":"2024-05-11T08:07:16.358660+0000","last_clean":"2024-05-11T08:07:16.358660+0000","last_became_active":"2024-05-11T08:05:45.771373+0000","last_became_peered":"2024-05-11T08:05:45.771373+0000","last_unstale":"2024-05-11T08:07:16.358660+0000","last_undegraded":"2024-05-11T08:07:16.358660+0000","last_fullsized":"2024-05-11T08:07:16.358660+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T08:05:44.616617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T08:05:44.616617+0000","last_clean_scrub_stamp":"2024-05-11T08:05:44.616617+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:02:08.078748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509464,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28152,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343688,"statfs":{"total":96636764160,"available":96607936512,"internally_reserved":0,"allocated":1159168,"data_stored":837898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509464,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27572,"kb_used_data":552,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344268,"statfs":{"total":96636764160,"available":96608530432,"internally_reserved":0,"allocated":565248,"data_stored":247530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509464,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27572,"kb_used_data":552,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344268,"statfs":{"total":96636764160,"available":96608530432,"internally_reserved":0,"allocated":565248,"data_stored":247530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509464,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27576,"kb_used_data":552,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344264,"statfs":{"total":96636764160,"available":96608526336,"internally_reserved":0,"allocated":565248,"data_stored":247530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509465,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27576,"kb_used_data":552,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344264,"statfs":{"total":96636764160,"available":96608526336,"internally_reserved":0,"allocated":565248,"data_stored":247530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509465,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27576,"kb_used_data":552,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344264,"statfs":{"total":96636764160,"available":96608526336,"internally_reserved":0,"allocated":565248,"data_stored":247530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509464,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28156,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343684,"statfs":{"total":96636764160,"available":96607932416,"internally_reserved":0,"allocated":1159168,"data_stored":837898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574872,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31736,"kb_used_data":552,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340104,"statfs":{"total":96636764160,"available":96604266496,"internally_reserved":0,"allocated":565248,"data_stored":247530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-11T08:07:24.221 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-05-11T08:05:44.616617+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=8, tm_min=5, tm_sec=44, tm_wday=5, tm_yday=132, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=8, tm_min=7, tm_sec=19, tm_wday=5, tm_yday=132, tm_isdst=0) 2024-05-11T08:07:24.221 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-05-11T08:07:26.079 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-11T08:07:26.082 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-11T08:07:44.222 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-11T08:07:44.338 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.337+0000 7f494fd40640 1 Processor -- start 2024-05-11T08:07:44.338 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.338+0000 7f494fd40640 1 -- start start 2024-05-11T08:07:44.338 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.339+0000 7f494fd40640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f49480d2cf0 0x7f49480cafb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:44.338 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.339+0000 7f494fd40640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f49480cb4f0 0x7f494806dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:44.339 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.339+0000 7f494fd40640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49480d1190 con 0x7f49481598d0 2024-05-11T08:07:44.339 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.339+0000 7f494fd40640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 -- 0x7f49480cb920 con 0x7f49480d2cf0 2024-05-11T08:07:44.339 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.339+0000 7f494fd40640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f494806d500 con 0x7f49480cb4f0 2024-05-11T08:07:44.340 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.339+0000 7f494e2dd640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f49480cb4f0 0x7f494806dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:44.340 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.339+0000 7f494d2db640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f49480d2cf0 0x7f49480cafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:44.340 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.339+0000 7f494e2dd640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f49480cb4f0 0x7f494806dfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.167:60936/0 (socket says 172.21.15.167:60936) 2024-05-11T08:07:44.340 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.339+0000 7f494dadc640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f49481598d0 0x7f49480d0640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.167:40010/0 (socket says 172.21.15.167:40010) 2024-05-11T08:07:44.340 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.339+0000 7f494e2dd640 1 -- 172.21.15.167:0/4049905309 learned_addr learned my addr 172.21.15.167:0/4049905309 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:07:44.340 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.339+0000 7f494cada640 1 -- 172.21.15.167:0/4049905309 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1364969639 0 0) 0x7f49480d1190 con 0x7f49481598d0 2024-05-11T08:07:44.340 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.339+0000 7f494cada640 1 -- 172.21.15.167:0/4049905309 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4934003520 con 0x7f49481598d0 2024-05-11T08:07:44.340 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.339+0000 7f494e2dd640 1 -- 172.21.15.167:0/4049905309 >> v2:172.21.15.204:3300/0 conn(0x7f49480d2cf0 msgr2=0x7f49480cafb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:07:44.340 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.339+0000 7f494e2dd640 1 --2- 172.21.15.167:0/4049905309 >> v2:172.21.15.204:3300/0 conn(0x7f49480d2cf0 0x7f49480cafb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:44.340 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.339+0000 7f494e2dd640 1 -- 172.21.15.167:0/4049905309 >> v1:172.21.15.204:6789/0 conn(0x7f49481598d0 legacy=0x7f49480d0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:44.340 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.339+0000 7f494e2dd640 1 -- 172.21.15.167:0/4049905309 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f494806d7e0 con 0x7f49480cb4f0 2024-05-11T08:07:44.340 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.339+0000 7f494d2db640 1 --2- 172.21.15.167:0/4049905309 >> v2:172.21.15.204:3300/0 conn(0x7f49480d2cf0 0x7f49480cafb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T08:07:44.341 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.340+0000 7f494e2dd640 1 --2- 172.21.15.167:0/4049905309 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f49480cb4f0 0x7f494806dfe0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4940009e50 tx=0x7f494002a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dab740fd7f3df29d server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:44.342 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.340+0000 7f494cada640 1 -- 172.21.15.167:0/4049905309 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4940036070 con 0x7f49480cb4f0 2024-05-11T08:07:44.342 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.340+0000 7f494cada640 1 -- 172.21.15.167:0/4049905309 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4940031600 con 0x7f49480cb4f0 2024-05-11T08:07:44.342 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.340+0000 7f494cada640 1 -- 172.21.15.167:0/4049905309 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49400319b0 con 0x7f49480cb4f0 2024-05-11T08:07:44.342 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.341+0000 7f494fd40640 1 -- 172.21.15.167:0/4049905309 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f49480cb4f0 msgr2=0x7f494806dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:44.342 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.341+0000 7f494fd40640 1 --2- 172.21.15.167:0/4049905309 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f49480cb4f0 0x7f494806dfe0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4940009e50 tx=0x7f494002a710 comp rx=0 tx=0).stop 2024-05-11T08:07:44.342 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.341+0000 7f494fd40640 1 -- 172.21.15.167:0/4049905309 shutdown_connections 2024-05-11T08:07:44.342 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.341+0000 7f494fd40640 1 --2- 172.21.15.167:0/4049905309 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f49480cb4f0 0x7f494806dfe0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:44.342 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.341+0000 7f494fd40640 1 --2- 172.21.15.167:0/4049905309 >> v2:172.21.15.204:3300/0 conn(0x7f49480d2cf0 0x7f49480cafb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:44.342 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.341+0000 7f494fd40640 1 -- 172.21.15.167:0/4049905309 >> 172.21.15.167:0/4049905309 conn(0x7f49480d1ba0 msgr2=0x7f49480d1fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:07:44.342 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.341+0000 7f494fd40640 1 -- 172.21.15.167:0/4049905309 shutdown_connections 2024-05-11T08:07:44.342 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.341+0000 7f494fd40640 1 -- 172.21.15.167:0/4049905309 wait complete. 2024-05-11T08:07:44.344 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.342+0000 7f494fd40640 1 Processor -- start 2024-05-11T08:07:44.344 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.342+0000 7f494fd40640 1 -- start start 2024-05-11T08:07:44.344 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.343+0000 7f494fd40640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f49480cb4f0 0x7f49481538b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:44.344 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.343+0000 7f494fd40640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f49480d2cf0 0x7f4948145c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:44.345 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.343+0000 7f494fd40640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f49481598d0 0x7f49480ba610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:44.345 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.343+0000 7f494fd40640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4948159cb0 con 0x7f49481598d0 2024-05-11T08:07:44.345 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.343+0000 7f494fd40640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f49481607c0 con 0x7f49480cb4f0 2024-05-11T08:07:44.345 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.343+0000 7f494fd40640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7f494806d1b0 con 0x7f49480d2cf0 2024-05-11T08:07:44.345 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.343+0000 7f494e2dd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f49481598d0 0x7f49480ba610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:44.345 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.343+0000 7f494e2dd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f49481598d0 0x7f49480ba610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:44686/0 (socket says 172.21.15.167:44686) 2024-05-11T08:07:44.345 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.343+0000 7f494e2dd640 1 -- 172.21.15.167:0/3561614167 learned_addr learned my addr 172.21.15.167:0/3561614167 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:07:44.345 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.344+0000 7f494dadc640 1 --2- 172.21.15.167:0/3561614167 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f49480cb4f0 0x7f49481538b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:44.345 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.344+0000 7f494d2db640 1 --2- 172.21.15.167:0/3561614167 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f49480d2cf0 0x7f4948145c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:44.345 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.344+0000 7f494e2dd640 1 -- 172.21.15.167:0/3561614167 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f49480d2cf0 msgr2=0x7f4948145c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:44.345 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.344+0000 7f494e2dd640 1 --2- 172.21.15.167:0/3561614167 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f49480d2cf0 0x7f4948145c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:44.345 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.344+0000 7f494e2dd640 1 -- 172.21.15.167:0/3561614167 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f49480cb4f0 msgr2=0x7f49481538b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:44.345 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.344+0000 7f494e2dd640 1 --2- 172.21.15.167:0/3561614167 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f49480cb4f0 0x7f49481538b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:44.345 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.344+0000 7f494e2dd640 1 -- 172.21.15.167:0/3561614167 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4948153ec0 con 0x7f49481598d0 2024-05-11T08:07:44.345 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.344+0000 7f494e2dd640 1 --2- 172.21.15.167:0/3561614167 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f49481598d0 0x7f49480ba610 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f4940034040 tx=0x7f4940004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:44.346 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.344+0000 7f493e7fc640 1 -- 172.21.15.167:0/3561614167 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4940041020 con 0x7f49481598d0 2024-05-11T08:07:44.346 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.344+0000 7f493e7fc640 1 -- 172.21.15.167:0/3561614167 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4940037040 con 0x7f49481598d0 2024-05-11T08:07:44.346 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.344+0000 7f494fd40640 1 -- 172.21.15.167:0/3561614167 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49481541a0 con 0x7f49481598d0 2024-05-11T08:07:44.346 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.344+0000 7f493e7fc640 1 -- 172.21.15.167:0/3561614167 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4940036050 con 0x7f49481598d0 2024-05-11T08:07:44.346 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.344+0000 7f494fd40640 1 -- 172.21.15.167:0/3561614167 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4948154760 con 0x7f49481598d0 2024-05-11T08:07:44.347 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.345+0000 7f494fd40640 1 -- 172.21.15.167:0/3561614167 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4948069590 con 0x7f49481598d0 2024-05-11T08:07:44.347 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.346+0000 7f493e7fc640 1 -- 172.21.15.167:0/3561614167 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f4940007c50 con 0x7f49481598d0 2024-05-11T08:07:44.347 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.346+0000 7f493e7fc640 1 --2- 172.21.15.167:0/3561614167 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f492c0762d0 0x7f492c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:44.347 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.347+0000 7f493e7fc640 1 -- 172.21.15.167:0/3561614167 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5335+0+0 (secure 0 0 0) 0x7f49400b7950 con 0x7f49481598d0 2024-05-11T08:07:44.347 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.347+0000 7f494dadc640 1 --2- 172.21.15.167:0/3561614167 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f492c0762d0 0x7f492c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:44.347 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.347+0000 7f494dadc640 1 --2- 172.21.15.167:0/3561614167 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f492c0762d0 0x7f492c078790 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f4944007f50 tx=0x7f4944007ee0 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:44.351 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.351+0000 7f493e7fc640 1 -- 172.21.15.167:0/3561614167 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f494004b060 con 0x7f49481598d0 2024-05-11T08:07:44.528 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.528+0000 7f494fd40640 1 -- 172.21.15.167:0/3561614167 --> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f49480cbfb0 con 0x7f492c0762d0 2024-05-11T08:07:44.528 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.528+0000 7f493e7fc640 1 -- 172.21.15.167:0/3561614167 <== mgr.4113 v2:172.21.15.167:6832/253569496 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13344 (secure 0 0 0) 0x7f49480cbfb0 con 0x7f492c0762d0 2024-05-11T08:07:44.529 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T08:07:44.529 INFO:teuthology.orchestra.run.smithi167.stderr:dumped all 2024-05-11T08:07:44.531 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.530+0000 7f494fd40640 1 -- 172.21.15.167:0/3561614167 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f492c0762d0 msgr2=0x7f492c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:44.531 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.530+0000 7f494fd40640 1 --2- 172.21.15.167:0/3561614167 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f492c0762d0 0x7f492c078790 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f4944007f50 tx=0x7f4944007ee0 comp rx=0 tx=0).stop 2024-05-11T08:07:44.531 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.530+0000 7f494fd40640 1 -- 172.21.15.167:0/3561614167 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f49481598d0 msgr2=0x7f49480ba610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:44.531 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.530+0000 7f494fd40640 1 --2- 172.21.15.167:0/3561614167 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f49481598d0 0x7f49480ba610 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f4940034040 tx=0x7f4940004a40 comp rx=0 tx=0).stop 2024-05-11T08:07:44.531 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.530+0000 7f494fd40640 1 -- 172.21.15.167:0/3561614167 shutdown_connections 2024-05-11T08:07:44.531 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.530+0000 7f494fd40640 1 --2- 172.21.15.167:0/3561614167 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7f492c0762d0 0x7f492c078790 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:44.531 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.530+0000 7f494fd40640 1 --2- 172.21.15.167:0/3561614167 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f49481598d0 0x7f49480ba610 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:44.531 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.530+0000 7f494fd40640 1 --2- 172.21.15.167:0/3561614167 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f49480d2cf0 0x7f4948145c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:44.531 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.530+0000 7f494fd40640 1 --2- 172.21.15.167:0/3561614167 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f49480cb4f0 0x7f49481538b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:44.531 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.530+0000 7f494fd40640 1 -- 172.21.15.167:0/3561614167 >> 172.21.15.167:0/3561614167 conn(0x7f49480d1ba0 msgr2=0x7f49480b7f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:07:44.531 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.531+0000 7f494fd40640 1 -- 172.21.15.167:0/3561614167 shutdown_connections 2024-05-11T08:07:44.532 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.531+0000 7f494fd40640 1 -- 172.21.15.167:0/3561614167 wait complete. 2024-05-11T08:07:44.542 INFO:teuthology.orchestra.run.smithi167.stdout:{"pg_ready":true,"pg_map":{"version":103,"stamp":"2024-05-11T08:07:43.937915+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":225904,"kb_used_data":5576,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754748816,"statfs":{"total":773094113280,"available":772862787584,"internally_reserved":0,"allocated":5709824,"data_stored":3160976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":225496625},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002427"},"pg_stats":[{"pgid":"1.0","version":"48'74","reported_seq":171,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-05-11T08:07:24.143578+0000","last_change":"2024-05-11T08:07:24.143613+0000","last_active":"2024-05-11T08:07:24.143578+0000","last_peered":"2024-05-11T08:07:24.143578+0000","last_clean":"2024-05-11T08:07:24.143578+0000","last_became_active":"2024-05-11T08:05:45.771373+0000","last_became_peered":"2024-05-11T08:05:45.771373+0000","last_unstale":"2024-05-11T08:07:24.143578+0000","last_undegraded":"2024-05-11T08:07:24.143578+0000","last_fullsized":"2024-05-11T08:07:24.143578+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"48'74","last_scrub_stamp":"2024-05-11T08:07:24.143494+0000","last_deep_scrub":"48'74","last_deep_scrub_stamp":"2024-05-11T08:07:24.143494+0000","last_clean_scrub_stamp":"2024-05-11T08:07:24.143494+0000","objects_scrubbed":2,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:41:47.663941+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509468,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28148,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343692,"statfs":{"total":96636764160,"available":96607940608,"internally_reserved":0,"allocated":1159168,"data_stored":837898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509468,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27572,"kb_used_data":552,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344268,"statfs":{"total":96636764160,"available":96608530432,"internally_reserved":0,"allocated":565248,"data_stored":247530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509468,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27572,"kb_used_data":552,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344268,"statfs":{"total":96636764160,"available":96608530432,"internally_reserved":0,"allocated":565248,"data_stored":247530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509468,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27576,"kb_used_data":552,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344264,"statfs":{"total":96636764160,"available":96608526336,"internally_reserved":0,"allocated":565248,"data_stored":247530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509469,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27576,"kb_used_data":552,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344264,"statfs":{"total":96636764160,"available":96608526336,"internally_reserved":0,"allocated":565248,"data_stored":247530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509469,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27576,"kb_used_data":552,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344264,"statfs":{"total":96636764160,"available":96608526336,"internally_reserved":0,"allocated":565248,"data_stored":247530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509468,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28148,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343692,"statfs":{"total":96636764160,"available":96607940608,"internally_reserved":0,"allocated":1159168,"data_stored":837898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574876,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31736,"kb_used_data":552,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340104,"statfs":{"total":96636764160,"available":96604266496,"internally_reserved":0,"allocated":565248,"data_stored":247530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-11T08:07:44.543 DEBUG:teuthology.orchestra.run.smithi167:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-05-11T08:07:44.642 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.642+0000 7eff05dc9640 1 Processor -- start 2024-05-11T08:07:44.642 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.642+0000 7eff05dc9640 1 -- start start 2024-05-11T08:07:44.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.643+0000 7eff05dc9640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7eff0015f290 0x7eff0006e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:44.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.643+0000 7eff05dc9640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7eff000d1070 0x7eff000ce020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:44.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.643+0000 7eff05dc9640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7eff000ce560 con 0x7eff000708a0 2024-05-11T08:07:44.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.643+0000 7eff05dc9640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7eff0006df40 con 0x7eff0015f290 2024-05-11T08:07:44.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.643+0000 7eff05dc9640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7eff0006d930 con 0x7eff000d1070 2024-05-11T08:07:44.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.643+0000 7efeff7fe640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7eff0015f290 0x7eff0006e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:44.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.643+0000 7efefffff640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7eff000708a0 0x7eff0006c450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:56830/0 (socket says 172.21.15.167:56830) 2024-05-11T08:07:44.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.643+0000 7efefffff640 1 -- 172.21.15.167:0/2786046401 learned_addr learned my addr 172.21.15.167:0/2786046401 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T08:07:44.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.643+0000 7efefeffd640 1 --2- 172.21.15.167:0/2786046401 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7eff000d1070 0x7eff000ce020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:44.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.643+0000 7efeff7fe640 1 -- 172.21.15.167:0/2786046401 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7eff000d1070 msgr2=0x7eff000ce020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:07:44.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.643+0000 7efeff7fe640 1 --2- 172.21.15.167:0/2786046401 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7eff000d1070 0x7eff000ce020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:44.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.643+0000 7efeff7fe640 1 -- 172.21.15.167:0/2786046401 >> v1:172.21.15.167:6789/0 conn(0x7eff000708a0 legacy=0x7eff0006c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:44.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.643+0000 7efeff7fe640 1 -- 172.21.15.167:0/2786046401 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff000ce7e0 con 0x7eff0015f290 2024-05-11T08:07:44.645 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.643+0000 7efefe7fc640 1 -- 172.21.15.167:0/2786046401 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 987944047 0 0) 0x7efef8001110 con 0x7eff000708a0 2024-05-11T08:07:44.645 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.644+0000 7efefeffd640 1 --2- 172.21.15.167:0/2786046401 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7eff000d1070 0x7eff000ce020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T08:07:44.646 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.644+0000 7efeff7fe640 1 --2- 172.21.15.167:0/2786046401 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7eff0015f290 0x7eff0006e3d0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7efef0009dc0 tx=0x7efef002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e69e17478fb309e server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:44.646 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.644+0000 7efefe7fc640 1 -- 172.21.15.167:0/2786046401 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efef002ae60 con 0x7eff0015f290 2024-05-11T08:07:44.646 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.644+0000 7efefe7fc640 1 -- 172.21.15.167:0/2786046401 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efef0007740 con 0x7eff0015f290 2024-05-11T08:07:44.646 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.644+0000 7efefe7fc640 1 -- 172.21.15.167:0/2786046401 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efef0007b10 con 0x7eff0015f290 2024-05-11T08:07:44.646 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.645+0000 7eff05dc9640 1 -- 172.21.15.167:0/2786046401 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7eff0015f290 msgr2=0x7eff0006e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:44.646 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.645+0000 7eff05dc9640 1 --2- 172.21.15.167:0/2786046401 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7eff0015f290 0x7eff0006e3d0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7efef0009dc0 tx=0x7efef002a680 comp rx=0 tx=0).stop 2024-05-11T08:07:44.647 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.645+0000 7eff05dc9640 1 -- 172.21.15.167:0/2786046401 shutdown_connections 2024-05-11T08:07:44.647 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.645+0000 7eff05dc9640 1 --2- 172.21.15.167:0/2786046401 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7eff000d1070 0x7eff000ce020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:44.647 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.645+0000 7eff05dc9640 1 --2- 172.21.15.167:0/2786046401 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7eff0015f290 0x7eff0006e3d0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:44.647 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.645+0000 7eff05dc9640 1 -- 172.21.15.167:0/2786046401 >> 172.21.15.167:0/2786046401 conn(0x7eff0006fb60 msgr2=0x7eff00070190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:07:44.647 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.645+0000 7eff05dc9640 1 -- 172.21.15.167:0/2786046401 shutdown_connections 2024-05-11T08:07:44.647 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.645+0000 7eff05dc9640 1 -- 172.21.15.167:0/2786046401 wait complete. 2024-05-11T08:07:44.647 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.646+0000 7eff05dc9640 1 Processor -- start 2024-05-11T08:07:44.647 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.646+0000 7eff05dc9640 1 -- start start 2024-05-11T08:07:44.649 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.647+0000 7eff05dc9640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff000708a0 0x7eff001f6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:44.649 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.647+0000 7eff05dc9640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7eff000d1070 0x7eff00151380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:44.649 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.647+0000 7eff05dc9640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7eff0015f290 0x7eff00153f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:44.649 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.647+0000 7eff05dc9640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7eff0015f670 con 0x7eff000d1070 2024-05-11T08:07:44.649 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.647+0000 7eff05dc9640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7eff0006dbd0 con 0x7eff000708a0 2024-05-11T08:07:44.649 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.647+0000 7eff05dc9640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7eff00070d30 con 0x7eff0015f290 2024-05-11T08:07:44.650 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.647+0000 7efefeffd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7eff000d1070 0x7eff00151380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:44.650 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.647+0000 7efefeffd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7eff000d1070 0x7eff00151380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:37584/0 (socket says 172.21.15.167:37584) 2024-05-11T08:07:44.650 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.647+0000 7efefeffd640 1 -- 172.21.15.167:0/833723139 learned_addr learned my addr 172.21.15.167:0/833723139 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:07:44.650 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.647+0000 7efeff7fe640 1 --2- 172.21.15.167:0/833723139 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff000708a0 0x7eff001f6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:44.650 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.647+0000 7efefffff640 1 --2- 172.21.15.167:0/833723139 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7eff0015f290 0x7eff00153f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:44.650 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.647+0000 7efefeffd640 1 -- 172.21.15.167:0/833723139 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7eff0015f290 msgr2=0x7eff00153f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:44.650 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.647+0000 7efefeffd640 1 --2- 172.21.15.167:0/833723139 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7eff0015f290 0x7eff00153f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:44.650 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.647+0000 7efefeffd640 1 -- 172.21.15.167:0/833723139 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff000708a0 msgr2=0x7eff001f6110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:44.650 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.647+0000 7efefeffd640 1 --2- 172.21.15.167:0/833723139 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff000708a0 0x7eff001f6110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:44.650 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.647+0000 7efefeffd640 1 -- 172.21.15.167:0/833723139 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff001f6650 con 0x7eff000d1070 2024-05-11T08:07:44.650 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.647+0000 7efeff7fe640 1 --2- 172.21.15.167:0/833723139 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff000708a0 0x7eff001f6110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T08:07:44.650 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.647+0000 7efefffff640 1 --2- 172.21.15.167:0/833723139 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7eff0015f290 0x7eff00153f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T08:07:44.650 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.647+0000 7efefeffd640 1 --2- 172.21.15.167:0/833723139 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7eff000d1070 0x7eff00151380 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7efef400e810 tx=0x7efef400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:44.650 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.648+0000 7efee7fff640 1 -- 172.21.15.167:0/833723139 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efef400f040 con 0x7eff000d1070 2024-05-11T08:07:44.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.648+0000 7efee7fff640 1 -- 172.21.15.167:0/833723139 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efef4004030 con 0x7eff000d1070 2024-05-11T08:07:44.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.648+0000 7efee7fff640 1 -- 172.21.15.167:0/833723139 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efef4004400 con 0x7eff000d1070 2024-05-11T08:07:44.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.648+0000 7eff05dc9640 1 -- 172.21.15.167:0/833723139 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff00155670 con 0x7eff000d1070 2024-05-11T08:07:44.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.648+0000 7eff05dc9640 1 -- 172.21.15.167:0/833723139 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff00155c30 con 0x7eff000d1070 2024-05-11T08:07:44.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.649+0000 7eff05dc9640 1 -- 172.21.15.167:0/833723139 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff0006d3d0 con 0x7eff000d1070 2024-05-11T08:07:44.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.649+0000 7efee7fff640 1 -- 172.21.15.167:0/833723139 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7efef400c6d0 con 0x7eff000d1070 2024-05-11T08:07:44.652 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.650+0000 7efee7fff640 1 --2- 172.21.15.167:0/833723139 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7efeec076300 0x7efeec0787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:44.652 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.650+0000 7efee7fff640 1 -- 172.21.15.167:0/833723139 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5335+0+0 (secure 0 0 0) 0x7efef4097220 con 0x7eff000d1070 2024-05-11T08:07:44.652 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.650+0000 7efeff7fe640 1 --2- 172.21.15.167:0/833723139 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7efeec076300 0x7efeec0787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:44.652 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.651+0000 7efeff7fe640 1 --2- 172.21.15.167:0/833723139 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7efeec076300 0x7efeec0787c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7efef0034000 tx=0x7efef0033040 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:44.654 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.654+0000 7efee7fff640 1 -- 172.21.15.167:0/833723139 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7eff0006d3d0 con 0x7eff000d1070 2024-05-11T08:07:44.838 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.838+0000 7eff05dc9640 1 -- 172.21.15.167:0/833723139 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7eff00069520 con 0x7eff000d1070 2024-05-11T08:07:44.841 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.841+0000 7efee7fff640 1 -- 172.21.15.167:0/833723139 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v15)=0 v15) ==== 123+0+0 (secure 0 0 0) 0x7eff001f6650 con 0x7eff000d1070 2024-05-11T08:07:44.842 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.842+0000 7efee7fff640 1 -- 172.21.15.167:0/833723139 <== mon.0 v2:172.21.15.167:3300/0 8 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7efef4060100 con 0x7eff000d1070 2024-05-11T08:07:44.845 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.844+0000 7eff05dc9640 1 -- 172.21.15.167:0/833723139 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7efeec076300 msgr2=0x7efeec0787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:44.845 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.844+0000 7eff05dc9640 1 --2- 172.21.15.167:0/833723139 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7efeec076300 0x7efeec0787c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7efef0034000 tx=0x7efef0033040 comp rx=0 tx=0).stop 2024-05-11T08:07:44.845 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.844+0000 7eff05dc9640 1 -- 172.21.15.167:0/833723139 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7eff000d1070 msgr2=0x7eff00151380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:44.845 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.844+0000 7eff05dc9640 1 --2- 172.21.15.167:0/833723139 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7eff000d1070 0x7eff00151380 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7efef400e810 tx=0x7efef400ecd0 comp rx=0 tx=0).stop 2024-05-11T08:07:44.845 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.845+0000 7eff05dc9640 1 -- 172.21.15.167:0/833723139 shutdown_connections 2024-05-11T08:07:44.845 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.845+0000 7eff05dc9640 1 --2- 172.21.15.167:0/833723139 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7efeec076300 0x7efeec0787c0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:44.845 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.845+0000 7eff05dc9640 1 --2- 172.21.15.167:0/833723139 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7eff0015f290 0x7eff00153f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:44.845 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.845+0000 7eff05dc9640 1 --2- 172.21.15.167:0/833723139 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7eff000d1070 0x7eff00151380 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:44.845 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.845+0000 7eff05dc9640 1 --2- 172.21.15.167:0/833723139 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff000708a0 0x7eff001f6110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:44.845 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.845+0000 7eff05dc9640 1 -- 172.21.15.167:0/833723139 >> 172.21.15.167:0/833723139 conn(0x7eff0006fb60 msgr2=0x7eff000b5740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:07:44.846 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.845+0000 7eff05dc9640 1 -- 172.21.15.167:0/833723139 shutdown_connections 2024-05-11T08:07:44.846 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.845+0000 7eff05dc9640 1 -- 172.21.15.167:0/833723139 wait complete. 2024-05-11T08:07:44.856 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-11T08:07:44.856 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-11T08:07:44.956 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.956+0000 7fbce5de7640 1 Processor -- start 2024-05-11T08:07:44.956 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.956+0000 7fbce5de7640 1 -- start start 2024-05-11T08:07:44.957 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.957+0000 7fbce5de7640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fbce00736a0 0x7fbce0072b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:44.957 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.957+0000 7fbce5de7640 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbce015bc40 con 0x7fbce0072640 2024-05-11T08:07:44.957 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.957+0000 7fbce5de7640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbce006e410 con 0x7fbce015b860 2024-05-11T08:07:44.957 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.957+0000 7fbce5de7640 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 -- 0x7fbce006e6f0 con 0x7fbce00736a0 2024-05-11T08:07:44.958 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.957+0000 7fbcdffff640 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7fbce0072640 0x7fbce00b47d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:56838/0 (socket says 172.21.15.167:56838) 2024-05-11T08:07:44.958 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.957+0000 7fbcdeffd640 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fbce00736a0 0x7fbce0072b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:44.958 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.957+0000 7fbcdffff640 1 -- 172.21.15.167:0/1619354086 learned_addr learned my addr 172.21.15.167:0/1619354086 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-05-11T08:07:44.958 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.957+0000 7fbcdeffd640 1 -- 172.21.15.167:0/1619354086 >> v1:172.21.15.204:6789/0 conn(0x7fbce015b860 legacy=0x7fbce006f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T08:07:44.958 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.957+0000 7fbcdeffd640 1 -- 172.21.15.167:0/1619354086 >> v1:172.21.15.167:6789/0 conn(0x7fbce0072640 legacy=0x7fbce00b47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:44.958 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.957+0000 7fbcdeffd640 1 -- 172.21.15.167:0/1619354086 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbce006e9d0 con 0x7fbce00736a0 2024-05-11T08:07:44.958 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.957+0000 7fbcde7fc640 1 -- 172.21.15.167:0/1619354086 <== mon.0 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2976582926 0 0) 0x7fbcd8001110 con 0x7fbce0072640 2024-05-11T08:07:44.959 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.958+0000 7fbcdeffd640 1 --2- 172.21.15.167:0/1619354086 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbce00736a0 0x7fbce0072b00 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fbcd400a940 tx=0x7fbcd402b550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42c9498a4ed22afe server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:44.959 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.958+0000 7fbcde7fc640 1 -- 172.21.15.167:0/1619354086 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcd400b040 con 0x7fbce00736a0 2024-05-11T08:07:44.959 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.958+0000 7fbcde7fc640 1 -- 172.21.15.167:0/1619354086 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fbcd4030710 con 0x7fbce00736a0 2024-05-11T08:07:44.959 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.958+0000 7fbcde7fc640 1 -- 172.21.15.167:0/1619354086 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcd4030b00 con 0x7fbce00736a0 2024-05-11T08:07:44.959 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.959+0000 7fbce5de7640 1 -- 172.21.15.167:0/1619354086 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbce00736a0 msgr2=0x7fbce0072b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:44.959 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.959+0000 7fbce5de7640 1 --2- 172.21.15.167:0/1619354086 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbce00736a0 0x7fbce0072b00 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fbcd400a940 tx=0x7fbcd402b550 comp rx=0 tx=0).stop 2024-05-11T08:07:44.960 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.959+0000 7fbce5de7640 1 -- 172.21.15.167:0/1619354086 shutdown_connections 2024-05-11T08:07:44.960 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.959+0000 7fbce5de7640 1 --2- 172.21.15.167:0/1619354086 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbce00736a0 0x7fbce0072b00 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:44.960 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.959+0000 7fbce5de7640 1 -- 172.21.15.167:0/1619354086 >> 172.21.15.167:0/1619354086 conn(0x7fbce00cd8f0 msgr2=0x7fbce00cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:07:44.960 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.959+0000 7fbce5de7640 1 -- 172.21.15.167:0/1619354086 shutdown_connections 2024-05-11T08:07:44.960 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.959+0000 7fbce5de7640 1 -- 172.21.15.167:0/1619354086 wait complete. 2024-05-11T08:07:44.960 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.960+0000 7fbce5de7640 1 Processor -- start 2024-05-11T08:07:44.960 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.960+0000 7fbce5de7640 1 -- start start 2024-05-11T08:07:44.962 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.961+0000 7fbce5de7640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbce0072640 0x7fbce01467c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:44.962 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.961+0000 7fbce5de7640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbce00736a0 0x7fbce015db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:44.962 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.961+0000 7fbce5de7640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbce015b860 0x7fbce0072ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:44.963 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.961+0000 7fbce5de7640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fbce006fbc0 con 0x7fbce00736a0 2024-05-11T08:07:44.963 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.961+0000 7fbce5de7640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fbce006ef90 con 0x7fbce015b860 2024-05-11T08:07:44.963 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.961+0000 7fbce5de7640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 -- 0x7fbce00b4ee0 con 0x7fbce0072640 2024-05-11T08:07:44.963 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.961+0000 7fbcdeffd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbce00736a0 0x7fbce015db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:44.963 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.961+0000 7fbcdeffd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbce00736a0 0x7fbce015db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:37608/0 (socket says 172.21.15.167:37608) 2024-05-11T08:07:44.963 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.961+0000 7fbcdeffd640 1 -- 172.21.15.167:0/3422498098 learned_addr learned my addr 172.21.15.167:0/3422498098 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-05-11T08:07:44.963 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.961+0000 7fbcdf7fe640 1 --2- 172.21.15.167:0/3422498098 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbce0072640 0x7fbce01467c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:44.963 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.961+0000 7fbcdffff640 1 --2- 172.21.15.167:0/3422498098 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbce015b860 0x7fbce0072ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:44.963 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.961+0000 7fbcdeffd640 1 -- 172.21.15.167:0/3422498098 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbce0072640 msgr2=0x7fbce01467c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:44.963 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.961+0000 7fbcdeffd640 1 --2- 172.21.15.167:0/3422498098 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbce0072640 0x7fbce01467c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:44.963 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.961+0000 7fbcdeffd640 1 -- 172.21.15.167:0/3422498098 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbce015b860 msgr2=0x7fbce0072ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:44.963 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.961+0000 7fbcdeffd640 1 --2- 172.21.15.167:0/3422498098 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbce015b860 0x7fbce0072ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:44.963 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.961+0000 7fbcdeffd640 1 -- 172.21.15.167:0/3422498098 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbce015cae0 con 0x7fbce00736a0 2024-05-11T08:07:44.963 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.961+0000 7fbcdf7fe640 1 --2- 172.21.15.167:0/3422498098 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbce0072640 0x7fbce01467c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T08:07:44.963 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.961+0000 7fbcdffff640 1 --2- 172.21.15.167:0/3422498098 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbce015b860 0x7fbce0072ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T08:07:44.964 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.962+0000 7fbcdeffd640 1 --2- 172.21.15.167:0/3422498098 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbce00736a0 0x7fbce015db80 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fbcd402b590 tx=0x7fbcd4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:44.964 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.962+0000 7fbcc7fff640 1 -- 172.21.15.167:0/3422498098 <== mon.0 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcd4040070 con 0x7fbce00736a0 2024-05-11T08:07:44.964 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.962+0000 7fbcc7fff640 1 -- 172.21.15.167:0/3422498098 <== mon.0 v2:172.21.15.167:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fbcd4033d50 con 0x7fbce00736a0 2024-05-11T08:07:44.964 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.962+0000 7fbce5de7640 1 -- 172.21.15.167:0/3422498098 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbce015cdc0 con 0x7fbce00736a0 2024-05-11T08:07:44.964 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.962+0000 7fbcc7fff640 1 -- 172.21.15.167:0/3422498098 <== mon.0 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcd4004290 con 0x7fbce00736a0 2024-05-11T08:07:44.964 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.962+0000 7fbce5de7640 1 -- 172.21.15.167:0/3422498098 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbce015d380 con 0x7fbce00736a0 2024-05-11T08:07:44.965 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.963+0000 7fbce5de7640 1 -- 172.21.15.167:0/3422498098 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbce0069590 con 0x7fbce00736a0 2024-05-11T08:07:44.965 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.963+0000 7fbcc7fff640 1 -- 172.21.15.167:0/3422498098 <== mon.0 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fbcd4043070 con 0x7fbce00736a0 2024-05-11T08:07:44.965 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.964+0000 7fbcc7fff640 1 --2- 172.21.15.167:0/3422498098 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fbccc0762d0 0x7fbccc078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T08:07:44.965 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.964+0000 7fbcdf7fe640 1 --2- 172.21.15.167:0/3422498098 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fbccc0762d0 0x7fbccc078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T08:07:44.965 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.964+0000 7fbcc7fff640 1 -- 172.21.15.167:0/3422498098 <== mon.0 v2:172.21.15.167:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5335+0+0 (secure 0 0 0) 0x7fbcd40b8390 con 0x7fbce00736a0 2024-05-11T08:07:44.965 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.965+0000 7fbcdf7fe640 1 --2- 172.21.15.167:0/3422498098 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fbccc0762d0 0x7fbccc078790 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fbcd0009fc0 tx=0x7fbcd0009550 comp rx=0 tx=0).ready entity=mgr.4113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T08:07:44.968 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:44.969+0000 7fbcc7fff640 1 -- 172.21.15.167:0/3422498098 <== mon.0 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fbcd4081e60 con 0x7fbce00736a0 2024-05-11T08:07:45.187 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:45.187+0000 7fbce5de7640 1 -- 172.21.15.167:0/3422498098 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbce00cf1d0 con 0x7fbce00736a0 2024-05-11T08:07:45.188 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:45.188+0000 7fbcc7fff640 1 -- 172.21.15.167:0/3422498098 <== mon.0 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+3881 (secure 0 0 0) 0x7fbcd4081760 con 0x7fbce00736a0 2024-05-11T08:07:45.188 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-05-11T08:07:45.188 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":19,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4604,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2674983673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":2674983673},{"type":"v1","addr":"172.21.15.167:6837","nonce":2674983673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4607,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6834/1018548066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1018548066},{"type":"v1","addr":"172.21.15.204:6834","nonce":1018548066}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4729,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3369722249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3369722249},{"type":"v1","addr":"172.21.15.167:6836","nonce":3369722249}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5068,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/2817410075","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6833","nonce":2817410075},{"type":"v1","addr":"172.21.15.204:6835","nonce":2817410075}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[]} 2024-05-11T08:07:45.189 INFO:teuthology.orchestra.run.smithi167.stderr:dumped fsmap epoch 19 2024-05-11T08:07:45.190 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:45.190+0000 7fbce5de7640 1 -- 172.21.15.167:0/3422498098 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fbccc0762d0 msgr2=0x7fbccc078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:45.190 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:45.190+0000 7fbce5de7640 1 --2- 172.21.15.167:0/3422498098 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fbccc0762d0 0x7fbccc078790 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fbcd0009fc0 tx=0x7fbcd0009550 comp rx=0 tx=0).stop 2024-05-11T08:07:45.190 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:45.190+0000 7fbce5de7640 1 -- 172.21.15.167:0/3422498098 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbce00736a0 msgr2=0x7fbce015db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T08:07:45.190 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:45.190+0000 7fbce5de7640 1 --2- 172.21.15.167:0/3422498098 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbce00736a0 0x7fbce015db80 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fbcd402b590 tx=0x7fbcd4004730 comp rx=0 tx=0).stop 2024-05-11T08:07:45.190 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:45.190+0000 7fbce5de7640 1 -- 172.21.15.167:0/3422498098 shutdown_connections 2024-05-11T08:07:45.191 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:45.190+0000 7fbce5de7640 1 --2- 172.21.15.167:0/3422498098 >> [v2:172.21.15.167:6832/253569496,v1:172.21.15.167:6833/253569496] conn(0x7fbccc0762d0 0x7fbccc078790 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:45.191 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:45.190+0000 7fbce5de7640 1 --2- 172.21.15.167:0/3422498098 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbce015b860 0x7fbce0072ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:45.191 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:45.190+0000 7fbce5de7640 1 --2- 172.21.15.167:0/3422498098 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbce00736a0 0x7fbce015db80 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:45.191 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:45.190+0000 7fbce5de7640 1 --2- 172.21.15.167:0/3422498098 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbce0072640 0x7fbce01467c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T08:07:45.191 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:45.190+0000 7fbce5de7640 1 -- 172.21.15.167:0/3422498098 >> 172.21.15.167:0/3422498098 conn(0x7fbce00cd8f0 msgr2=0x7fbce0150310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T08:07:45.191 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:45.190+0000 7fbce5de7640 1 -- 172.21.15.167:0/3422498098 shutdown_connections 2024-05-11T08:07:45.191 INFO:teuthology.orchestra.run.smithi167.stderr:2024-05-11T08:07:45.190+0000 7fbce5de7640 1 -- 172.21.15.167:0/3422498098 wait complete. 2024-05-11T08:07:45.201 DEBUG:tasks.cephfs.filesystem:already dead... 2024-05-11T08:07:45.202 INFO:teuthology.misc:Shutting down mds daemons... 2024-05-11T08:07:45.202 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-11T08:07:45.202 INFO:teuthology.orchestra.run:waiting for 300 2024-05-11T08:07:45.211 INFO:tasks.ceph.mds.a:Stopped 2024-05-11T08:07:45.211 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-11T08:07:45.211 INFO:teuthology.orchestra.run:waiting for 300 2024-05-11T08:07:45.260 INFO:tasks.ceph.mds.c:Stopped 2024-05-11T08:07:45.260 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-11T08:07:45.261 INFO:teuthology.orchestra.run:waiting for 300 2024-05-11T08:07:45.273 INFO:tasks.ceph.mds.b:Stopped 2024-05-11T08:07:45.273 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-11T08:07:45.274 INFO:teuthology.orchestra.run:waiting for 300 2024-05-11T08:07:45.325 INFO:tasks.ceph.mds.d:Stopped 2024-05-11T08:07:45.326 INFO:teuthology.misc:Shutting down osd daemons... 2024-05-11T08:07:45.326 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-05-11T08:07:45.326 INFO:teuthology.orchestra.run:waiting for 300 2024-05-11T08:07:45.357 INFO:tasks.ceph.osd.0:Stopped 2024-05-11T08:07:45.357 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-05-11T08:07:45.357 INFO:teuthology.orchestra.run:waiting for 300 2024-05-11T08:07:45.431 INFO:tasks.ceph.osd.1:Stopped 2024-05-11T08:07:45.431 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-05-11T08:07:45.431 INFO:teuthology.orchestra.run:waiting for 300 2024-05-11T08:07:45.499 INFO:tasks.ceph.osd.2:Stopped 2024-05-11T08:07:45.499 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-05-11T08:07:45.499 INFO:teuthology.orchestra.run:waiting for 300 2024-05-11T08:07:45.565 INFO:tasks.ceph.osd.3:Stopped 2024-05-11T08:07:45.565 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-05-11T08:07:45.565 INFO:teuthology.orchestra.run:waiting for 300 2024-05-11T08:07:45.596 INFO:tasks.ceph.osd.4:Stopped 2024-05-11T08:07:45.596 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-05-11T08:07:45.597 INFO:teuthology.orchestra.run:waiting for 300 2024-05-11T08:07:45.669 INFO:tasks.ceph.osd.5:Stopped 2024-05-11T08:07:45.669 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-05-11T08:07:45.669 INFO:teuthology.orchestra.run:waiting for 300 2024-05-11T08:07:45.741 INFO:tasks.ceph.osd.6:Stopped 2024-05-11T08:07:45.741 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-05-11T08:07:45.741 INFO:teuthology.orchestra.run:waiting for 300 2024-05-11T08:07:45.813 INFO:tasks.ceph.osd.7:Stopped 2024-05-11T08:07:45.813 INFO:teuthology.misc:Shutting down mgr daemons... 2024-05-11T08:07:45.814 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-05-11T08:07:45.814 INFO:teuthology.orchestra.run:waiting for 300 2024-05-11T08:07:45.839 INFO:tasks.ceph.mgr.y:Stopped 2024-05-11T08:07:45.840 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-05-11T08:07:45.840 INFO:teuthology.orchestra.run:waiting for 300 2024-05-11T08:07:45.883 INFO:tasks.ceph.mgr.x:Stopped 2024-05-11T08:07:45.883 INFO:teuthology.misc:Shutting down mon daemons... 2024-05-11T08:07:45.883 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-05-11T08:07:45.883 INFO:teuthology.orchestra.run:waiting for 300 2024-05-11T08:07:45.893 INFO:tasks.ceph.mon.a:Stopped 2024-05-11T08:07:45.894 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-05-11T08:07:45.894 INFO:teuthology.orchestra.run:waiting for 300 2024-05-11T08:07:45.935 INFO:tasks.ceph.mon.b:Stopped 2024-05-11T08:07:45.935 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-05-11T08:07:45.936 INFO:teuthology.orchestra.run:waiting for 300 2024-05-11T08:07:45.988 INFO:tasks.ceph.mon.c:Stopped 2024-05-11T08:07:45.988 INFO:tasks.ceph:Checking cluster log for badness... 2024-05-11T08:07:45.988 DEBUG:teuthology.orchestra.run.smithi167:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | head -n 1 2024-05-11T08:07:46.020 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi167.front.sepia.ceph.com 2024-05-11T08:07:46.020 DEBUG:teuthology.orchestra.run.smithi167:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-05-11T08:07:46.238 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi167.front.sepia.ceph.com 2024-05-11T08:07:46.238 DEBUG:teuthology.orchestra.run.smithi167:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-05-11T08:07:46.359 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi167.front.sepia.ceph.com 2024-05-11T08:07:46.359 DEBUG:teuthology.orchestra.run.smithi167:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-05-11T08:07:46.424 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi167.front.sepia.ceph.com 2024-05-11T08:07:46.425 DEBUG:teuthology.orchestra.run.smithi167:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-05-11T08:07:46.491 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi204.front.sepia.ceph.com 2024-05-11T08:07:46.491 DEBUG:teuthology.orchestra.run.smithi204:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-05-11T08:07:46.681 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi204.front.sepia.ceph.com 2024-05-11T08:07:46.682 DEBUG:teuthology.orchestra.run.smithi204:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-05-11T08:07:46.766 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi204.front.sepia.ceph.com 2024-05-11T08:07:46.766 DEBUG:teuthology.orchestra.run.smithi204:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-05-11T08:07:46.807 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi204.front.sepia.ceph.com 2024-05-11T08:07:46.807 DEBUG:teuthology.orchestra.run.smithi204:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-05-11T08:07:46.899 INFO:tasks.ceph:Archiving mon data... 2024-05-11T08:07:46.901 DEBUG:teuthology.misc:Transferring archived files from smithi167:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/rishabh-2024-05-11_05:57:22-fs-wip-rishabh-testing-20240510.053205-testing-default-smithi/7702178/data/mon.a.tgz 2024-05-11T08:07:46.901 DEBUG:teuthology.orchestra.run.smithi167:> mktemp 2024-05-11T08:07:46.917 INFO:teuthology.orchestra.run.smithi167.stdout:/tmp/tmp.pb9xwUrFji 2024-05-11T08:07:46.917 DEBUG:teuthology.orchestra.run.smithi167:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.pb9xwUrFji 2024-05-11T08:07:47.049 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod 0666 /tmp/tmp.pb9xwUrFji 2024-05-11T08:07:47.131 DEBUG:teuthology.orchestra.remote:smithi167:/tmp/tmp.pb9xwUrFji is 305KB 2024-05-11T08:07:47.198 DEBUG:teuthology.orchestra.run.smithi167:> rm -fr /tmp/tmp.pb9xwUrFji 2024-05-11T08:07:47.213 DEBUG:teuthology.misc:Transferring archived files from smithi204:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/rishabh-2024-05-11_05:57:22-fs-wip-rishabh-testing-20240510.053205-testing-default-smithi/7702178/data/mon.b.tgz 2024-05-11T08:07:47.214 DEBUG:teuthology.orchestra.run.smithi204:> mktemp 2024-05-11T08:07:47.232 INFO:teuthology.orchestra.run.smithi204.stdout:/tmp/tmp.48VOIH8f4F 2024-05-11T08:07:47.232 DEBUG:teuthology.orchestra.run.smithi204:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.48VOIH8f4F 2024-05-11T08:07:47.385 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod 0666 /tmp/tmp.48VOIH8f4F 2024-05-11T08:07:47.469 DEBUG:teuthology.orchestra.remote:smithi204:/tmp/tmp.48VOIH8f4F is 409KB 2024-05-11T08:07:47.568 DEBUG:teuthology.orchestra.run.smithi204:> rm -fr /tmp/tmp.48VOIH8f4F 2024-05-11T08:07:47.584 DEBUG:teuthology.misc:Transferring archived files from smithi204:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/rishabh-2024-05-11_05:57:22-fs-wip-rishabh-testing-20240510.053205-testing-default-smithi/7702178/data/mon.c.tgz 2024-05-11T08:07:47.584 DEBUG:teuthology.orchestra.run.smithi204:> mktemp 2024-05-11T08:07:47.638 INFO:teuthology.orchestra.run.smithi204.stdout:/tmp/tmp.6p3cfwzla2 2024-05-11T08:07:47.638 DEBUG:teuthology.orchestra.run.smithi204:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.6p3cfwzla2 2024-05-11T08:07:47.776 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod 0666 /tmp/tmp.6p3cfwzla2 2024-05-11T08:07:47.859 DEBUG:teuthology.orchestra.remote:smithi204:/tmp/tmp.6p3cfwzla2 is 410KB 2024-05-11T08:07:47.928 DEBUG:teuthology.orchestra.run.smithi204:> rm -fr /tmp/tmp.6p3cfwzla2 2024-05-11T08:07:47.943 INFO:tasks.ceph:Cleaning ceph cluster... 2024-05-11T08:07:47.943 DEBUG:teuthology.orchestra.run.smithi167:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-05-11T08:07:47.946 DEBUG:teuthology.orchestra.run.smithi204:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-05-11T08:07:48.029 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-05-11T08:07:48.063 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-05-11T08:07:48.063 INFO:tasks.ceph:Archiving crash dumps... 2024-05-11T08:07:48.065 DEBUG:teuthology.misc:Transferring archived files from smithi167:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2024-05-11_05:57:22-fs-wip-rishabh-testing-20240510.053205-testing-default-smithi/7702178/remote/smithi167/crash 2024-05-11T08:07:48.066 DEBUG:teuthology.orchestra.run.smithi167:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-05-11T08:07:48.096 DEBUG:teuthology.misc:Transferring archived files from smithi204:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2024-05-11_05:57:22-fs-wip-rishabh-testing-20240510.053205-testing-default-smithi/7702178/remote/smithi204/crash 2024-05-11T08:07:48.096 DEBUG:teuthology.orchestra.run.smithi204:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-05-11T08:07:48.128 INFO:tasks.ceph:Shutting down logrotate 2024-05-11T08:07:48.128 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-11T08:07:48.140 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-11T08:07:48.220 DEBUG:teuthology.orchestra.run.smithi167:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-05-11T08:07:48.246 DEBUG:teuthology.orchestra.run.smithi204:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-05-11T08:07:48.270 INFO:tasks.ceph:Compressing logs... 2024-05-11T08:07:48.270 DEBUG:teuthology.orchestra.run.smithi167:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-11T08:07:48.288 DEBUG:teuthology.orchestra.run.smithi204:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-11T08:07:48.310 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48178.log 2024-05-11T08:07:48.311 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50076.log 2024-05-11T08:07:48.369 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35726.log 2024-05-11T08:07:48.370 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-05-11T08:07:48.370 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-05-11T08:07:48.370 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.35726.log: /var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-05-11T08:07:48.370 INFO:teuthology.orchestra.run.smithi204.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.35726.log.gz 2024-05-11T08:07:48.370 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-05-11T08:07:48.370 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-05-11T08:07:48.370 INFO:teuthology.orchestra.run.smithi204.stderr: 91.3% -- replaced with /var/log/ceph/ceph.log.gz 2024-05-11T08:07:48.370 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-05-11T08:07:48.370 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35897.log 2024-05-11T08:07:48.370 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-05-11T08:07:48.370 INFO:teuthology.orchestra.run.smithi204.stderr: 93.7% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-05-11T08:07:48.370 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.35897.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-05-11T08:07:48.370 INFO:teuthology.orchestra.run.smithi204.stderr:gzip 87.2% -- replaced with /var/log/ceph/ceph-client.admin.35897.log.gz 2024-05-11T08:07:48.371 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-mgr.x.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.36017.log 2024-05-11T08:07:48.371 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35806.log 2024-05-11T08:07:48.371 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.36017.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-05-11T08:07:48.371 INFO:teuthology.orchestra.run.smithi204.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.36017.log.gz 2024-05-11T08:07:48.371 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.35806.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35850.log 2024-05-11T08:07:48.371 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-osd.6.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.35806.log.gz 2024-05-11T08:07:48.371 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.42316.log 2024-05-11T08:07:48.371 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.35850.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.35850.log.gz 2024-05-11T08:07:48.371 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.42321.log 2024-05-11T08:07:48.371 INFO:teuthology.orchestra.run.smithi204.stderr: 92.5% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-05-11T08:07:48.371 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.1.42316.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.42316.log.gz 2024-05-11T08:07:48.371 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35977.log 2024-05-11T08:07:48.371 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.1.42321.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35766.log 2024-05-11T08:07:48.371 INFO:teuthology.orchestra.run.smithi204.stderr: -- replaced with /var/log/ceph/ceph-client.1.42321.log.gz 2024-05-11T08:07:48.372 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.35977.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.35977.log.gz 2024-05-11T08:07:48.372 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35937.log 2024-05-11T08:07:48.372 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.35766.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.42717.log 2024-05-11T08:07:48.372 INFO:teuthology.orchestra.run.smithi204.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.35766.log.gz 2024-05-11T08:07:48.372 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.35937.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.35937.log.gz 2024-05-11T08:07:48.372 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-05-11T08:07:48.372 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.3.42717.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-05-11T08:07:48.372 INFO:teuthology.orchestra.run.smithi204.stderr: -- replaced with /var/log/ceph/ceph-client.3.42717.log.gz 2024-05-11T08:07:48.372 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.3.42721.log 2024-05-11T08:07:48.372 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-mds.d.log: /var/log/ceph/ceph-mon.b.log: 91.7% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-05-11T08:07:48.372 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.3.42721.log: 0.0% -- replaced with /var/log/ceph/ceph-client.3.42721.log.gz 2024-05-11T08:07:48.408 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.48178.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46512.log 2024-05-11T08:07:48.409 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.50076.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41299.log 2024-05-11T08:07:48.409 INFO:teuthology.orchestra.run.smithi167.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.48178.log.gz 2024-05-11T08:07:48.409 INFO:teuthology.orchestra.run.smithi167.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50076.log.gz 2024-05-11T08:07:48.409 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.46512.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40993.log 2024-05-11T08:07:48.409 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.41299.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41445.log 2024-05-11T08:07:48.409 INFO:teuthology.orchestra.run.smithi167.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46512.log.gz 2024-05-11T08:07:48.409 INFO:teuthology.orchestra.run.smithi167.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.41299.log.gz 2024-05-11T08:07:48.409 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.40993.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48500.log 2024-05-11T08:07:48.409 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.41445.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.40993.log.gz 2024-05-11T08:07:48.409 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47052.log 2024-05-11T08:07:48.409 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.48500.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.41445.log.gz 2024-05-11T08:07:48.409 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47138.log 2024-05-11T08:07:48.409 INFO:teuthology.orchestra.run.smithi167.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.48500.log.gz/var/log/ceph/ceph-client.admin.47052.log: 2024-05-11T08:07:48.410 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46799.log 2024-05-11T08:07:48.410 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.47138.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47052.log.gz 2024-05-11T08:07:48.410 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47224.log 2024-05-11T08:07:48.410 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.46799.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47138.log.gz 2024-05-11T08:07:48.410 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41089.log 2024-05-11T08:07:48.410 INFO:teuthology.orchestra.run.smithi167.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46799.log.gz 2024-05-11T08:07:48.410 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.47224.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41343.log 2024-05-11T08:07:48.410 INFO:teuthology.orchestra.run.smithi167.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.47224.log.gz 2024-05-11T08:07:48.410 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.41089.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42922.log 2024-05-11T08:07:48.410 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.41343.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43855.log 2024-05-11T08:07:48.410 INFO:teuthology.orchestra.run.smithi167.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41089.log.gz 2024-05-11T08:07:48.410 INFO:teuthology.orchestra.run.smithi167.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.41343.log.gz 2024-05-11T08:07:48.410 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.42922.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41657.log 2024-05-11T08:07:48.410 INFO:teuthology.orchestra.run.smithi167.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.42922.log.gz 2024-05-11T08:07:48.411 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.43855.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44140.log 2024-05-11T08:07:48.411 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.41657.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43855.log.gz 2024-05-11T08:07:48.411 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43184.log 2024-05-11T08:07:48.411 INFO:teuthology.orchestra.run.smithi167.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.41657.log.gz 2024-05-11T08:07:48.411 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.44140.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41255.log 2024-05-11T08:07:48.411 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.43184.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44140.log.gz 2024-05-11T08:07:48.411 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48944.log 2024-05-11T08:07:48.411 INFO:teuthology.orchestra.run.smithi167.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43184.log.gz 2024-05-11T08:07:48.411 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.41255.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41489.log 2024-05-11T08:07:48.411 INFO:teuthology.orchestra.run.smithi167.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.41255.log.gz 2024-05-11T08:07:48.411 INFO:teuthology.orchestra.run.smithi167.stderr:gzip/var/log/ceph/ceph-client.admin.48944.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.48900.log 2024-05-11T08:07:48.411 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.41489.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43519.log 2024-05-11T08:07:48.411 INFO:teuthology.orchestra.run.smithi167.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.48944.log.gz 2024-05-11T08:07:48.411 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.48900.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.41489.log.gz 2024-05-11T08:07:48.412 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46057.log 2024-05-11T08:07:48.412 INFO:teuthology.orchestra.run.smithi167.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.48900.log.gz 2024-05-11T08:07:48.412 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.43519.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43859.log 2024-05-11T08:07:48.412 INFO:teuthology.orchestra.run.smithi167.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.43519.log.gz 2024-05-11T08:07:48.412 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.46057.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-05-11T08:07:48.412 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.43859.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44308.log 2024-05-11T08:07:48.412 INFO:teuthology.orchestra.run.smithi167.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.46057.log.gz 2024-05-11T08:07:48.412 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose 88.8% -- -- replaced with /var/log/ceph/ceph-client.admin.43859.log.gz /var/log/ceph/ceph-client.admin.51358.log 2024-05-11T08:07:48.412 INFO:teuthology.orchestra.run.smithi167.stderr: 2024-05-11T08:07:48.412 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.44308.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-05-11T08:07:48.412 INFO:teuthology.orchestra.run.smithi167.stderr: 84.0% -- replaced with /var/log/ceph/ceph.log.gz 2024-05-11T08:07:48.412 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.51358.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44308.log.gz 2024-05-11T08:07:48.412 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45442.log 2024-05-11T08:07:48.412 INFO:teuthology.orchestra.run.smithi167.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51358.log.gz 2024-05-11T08:07:48.413 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.48100.log 2024-05-11T08:07:48.413 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.45442.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45967.log 2024-05-11T08:07:48.413 INFO:teuthology.orchestra.run.smithi167.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.45442.log.gz 2024-05-11T08:07:48.413 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.0.48100.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.44833.log 2024-05-11T08:07:48.413 INFO:teuthology.orchestra.run.smithi167.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.48100.log.gz 2024-05-11T08:07:48.413 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.45967.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43006.log 2024-05-11T08:07:48.413 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.0.44833.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45967.log.gz 2024-05-11T08:07:48.413 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44224.log 2024-05-11T08:07:48.413 INFO:teuthology.orchestra.run.smithi167.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.44833.log.gz 2024-05-11T08:07:48.413 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.43006.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42880.log 2024-05-11T08:07:48.413 INFO:teuthology.orchestra.run.smithi167.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43006.log.gz 2024-05-11T08:07:48.413 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.44224.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45652.log 2024-05-11T08:07:48.413 INFO:teuthology.orchestra.run.smithi167.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44224.log.gz 2024-05-11T08:07:48.413 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.42880.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42509.log 2024-05-11T08:07:48.413 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.45652.log: gzip 87.6% -5 -- replaced with /var/log/ceph/ceph-client.admin.42880.log.gz --verbose 2024-05-11T08:07:48.414 INFO:teuthology.orchestra.run.smithi167.stderr: -- /var/log/ceph/ceph-client.admin.47182.log 2024-05-11T08:07:48.414 INFO:teuthology.orchestra.run.smithi167.stderr: 89.1%/var/log/ceph/ceph-client.admin.42509.log: -- replaced with /var/log/ceph/ceph-client.admin.45652.log.gz 2024-05-11T08:07:48.414 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35954.log 2024-05-11T08:07:48.414 INFO:teuthology.orchestra.run.smithi167.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42509.log.gz 2024-05-11T08:07:48.414 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.47182.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51274.log 2024-05-11T08:07:48.414 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.35954.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.47182.log.gz 2024-05-11T08:07:48.414 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49992.log 2024-05-11T08:07:48.414 INFO:teuthology.orchestra.run.smithi167.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.35954.log.gz 2024-05-11T08:07:48.414 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.51274.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49487.log 2024-05-11T08:07:48.414 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.49992.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.51274.log.gz 2024-05-11T08:07:48.414 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47705.log 2024-05-11T08:07:48.414 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.49487.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.49992.log.gz 2024-05-11T08:07:48.414 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-05-11T08:07:48.415 INFO:teuthology.orchestra.run.smithi167.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.49487.log.gz 2024-05-11T08:07:48.415 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.47705.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47010.log 2024-05-11T08:07:48.415 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph.audit.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.47705.log.gz 2024-05-11T08:07:48.415 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43521.log 2024-05-11T08:07:48.415 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.47010.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50728.log 2024-05-11T08:07:48.415 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.43521.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47010.log.gz 2024-05-11T08:07:48.415 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43520.log 2024-05-11T08:07:48.415 INFO:teuthology.orchestra.run.smithi167.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43521.log.gz 2024-05-11T08:07:48.415 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.50728.log: 89.2% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-05-11T08:07:48.415 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44350.log 2024-05-11T08:07:48.415 INFO:teuthology.orchestra.run.smithi167.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.50728.log.gz 2024-05-11T08:07:48.415 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.43520.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44098.log 2024-05-11T08:07:48.415 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.44350.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43520.log.gz 2024-05-11T08:07:48.415 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42712.log 2024-05-11T08:07:48.415 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.44098.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46596.log 2024-05-11T08:07:48.416 INFO:teuthology.orchestra.run.smithi167.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44098.log.gz 2024-05-11T08:07:48.416 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.42712.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.44350.log.gz 2024-05-11T08:07:48.416 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43524.log 2024-05-11T08:07:48.416 INFO:teuthology.orchestra.run.smithi167.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42712.log.gz 2024-05-11T08:07:48.416 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48646.log 2024-05-11T08:07:48.416 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.46596.log: /var/log/ceph/ceph-client.admin.43524.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.29559.log 2024-05-11T08:07:48.416 INFO:teuthology.orchestra.run.smithi167.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46596.log.gz 2024-05-11T08:07:48.416 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.48646.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43524.log.gz 2024-05-11T08:07:48.416 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42451.log 2024-05-11T08:07:48.416 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph.tmp-client.admin.29559.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.48646.log.gz 2024-05-11T08:07:48.416 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51022.log 2024-05-11T08:07:48.416 INFO:teuthology.orchestra.run.smithi167.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.29559.log.gz 2024-05-11T08:07:48.416 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.42451.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47094.log 2024-05-11T08:07:48.417 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.51022.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42451.log.gz 2024-05-11T08:07:48.417 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43188.log 2024-05-11T08:07:48.417 INFO:teuthology.orchestra.run.smithi167.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.51022.log.gz 2024-05-11T08:07:48.417 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.47094.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48623.log 2024-05-11T08:07:48.417 INFO:teuthology.orchestra.run.smithi167.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47094.log.gz/var/log/ceph/ceph-client.admin.43188.log: 2024-05-11T08:07:48.417 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47560.log 2024-05-11T08:07:48.417 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.48623.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43188.log.gz 2024-05-11T08:07:48.417 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-05-11T08:07:48.417 INFO:teuthology.orchestra.run.smithi167.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48623.log.gz 2024-05-11T08:07:48.417 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.47560.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41615.log 2024-05-11T08:07:48.417 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-osd.3.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47560.log.gz 2024-05-11T08:07:48.417 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43187.log 2024-05-11T08:07:48.417 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.41615.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47434.log 2024-05-11T08:07:48.417 INFO:teuthology.orchestra.run.smithi167.stderr: 88.5%/var/log/ceph/ceph-client.admin.43187.log: -- replaced with /var/log/ceph/ceph-client.admin.41615.log.gz 2024-05-11T08:07:48.418 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49163.log 2024-05-11T08:07:48.418 INFO:teuthology.orchestra.run.smithi167.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43187.log.gz 2024-05-11T08:07:48.418 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.47434.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44266.log 2024-05-11T08:07:48.418 INFO:teuthology.orchestra.run.smithi167.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47434.log.gz 2024-05-11T08:07:48.418 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-05-11T08:07:48.418 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.49163.log: /var/log/ceph/ceph-client.admin.44266.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42586.log 2024-05-11T08:07:48.418 INFO:teuthology.orchestra.run.smithi167.stderr: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.49163.log.gz 2024-05-11T08:07:48.418 INFO:teuthology.orchestra.run.smithi167.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44266.log.gz 2024-05-11T08:07:48.418 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50896.log 2024-05-11T08:07:48.418 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46676.log 2024-05-11T08:07:48.418 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.42586.log: /var/log/ceph/ceph-client.admin.50896.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50980.log 2024-05-11T08:07:48.418 INFO:teuthology.orchestra.run.smithi167.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.42586.log.gz 2024-05-11T08:07:48.418 INFO:teuthology.orchestra.run.smithi167.stderr: 91.8% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-05-11T08:07:48.419 INFO:teuthology.orchestra.run.smithi167.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50896.log.gz 2024-05-11T08:07:48.419 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.46676.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43182.log 2024-05-11T08:07:48.419 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.50980.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48986.log 2024-05-11T08:07:48.419 INFO:teuthology.orchestra.run.smithi167.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.46676.log.gz 2024-05-11T08:07:48.419 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.43182.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.50980.log.gz 2024-05-11T08:07:48.419 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-05-11T08:07:48.419 INFO:teuthology.orchestra.run.smithi167.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43182.log.gz 2024-05-11T08:07:48.419 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.48986.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-05-11T08:07:48.419 INFO:teuthology.orchestra.run.smithi167.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48986.log.gz 2024-05-11T08:07:48.419 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50202.log 2024-05-11T08:07:48.419 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose/var/log/ceph/ceph-mgr.y.log: -- /var/log/ceph/ceph-client.admin.43186.log 2024-05-11T08:07:48.419 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.50202.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50160.log 2024-05-11T08:07:48.419 INFO:teuthology.orchestra.run.smithi167.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50202.log.gz/var/log/ceph/ceph-client.admin.43186.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48311.log 2024-05-11T08:07:48.419 INFO:teuthology.orchestra.run.smithi167.stderr: 2024-05-11T08:07:48.419 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.50160.log: gzip 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43186.log.gz 2024-05-11T08:07:48.420 INFO:teuthology.orchestra.run.smithi167.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.47476.log 2024-05-11T08:07:48.420 INFO:teuthology.orchestra.run.smithi167.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.50160.log.gz 2024-05-11T08:07:48.420 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.48311.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-05-11T08:07:48.420 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 89.6% -- replaced with /var/log/ceph/ceph-client.admin.48311.log.gz 2024-05-11T08:07:48.420 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.47476.log: --verbose -- /var/log/ceph/ceph-client.admin.43514.log 2024-05-11T08:07:48.420 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-osd.0.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.47476.log.gz 2024-05-11T08:07:48.420 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51430.log 2024-05-11T08:07:48.420 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.43514.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47350.log 2024-05-11T08:07:48.420 INFO:teuthology.orchestra.run.smithi167.stderr: 89.1%/var/log/ceph/ceph-client.admin.51430.log: -- replaced with /var/log/ceph/ceph-client.admin.43514.log.gz 2024-05-11T08:07:48.420 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47602.log 2024-05-11T08:07:48.420 INFO:teuthology.orchestra.run.smithi167.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.51430.log.gz 2024-05-11T08:07:48.420 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46134.log 2024-05-11T08:07:48.420 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.47350.log: /var/log/ceph/ceph-client.admin.47602.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47350.log.gz 2024-05-11T08:07:48.420 INFO:teuthology.orchestra.run.smithi167.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47602.log.gz 2024-05-11T08:07:48.420 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47750.log 2024-05-11T08:07:48.421 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.46134.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49866.log 2024-05-11T08:07:48.421 INFO:teuthology.orchestra.run.smithi167.stderr: 87.8%/var/log/ceph/ceph-client.admin.47750.log: -- replaced with /var/log/ceph/ceph-client.admin.46134.log.gz 2024-05-11T08:07:48.421 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48288.log 2024-05-11T08:07:48.421 INFO:teuthology.orchestra.run.smithi167.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47750.log.gz 2024-05-11T08:07:48.421 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.49866.log: -- /var/log/ceph/ceph-client.admin.44514.log 2024-05-11T08:07:48.421 INFO:teuthology.orchestra.run.smithi167.stderr:gzip 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49866.log.gz 2024-05-11T08:07:48.421 INFO:teuthology.orchestra.run.smithi167.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.43522.log 2024-05-11T08:07:48.421 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.48288.log: /var/log/ceph/ceph-client.admin.44514.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41573.log 2024-05-11T08:07:48.421 INFO:teuthology.orchestra.run.smithi167.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.44514.log.gz/var/log/ceph/ceph-client.admin.43522.log: 2024-05-11T08:07:48.421 INFO:teuthology.orchestra.run.smithi167.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48288.log.gz 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43522.log.gz 2024-05-11T08:07:48.421 INFO:teuthology.orchestra.run.smithi167.stderr: 2024-05-11T08:07:48.421 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43857.log 2024-05-11T08:07:48.421 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.41573.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47908.log 2024-05-11T08:07:48.421 INFO:teuthology.orchestra.run.smithi167.stderr: 88.1%/var/log/ceph/ceph-client.admin.43857.log: -- replaced with /var/log/ceph/ceph-client.admin.41573.log.gz 2024-05-11T08:07:48.421 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51190.log 2024-05-11T08:07:48.422 INFO:teuthology.orchestra.run.smithi167.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.43857.log.gz 2024-05-11T08:07:48.422 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.47908.log: /var/log/ceph/ceph-client.admin.41531.log 2024-05-11T08:07:48.422 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.51190.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.47908.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51064.log 2024-05-11T08:07:48.422 INFO:teuthology.orchestra.run.smithi167.stderr: 2024-05-11T08:07:48.422 INFO:teuthology.orchestra.run.smithi167.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51190.log.gz 2024-05-11T08:07:48.422 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47392.log 2024-05-11T08:07:48.422 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.41531.log: /var/log/ceph/ceph-client.admin.51064.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50518.log 2024-05-11T08:07:48.422 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.47392.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41531.log.gz 2024-05-11T08:07:48.422 INFO:teuthology.orchestra.run.smithi167.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.51064.log.gz 2024-05-11T08:07:48.422 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42670.log 2024-05-11T08:07:48.422 INFO:teuthology.orchestra.run.smithi167.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.47392.log.gz 2024-05-11T08:07:48.422 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.50518.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41741.log 2024-05-11T08:07:48.422 INFO:teuthology.orchestra.run.smithi167.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.50518.log.gz 2024-05-11T08:07:48.423 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.42670.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49216.log 2024-05-11T08:07:48.423 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.41741.log: gzip 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42670.log.gz 2024-05-11T08:07:48.423 INFO:teuthology.orchestra.run.smithi167.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.35590.log 2024-05-11T08:07:48.423 INFO:teuthology.orchestra.run.smithi167.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.41741.log.gz 2024-05-11T08:07:48.423 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.2.45305.log 2024-05-11T08:07:48.423 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.49216.log: /var/log/ceph/ceph-client.admin.35590.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51316.log 2024-05-11T08:07:48.423 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.2.45305.log: 89.1% 89.6% -- replaced with /var/log/ceph/ceph-client.admin.49216.log.gz 2024-05-11T08:07:48.423 INFO:teuthology.orchestra.run.smithi167.stderr: -- replaced with /var/log/ceph/ceph-client.admin.35590.log.gz 0.0% -- replaced with /var/log/ceph/ceph-client.2.45305.log.gz 2024-05-11T08:07:48.423 INFO:teuthology.orchestra.run.smithi167.stderr: 2024-05-11T08:07:48.423 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43181.log 2024-05-11T08:07:48.423 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49398.log 2024-05-11T08:07:48.423 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.51316.log: gzip/var/log/ceph/ceph-client.admin.43181.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.46926.log 2024-05-11T08:07:48.423 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.49398.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43181.log.gz 2024-05-11T08:07:48.423 INFO:teuthology.orchestra.run.smithi167.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.51316.log.gz 2024-05-11T08:07:48.424 INFO:teuthology.orchestra.run.smithi167.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.49398.log.gz 2024-05-11T08:07:48.424 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-05-11T08:07:48.424 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.46926.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47308.log 2024-05-11T08:07:48.424 INFO:teuthology.orchestra.run.smithi167.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46926.log.gz 2024-05-11T08:07:48.424 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43508.log 2024-05-11T08:07:48.424 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48699.log 2024-05-11T08:07:48.424 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.47308.log: /var/log/ceph/ceph-client.admin.43508.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.47308.log.gz 2024-05-11T08:07:48.424 INFO:teuthology.orchestra.run.smithi167.stderr: 87.9%gzip -- replaced with /var/log/ceph/ceph-client.admin.43508.log.gz -5 --verbose -- /var/log/ceph/ceph-client.admin.51148.log 2024-05-11T08:07:48.424 INFO:teuthology.orchestra.run.smithi167.stderr: 2024-05-11T08:07:48.424 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.48096.log 2024-05-11T08:07:48.424 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.48699.log: /var/log/ceph/ceph-client.admin.51148.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.51148.log.gz 2024-05-11T08:07:48.424 INFO:teuthology.orchestra.run.smithi167.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.48699.log.gz 2024-05-11T08:07:48.424 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48828.log 2024-05-11T08:07:48.424 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.0.48096.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.48096.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51232.log 2024-05-11T08:07:48.425 INFO:teuthology.orchestra.run.smithi167.stderr: 2024-05-11T08:07:48.425 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43183.log 2024-05-11T08:07:48.425 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.48828.log: /var/log/ceph/ceph-client.admin.51232.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.51232.log.gz 2024-05-11T08:07:48.425 INFO:teuthology.orchestra.run.smithi167.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.48828.log.gz 2024-05-11T08:07:48.425 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46302.log 2024-05-11T08:07:48.425 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45889.log 2024-05-11T08:07:48.425 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.43183.log: /var/log/ceph/ceph-client.admin.46302.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43183.log.gz 2024-05-11T08:07:48.425 INFO:teuthology.orchestra.run.smithi167.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46302.log.gz 2024-05-11T08:07:48.425 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48805.log 2024-05-11T08:07:48.425 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42838.log 2024-05-11T08:07:48.425 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.48805.log: /var/log/ceph/ceph-client.admin.45889.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.45889.log.gz 2024-05-11T08:07:48.425 INFO:teuthology.orchestra.run.smithi167.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48805.log.gz 2024-05-11T08:07:48.425 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48367.log 2024-05-11T08:07:48.425 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.44838.log 2024-05-11T08:07:48.425 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.42838.log: /var/log/ceph/ceph-client.admin.48367.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.42838.log.gz 2024-05-11T08:07:48.426 INFO:teuthology.orchestra.run.smithi167.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.48367.log.gz 2024-05-11T08:07:48.426 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50938.log 2024-05-11T08:07:48.426 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43858.log 2024-05-11T08:07:48.426 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.50938.log: /var/log/ceph/ceph-client.0.44838.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.44838.log.gz 2024-05-11T08:07:48.426 INFO:teuthology.orchestra.run.smithi167.stderr: 87.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46554.log 2024-05-11T08:07:48.426 INFO:teuthology.orchestra.run.smithi167.stderr: -- replaced with /var/log/ceph/ceph-client.admin.50938.log.gz 2024-05-11T08:07:48.426 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46470.log 2024-05-11T08:07:48.426 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.43858.log: /var/log/ceph/ceph-client.admin.46554.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43858.log.gz 2024-05-11T08:07:48.426 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42964.log 2024-05-11T08:07:48.426 INFO:teuthology.orchestra.run.smithi167.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46554.log.gz 2024-05-11T08:07:48.426 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.46470.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49782.log 2024-05-11T08:07:48.426 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.42964.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.46470.log.gz 2024-05-11T08:07:48.426 INFO:teuthology.orchestra.run.smithi167.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.42964.log.gz 2024-05-11T08:07:48.427 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51106.log 2024-05-11T08:07:48.427 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50602.log 2024-05-11T08:07:48.427 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.49782.log: /var/log/ceph/ceph-client.admin.51106.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.49782.log.gz 2024-05-11T08:07:48.427 INFO:teuthology.orchestra.run.smithi167.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.51106.log.gzgzip 2024-05-11T08:07:48.427 INFO:teuthology.orchestra.run.smithi167.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.45484.log 2024-05-11T08:07:48.427 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51470.log 2024-05-11T08:07:48.427 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.45484.log: /var/log/ceph/ceph-client.admin.50602.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.50602.log.gz 2024-05-11T08:07:48.427 INFO:teuthology.orchestra.run.smithi167.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45484.log.gz 2024-05-11T08:07:48.427 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43739.log 2024-05-11T08:07:48.427 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46968.log 2024-05-11T08:07:48.427 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.51470.log: /var/log/ceph/ceph-client.admin.43739.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.51470.log.gz 2024-05-11T08:07:48.427 INFO:teuthology.orchestra.run.smithi167.stderr: 88.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49824.log 2024-05-11T08:07:48.427 INFO:teuthology.orchestra.run.smithi167.stderr: -- replaced with /var/log/ceph/ceph-client.admin.43739.log.gz 2024-05-11T08:07:48.427 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46884.log 2024-05-11T08:07:48.428 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.46968.log: /var/log/ceph/ceph-client.admin.49824.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46968.log.gz 2024-05-11T08:07:48.428 INFO:teuthology.orchestra.run.smithi167.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.49824.log.gz 2024-05-11T08:07:48.428 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47518.log 2024-05-11T08:07:48.428 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41213.log 2024-05-11T08:07:48.428 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.46884.log: /var/log/ceph/ceph-client.admin.47518.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46884.log.gz 2024-05-11T08:07:48.428 INFO:teuthology.orchestra.run.smithi167.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47518.log.gz 2024-05-11T08:07:48.428 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41699.log 2024-05-11T08:07:48.428 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41849.log 2024-05-11T08:07:48.428 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.41213.log: /var/log/ceph/ceph-client.admin.41699.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.41213.log.gz 2024-05-11T08:07:48.428 INFO:teuthology.orchestra.run.smithi167.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.41699.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43740.log 2024-05-11T08:07:48.428 INFO:teuthology.orchestra.run.smithi167.stderr: 2024-05-11T08:07:48.428 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49908.log 2024-05-11T08:07:48.428 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.41849.log: /var/log/ceph/ceph-client.admin.43740.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.41849.log.gz 2024-05-11T08:07:48.428 INFO:teuthology.orchestra.run.smithi167.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43740.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45568.log 2024-05-11T08:07:48.429 INFO:teuthology.orchestra.run.smithi167.stderr: 2024-05-11T08:07:48.429 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41783.log 2024-05-11T08:07:48.429 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.45568.log: /var/log/ceph/ceph-client.admin.49908.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.49908.log.gz 2024-05-11T08:07:48.429 INFO:teuthology.orchestra.run.smithi167.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.45568.log.gz 2024-05-11T08:07:48.429 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49563.log 2024-05-11T08:07:48.429 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43856.log 2024-05-11T08:07:48.429 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.41783.log: /var/log/ceph/ceph-client.admin.49563.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.49563.log.gz 2024-05-11T08:07:48.429 INFO:teuthology.orchestra.run.smithi167.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.41783.log.gz 2024-05-11T08:07:48.429 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44182.log 2024-05-11T08:07:48.429 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50244.log 2024-05-11T08:07:48.429 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.43856.log: /var/log/ceph/ceph-client.admin.44182.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.43856.log.gz 2024-05-11T08:07:48.429 INFO:teuthology.orchestra.run.smithi167.stderr: 89.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40901.log 2024-05-11T08:07:48.429 INFO:teuthology.orchestra.run.smithi167.stderr: -- replaced with /var/log/ceph/ceph-client.admin.44182.log.gz 2024-05-11T08:07:48.429 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50328.log 2024-05-11T08:07:48.430 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.50244.log: /var/log/ceph/ceph-client.admin.40901.log: 88.7% 89.0% -- replaced with /var/log/ceph/ceph-client.admin.40901.log.gz 2024-05-11T08:07:48.430 INFO:teuthology.orchestra.run.smithi167.stderr: -- replaced with /var/log/ceph/ceph-client.admin.50244.log.gz 2024-05-11T08:07:48.430 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50770.log 2024-05-11T08:07:48.430 INFO:teuthology.orchestra.run.smithi167.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-05-11T08:07:48.430 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.50328.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50644.log 2024-05-11T08:07:48.430 INFO:teuthology.orchestra.run.smithi167.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50328.log.gz 2024-05-11T08:07:48.430 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.2.45309.log 2024-05-11T08:07:48.430 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.50770.log: /var/log/ceph/ceph-client.admin.50644.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49950.log 2024-05-11T08:07:48.430 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.2.45309.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.50770.log.gz 2024-05-11T08:07:48.430 INFO:teuthology.orchestra.run.smithi167.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.50644.log.gz 0.0% -- replaced with /var/log/ceph/ceph-client.2.45309.log.gz 2024-05-11T08:07:48.430 INFO:teuthology.orchestra.run.smithi167.stderr: 2024-05-11T08:07:48.430 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49087.log 2024-05-11T08:07:48.430 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42796.log 2024-05-11T08:07:48.430 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.49950.log: /var/log/ceph/ceph-client.admin.49087.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47799.log 2024-05-11T08:07:48.431 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.42796.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49950.log.gz 2024-05-11T08:07:48.431 INFO:teuthology.orchestra.run.smithi167.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.49087.log.gz 2024-05-11T08:07:48.431 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43860.log 2024-05-11T08:07:48.431 INFO:teuthology.orchestra.run.smithi167.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.42796.log.gz 2024-05-11T08:07:48.431 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51512.log 2024-05-11T08:07:48.431 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.47799.log: /var/log/ceph/ceph-client.admin.43860.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45778.log 2024-05-11T08:07:48.431 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.51512.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.47799.log.gz 2024-05-11T08:07:48.431 INFO:teuthology.orchestra.run.smithi167.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43860.log.gz 2024-05-11T08:07:48.431 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46344.log 2024-05-11T08:07:48.431 INFO:teuthology.orchestra.run.smithi167.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.51512.log.gz 2024-05-11T08:07:48.431 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43523.log 2024-05-11T08:07:48.431 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.45778.log: /var/log/ceph/ceph-client.admin.46344.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50812.log 2024-05-11T08:07:48.431 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.43523.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45778.log.gz 2024-05-11T08:07:48.432 INFO:teuthology.orchestra.run.smithi167.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.46344.log.gz 2024-05-11T08:07:48.432 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50686.log 2024-05-11T08:07:48.432 INFO:teuthology.orchestra.run.smithi167.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43523.log.gz 2024-05-11T08:07:48.432 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42754.log 2024-05-11T08:07:48.432 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.50812.log: /var/log/ceph/ceph-client.admin.50686.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46260.log 2024-05-11T08:07:48.432 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.42754.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.50812.log.gz 2024-05-11T08:07:48.432 INFO:teuthology.orchestra.run.smithi167.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50686.log.gz 2024-05-11T08:07:48.432 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42628.log 2024-05-11T08:07:48.432 INFO:teuthology.orchestra.run.smithi167.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.42754.log.gz 2024-05-11T08:07:48.432 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41385.log 2024-05-11T08:07:48.432 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.46260.log: /var/log/ceph/ceph-client.admin.42628.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44972.log 2024-05-11T08:07:48.432 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.41385.log: 87.9% 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46260.log.gz 2024-05-11T08:07:48.432 INFO:teuthology.orchestra.run.smithi167.stderr: -- replaced with /var/log/ceph/ceph-client.admin.42628.log.gz 2024-05-11T08:07:48.432 INFO:teuthology.orchestra.run.smithi167.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.41385.log.gz 2024-05-11T08:07:48.432 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45526.log 2024-05-11T08:07:48.433 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49322.log 2024-05-11T08:07:48.433 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.44972.log: /var/log/ceph/ceph-client.admin.45526.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50560.log 2024-05-11T08:07:48.433 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.49322.log: 89.8% 89.1% -- replaced with /var/log/ceph/ceph-client.admin.44972.log.gz 2024-05-11T08:07:48.433 INFO:teuthology.orchestra.run.smithi167.stderr: -- replaced with /var/log/ceph/ceph-client.admin.45526.log.gz 2024-05-11T08:07:48.433 INFO:teuthology.orchestra.run.smithi167.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49322.log.gz 2024-05-11T08:07:48.433 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44056.log 2024-05-11T08:07:48.433 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49028.log 2024-05-11T08:07:48.433 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.50560.log: /var/log/ceph/ceph-client.admin.44056.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50286.log 2024-05-11T08:07:48.433 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.49028.log: 88.5% 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50560.log.gz 2024-05-11T08:07:48.433 INFO:teuthology.orchestra.run.smithi167.stderr: -- replaced with /var/log/ceph/ceph-client.admin.44056.log.gz 2024-05-11T08:07:48.433 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46841.log 2024-05-11T08:07:48.433 INFO:teuthology.orchestra.run.smithi167.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.49028.log.gz 2024-05-11T08:07:48.433 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46218.log 2024-05-11T08:07:48.433 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.50286.log: /var/log/ceph/ceph-client.admin.46841.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-05-11T08:07:48.434 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.46218.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50286.log.gz 2024-05-11T08:07:48.434 INFO:teuthology.orchestra.run.smithi167.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46841.log.gz 2024-05-11T08:07:48.434 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41171.log 2024-05-11T08:07:48.434 INFO:teuthology.orchestra.run.smithi167.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46218.log.gz 2024-05-11T08:07:48.434 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50854.log 2024-05-11T08:07:48.434 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-mds.a.log: /var/log/ceph/ceph-client.admin.41171.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46386.log 2024-05-11T08:07:48.434 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.50854.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.41171.log.gz 2024-05-11T08:07:48.434 INFO:teuthology.orchestra.run.smithi167.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.50854.log.gz 2024-05-11T08:07:48.434 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49345.log 2024-05-11T08:07:48.434 INFO:teuthology.orchestra.run.smithi167.stderr: 91.8% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-05-11T08:07:48.434 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.46386.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43185.log 2024-05-11T08:07:48.434 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.49345.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46386.log.gz 2024-05-11T08:07:48.434 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47266.log 2024-05-11T08:07:48.434 INFO:teuthology.orchestra.run.smithi167.stderr: 90.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45610.log 2024-05-11T08:07:48.435 INFO:teuthology.orchestra.run.smithi167.stderr: -- replaced with /var/log/ceph/ceph-client.admin.49345.log.gz 2024-05-11T08:07:48.435 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.43185.log: /var/log/ceph/ceph-client.admin.47266.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50118.log 2024-05-11T08:07:48.435 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.45610.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43185.log.gz 2024-05-11T08:07:48.435 INFO:teuthology.orchestra.run.smithi167.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.47266.log.gz 2024-05-11T08:07:48.435 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45694.log 2024-05-11T08:07:48.435 INFO:teuthology.orchestra.run.smithi167.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45610.log.gz 2024-05-11T08:07:48.435 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49110.log 2024-05-11T08:07:48.435 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.50118.log: /var/log/ceph/ceph-client.admin.45694.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35856.log 2024-05-11T08:07:48.435 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.49110.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45694.log.gz 2024-05-11T08:07:48.435 INFO:teuthology.orchestra.run.smithi167.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.50118.log.gz 2024-05-11T08:07:48.435 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49510.log 89.9% -- replaced with /var/log/ceph/ceph-client.admin.49110.log.gz 2024-05-11T08:07:48.435 INFO:teuthology.orchestra.run.smithi167.stderr: 2024-05-11T08:07:48.435 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.35856.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46176.log 2024-05-11T08:07:48.435 INFO:teuthology.orchestra.run.smithi167.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.35856.log.gz 2024-05-11T08:07:48.436 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.49510.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49688.log 2024-05-11T08:07:48.436 INFO:teuthology.orchestra.run.smithi167.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.49510.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48477.log 2024-05-11T08:07:48.436 INFO:teuthology.orchestra.run.smithi167.stderr: 2024-05-11T08:07:48.436 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.46176.log: /var/log/ceph/ceph-client.admin.49688.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47842.log 2024-05-11T08:07:48.436 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.48477.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.46176.log.gz 2024-05-11T08:07:48.436 INFO:teuthology.orchestra.run.smithi167.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49688.log.gz 2024-05-11T08:07:48.436 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42259.log 2024-05-11T08:07:48.436 INFO:teuthology.orchestra.run.smithi167.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.48477.log.gz 2024-05-11T08:07:48.436 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47644.log 2024-05-11T08:07:48.436 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.47842.log: /var/log/ceph/ceph-client.admin.42259.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45736.log 2024-05-11T08:07:48.436 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.47644.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47644.log.gz 2024-05-11T08:07:48.436 INFO:teuthology.orchestra.run.smithi167.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.47842.log.gz 2024-05-11T08:07:48.436 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46428.log 2024-05-11T08:07:48.436 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46757.log 2024-05-11T08:07:48.437 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.45736.log: 92.4%/var/log/ceph/ceph-client.admin.46428.log: -- replaced with /var/log/ceph/ceph-client.admin.42259.log.gz 2024-05-11T08:07:48.437 INFO:teuthology.orchestra.run.smithi167.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45736.log.gz 2024-05-11T08:07:48.437 INFO:teuthology.orchestra.run.smithi167.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46428.log.gz 2024-05-11T08:07:48.437 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50034.log 2024-05-11T08:07:48.437 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.46757.log: /var/log/ceph/ceph-client.admin.50034.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.46757.log.gz 2024-05-11T08:07:48.437 INFO:teuthology.orchestra.run.smithi167.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.50034.log.gz 2024-05-11T08:07:48.437 INFO:teuthology.orchestra.run.smithi204.stderr: 91.7% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-05-11T08:07:48.439 INFO:teuthology.orchestra.run.smithi204.stderr: 91.1% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-05-11T08:07:48.451 INFO:teuthology.orchestra.run.smithi204.stderr: 92.2% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-05-11T08:07:48.593 INFO:teuthology.orchestra.run.smithi167.stderr: 91.0% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-05-11T08:07:48.953 INFO:teuthology.orchestra.run.smithi167.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-05-11T08:07:48.969 INFO:teuthology.orchestra.run.smithi167.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-05-11T08:07:48.981 INFO:teuthology.orchestra.run.smithi204.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-05-11T08:07:48.987 INFO:teuthology.orchestra.run.smithi167.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-05-11T08:07:49.007 INFO:teuthology.orchestra.run.smithi204.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-05-11T08:07:49.020 INFO:teuthology.orchestra.run.smithi204.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-05-11T08:07:49.057 INFO:teuthology.orchestra.run.smithi204.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-05-11T08:07:49.059 INFO:teuthology.orchestra.run.smithi204.stderr: 2024-05-11T08:07:49.059 INFO:teuthology.orchestra.run.smithi204.stderr:real 0m0.727s 2024-05-11T08:07:49.059 INFO:teuthology.orchestra.run.smithi204.stderr:user 0m2.881s 2024-05-11T08:07:49.059 INFO:teuthology.orchestra.run.smithi204.stderr:sys 0m0.129s 2024-05-11T08:07:49.082 INFO:teuthology.orchestra.run.smithi167.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-05-11T08:07:49.084 INFO:teuthology.orchestra.run.smithi167.stderr: 2024-05-11T08:07:49.084 INFO:teuthology.orchestra.run.smithi167.stderr:real 0m0.783s 2024-05-11T08:07:49.085 INFO:teuthology.orchestra.run.smithi167.stderr:user 0m2.950s 2024-05-11T08:07:49.085 INFO:teuthology.orchestra.run.smithi167.stderr:sys 0m0.238s 2024-05-11T08:07:49.085 INFO:tasks.ceph:Archiving logs... 2024-05-11T08:07:49.085 DEBUG:teuthology.misc:Transferring archived files from smithi167:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-05-11_05:57:22-fs-wip-rishabh-testing-20240510.053205-testing-default-smithi/7702178/remote/smithi167/log 2024-05-11T08:07:49.086 DEBUG:teuthology.orchestra.run.smithi167:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-11T08:07:49.705 DEBUG:teuthology.misc:Transferring archived files from smithi204:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-05-11_05:57:22-fs-wip-rishabh-testing-20240510.053205-testing-default-smithi/7702178/remote/smithi204/log 2024-05-11T08:07:49.706 DEBUG:teuthology.orchestra.run.smithi204:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-11T08:07:50.128 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-05-11T08:07:50.138 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-05-11T08:07:50.138 DEBUG:teuthology.orchestra.run.smithi167:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-11T08:07:50.141 DEBUG:teuthology.orchestra.run.smithi204:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-11T08:07:50.223 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-05-11T08:07:50.234 INFO:teuthology.task.clock:Checking final clock skew... 2024-05-11T08:07:50.235 DEBUG:teuthology.orchestra.run.smithi167:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-11T08:07:50.237 DEBUG:teuthology.orchestra.run.smithi204:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-11T08:07:50.250 INFO:teuthology.orchestra.run.smithi167.stderr:bash: line 1: ntpq: command not found 2024-05-11T08:07:50.252 INFO:teuthology.orchestra.run.smithi204.stderr:bash: line 1: ntpq: command not found 2024-05-11T08:07:50.255 INFO:teuthology.orchestra.run.smithi167.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-11T08:07:50.255 INFO:teuthology.orchestra.run.smithi167.stdout:=============================================================================== 2024-05-11T08:07:50.255 INFO:teuthology.orchestra.run.smithi167.stdout:^* hv01.front.sepia.ceph.com 3 6 377 38 +782us[ +880us] +/- 38ms 2024-05-11T08:07:50.255 INFO:teuthology.orchestra.run.smithi167.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 38 -955us[ -857us] +/- 80ms 2024-05-11T08:07:50.255 INFO:teuthology.orchestra.run.smithi167.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 36 -439us[ -439us] +/- 64ms 2024-05-11T08:07:50.255 INFO:teuthology.orchestra.run.smithi167.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-05-11T08:07:50.257 INFO:teuthology.orchestra.run.smithi204.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-11T08:07:50.257 INFO:teuthology.orchestra.run.smithi204.stdout:=============================================================================== 2024-05-11T08:07:50.257 INFO:teuthology.orchestra.run.smithi204.stdout:^* hv01.front.sepia.ceph.com 3 6 377 36 +758us[ +854us] +/- 38ms 2024-05-11T08:07:50.257 INFO:teuthology.orchestra.run.smithi204.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 36 -952us[ -856us] +/- 80ms 2024-05-11T08:07:50.258 INFO:teuthology.orchestra.run.smithi204.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 36 -434us[ -338us] +/- 64ms 2024-05-11T08:07:50.258 INFO:teuthology.orchestra.run.smithi204.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-05-11T08:07:50.258 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-05-11T08:07:50.294 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-05-11T08:07:50.295 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-05-11T08:07:50.352 DEBUG:teuthology.orchestra.run.smithi167:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-11T08:07:50.420 DEBUG:teuthology.orchestra.run.smithi204:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-11T08:07:50.482 DEBUG:teuthology.orchestra.run.smithi167:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-11T08:07:50.510 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-11T08:07:50.511 DEBUG:teuthology.orchestra.run.smithi204:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-11T08:07:50.538 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-11T08:07:50.539 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-05-11T08:07:50.548 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-05-11T08:07:50.602 INFO:teuthology.task.internal:Duration was 545.671626 seconds 2024-05-11T08:07:50.603 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-05-11T08:07:50.678 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-05-11T08:07:50.678 DEBUG:teuthology.orchestra.run.smithi167:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-11T08:07:50.680 DEBUG:teuthology.orchestra.run.smithi204:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-11T08:07:50.717 INFO:teuthology.orchestra.run.smithi167.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-11T08:07:50.719 INFO:teuthology.orchestra.run.smithi204.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-11T08:07:51.167 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-05-11T08:07:51.167 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi167.front.sepia.ceph.com 2024-05-11T08:07:51.167 DEBUG:teuthology.orchestra.run.smithi167:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-11T08:07:51.188 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi204.front.sepia.ceph.com 2024-05-11T08:07:51.188 DEBUG:teuthology.orchestra.run.smithi204:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-11T08:07:51.230 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-05-11T08:07:51.230 DEBUG:teuthology.orchestra.run.smithi167:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-11T08:07:51.233 DEBUG:teuthology.orchestra.run.smithi204:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-11T08:07:51.298 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-05-11T08:07:51.298 DEBUG:teuthology.orchestra.run.smithi167:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-11T08:07:51.301 DEBUG:teuthology.orchestra.run.smithi204:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-11T08:07:51.434 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-05-11T08:07:51.475 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-05-11T08:07:51.475 DEBUG:teuthology.orchestra.run.smithi167:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-11T08:07:51.501 DEBUG:teuthology.orchestra.run.smithi204:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-11T08:07:51.527 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-05-11T08:07:51.557 DEBUG:teuthology.orchestra.run.smithi167:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-11T08:07:51.561 DEBUG:teuthology.orchestra.run.smithi204:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-11T08:07:51.582 INFO:teuthology.orchestra.run.smithi167.stdout:kernel.core_pattern = core 2024-05-11T08:07:51.600 INFO:teuthology.orchestra.run.smithi204.stdout:kernel.core_pattern = core 2024-05-11T08:07:51.614 DEBUG:teuthology.orchestra.run.smithi167:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-11T08:07:51.656 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-11T08:07:51.657 DEBUG:teuthology.orchestra.run.smithi204:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-11T08:07:51.673 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-11T08:07:51.674 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-05-11T08:07:51.704 INFO:teuthology.task.internal:Transferring archived files... 2024-05-11T08:07:51.704 DEBUG:teuthology.misc:Transferring archived files from smithi167:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-05-11_05:57:22-fs-wip-rishabh-testing-20240510.053205-testing-default-smithi/7702178/remote/smithi167 2024-05-11T08:07:51.705 DEBUG:teuthology.orchestra.run.smithi167:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-11T08:07:51.800 DEBUG:teuthology.misc:Transferring archived files from smithi204:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-05-11_05:57:22-fs-wip-rishabh-testing-20240510.053205-testing-default-smithi/7702178/remote/smithi204 2024-05-11T08:07:51.800 DEBUG:teuthology.orchestra.run.smithi204:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-11T08:07:51.882 INFO:teuthology.task.internal:Removing archive directory... 2024-05-11T08:07:51.882 DEBUG:teuthology.orchestra.run.smithi167:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-11T08:07:51.885 DEBUG:teuthology.orchestra.run.smithi204:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-11T08:07:51.902 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-05-11T08:07:51.919 INFO:teuthology.task.internal:Not uploading archives. 2024-05-11T08:07:51.919 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-05-11T08:07:52.009 INFO:teuthology.task.internal:Tidying up after the test... 2024-05-11T08:07:52.009 DEBUG:teuthology.orchestra.run.smithi167:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-11T08:07:52.012 DEBUG:teuthology.orchestra.run.smithi204:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-11T08:07:52.024 INFO:teuthology.orchestra.run.smithi167.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 11 08:07 /home/ubuntu/cephtest 2024-05-11T08:07:52.031 INFO:teuthology.orchestra.run.smithi204.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 11 08:07 /home/ubuntu/cephtest 2024-05-11T08:07:52.033 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-05-11T08:07:52.100 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-05-11T08:07:52.224 INFO:teuthology.run:Summary data: description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/backtrace} duration: 545.6716256141663 flavor: default owner: scheduled_rishabh@teuthology success: true 2024-05-11T08:07:52.224 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-11T08:07:52.328 INFO:teuthology.run:pass